dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
Kestra is an open-source, event-driven orchestration platform that makes both scheduled and event-driven workflows easy. By bringing Infrastructure as Code best practices to data, process, and microservice orchestration, you can build reliable workflows directly from the UI in just a few lines of YAML.
Streamlit is an open source app framework in Python language. It helps us create web apps for data science and machine learning in a short time.
Flyte is an open-source, container-native, structured programming and distributed processing platform implemented in Golang. It enables highly concurrent, scalable and maintainable workflows for machine learning and data processing.
FillDB is a free tool that lets you quickly generate large volumes of custom data in MySql format to use in testing software and populating databases with random data.
Data validation and settings management using python type annotations.
Jina is a neural search framework that allows to build deep learning search applications in minutes. It provides scalable indexing, querying, understanding of any data: video, image, long/short text, music, source code, PDF, etc.
Apache Superset is a modern data exploration and visualization platform
Splitgraph is a data management, building and sharing tool inspired by Docker and Git that works on top of PostgreSQL and integrates seamlessly with anything that uses PostgreSQL.
Splitgraph allows the user to manipulate data images (snapshots of SQL tables at a given point in time) as if they were code repositories by versioning, pushing and pulling them.
Dolt is a relational database, i.e. it has tables, and you can execute SQL queries against those tables. It also has version control primitives that operate at the level of table cell. Thus Dolt is a database that supports fine grained value-wise version control, where all changes to data and schema are stored in commit log.
Context aware, pluggable and customizable data protection and PII data anonymization service for text and images
librehosters is a network of cooperation and solidarity that uses free software to encourage decentralisation through federation and distributed platforms. Our values connect transparency, fairness and privacy with a culture of data portability and public contributions to the commons.
A new way to develop parsers for binary structures.
Declarative: describe the very structure of the data, not how you read or write it
Engineering managers and maintainers of large code bases are starting to realize the potential of Code as Data or how source code can be treated as an analyzable dataset proving valuable information. Think Business Intelligence and processes optimization based on the source code engineers write, rather than adjacent metrics.
pg_chameleon is a MySQL to PostgreSQL replica system written in Python 3. The tool can connect to the mysql replication protocol and replicate the data changes in PostgreSQL. Whether the user needs to setup a permanent replica between MySQL and PostgreSQL or perform an engine migration, pg_chamaleon is the perfect tool for the job.
pgloader loads data into PostgreSQL and allows you to implement Continuous Migration from your current database to PostgreSQL.
pgLoader has two modes of operation. It can either load data from files, such as CSV or Fixed-File Format; or migrate a whole database to PostgreSQL.
pgLoader supports several RDBMS solutions as a migration source, and fetches information from the catalog tables over a connection to then create an equivalent schema in PostgreSQL. This means that you can migrate to PostgreSQL in a single command-line!
Falcor is the innovative data platform that powers the Netflix UIs. Falcor allows you to model all your backend data as a single Virtual JSON object on your Node server. On the client you work with your remote JSON object using familiar JavaScript operations like get, set, and call. If you know your data, you know your API.
Falcor is middleware. It is not a replacement for your application server, database, or MVC framework. Instead Falcor can be used to optimize communication between the layers of a new or existing application.
A collective list of free APIs for use in web development.
Dotmesh captures snapshots from stateful Docker and Kubernetes applications as ’datadots‘, and gives you the ability to label, store and share them as you would with version-controlled code. Dotmesh is an open–source primitive that can be accessed via CLI or API.
Sanity is a headless, real-time CMS where the front end is an open source React-based construction kit and the back end is a graph-oriented cloud datastore with a globally distributed CDN.