Dev tools · alternatives · 2026
24 alternatives to Dlt – Python library to automate the creation of datasets
Hi HN, We're Anna, Adrian, Marcin and Matt, developers of dlt. dlt is an open source library to automatically create datasets out of messy, unstructured data sources. You can use the library to move data from about anywhere into most of well known SQL and vector stores, data lakes, storage buckets, or local engines like DuckDB. It automates many cumbersome data engineering tasks and can by handled by anyone who knows Python. Here’s our Github: https://github.com/dlt-hub/dlt Here’s our Colab demo:…
Dlt is an open-source Python library that automates the creation of structured datasets from messy, unstructured data sources. It handles data movement from various origins into SQL databases, vector stores, data… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Dlt – Python library to automate the creation of datasets launched in 2023; newer entries below may have overtaken it.
- 1
Hi Show HN, we are Dave, Marcin, Alena, and Adrian, authors of data load tool (dlt), a Python library that automatically creates datasets from any kind of messy, unstructured data. We launched dlt on HN 7 months ago with a mission to make getting datasets fast and easy. Now dlt helps its users to code around a thousand new data sources each month and to maintain many thousands of live datasets in production. Today we are releasing *dlt-init-openapi,* a Python CLI tool that generates a dlt data pipeline from any OpenAPI spec. It brings the time to create a dataset down to a few minutes.…
2024 · colab.research.google.com · its alternatives →
- 2DA
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
2020 · its alternatives →
- 3
Long story short: We (Dataherald) just open-sourced our entire codebase, including the core engine, the clients that interact with it and the backend application layer for authentication and RBAC. You can now use the full solution to build text-to-SQL into your product. The Problem: modern LLMs write syntactically correct SQL, but they struggle with real-world relational data. This is because real world data and schema is messy, natural language can often be ambiguous and LLMs are not trained on your specific dataset. Solution: The core NL-to-SQL engine in Dataherald is an LLM based agent…
2024 · github.com · its alternatives →
- 4
I wrote a relational database management system (RDBMS) (sqlite clone) from scratch in pure Python.
2023 · github.com · its alternatives →
- 5
- 6
Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
2024 · github.com · its alternatives →
- 7
- 8
Orchest▲112An open source tool for creating data science pipelines
2020 · orchest.io · its alternatives →
- 9
Python library for creating data pipelines with chain functional programming - EntilZha/PyFunctional
2017 · github.com · its alternatives →
- 10
Hi HN community. We are excited to open source Dataherald’s natural-language-to-SQL engine today (https://github.com/Dataherald/dataherald). This engine allows you to set up an API from your structured database that can answer questions in plain English. GPT-4 class LLMs have gotten remarkably good at writing SQL. However, out-of-the-box LLMs and existing frameworks would not work with our own structured data at a necessary quality level. For example, given the question “what was the average rent in Los Angeles in May 2023?” a reasonable human would either assume the…
2023 · github.com · its alternatives →
- 11
- 12AT
A bunch of developers and myself have created RepliByte - an open-source tool to seed a development database from a production database. Features: - Support data backup and restore for PostgreSQL, MySQL and MongoDB - Replace sensitive data with fake data - Works on large database (> 10GB) (read Design) - Database Subsetting: Scale down a production database to a more reasonable size - Start a local database with the prod data in a single command - On-the-fly data (de)compression (Zlib) - On-the-fly data de/encryption (AES-256) - Fully stateless (no server, no daemon) and lightweight…
2022 · its alternatives →
- 13
Hi HN! I'm excited to share Autolabel, an open-source Python library to label and enrich text datasets with any Large Language Model (LLM) of your choice. We built Autolabel because access to clean, labeled data is a huge bottleneck for most ML/data science teams. The most capable LLMs are able to label data with high accuracy, and at a fraction of the cost and time compared to manual labeling. With Autolabel, you can leverage LLMs to label any text dataset with <5 lines of code. We’re eager for your feedback!
2023 · github.com · its alternatives →
- 14DD
Gleb, Alex, Erez and Simon here – we are building an open-source tool for comparing data within and across databases at any scale. The repo is at https://github.com/datafold/data-diff, and our home page is https://datafold.com/. As a company, Datafold builds tools for data engineers to automate the most tedious and error-prone tasks falling through the cracks of the modern data stack, such as data testing and lineage. We launched two years ago with a tool for regression-testing changes to ETL code…
2022 · its alternatives →
- 15OD
Hello Hacker News! We are Rick & Yannick from Orchest (https://www.orchest.io - https://github.com/orchest/orchest). We're building a visual pipeline tool for data scientists. The tool can be considered to be high-code because you write your own Python/R notebooks and scripts, but we manage the underlying infrastructure to make it 'just work™'. You can think of it as a simplified version of Kubeflow. We created Orchest to free data scientists from the tedious engineering related tasks of their job. Similar to how companies like Netflix, Uber and Booking.com…
2020 · its alternatives →
- 16SP
Hi HN, Over the past 6 months I've been working on a technical book focused on helping aspiring data scientists to get hands-on experience with cloud computing environments using the Python ecosystem. The book is targeted at readers already familiar with libraries such as Pandas and scikit-learn that are looking to build out a portfolio of applied projects. To author the book, I used the Leanpub platform to provide drafts of the text as I completed each chapter. To typeset the book, I used the R bookdown package by Yihui Xie to translate my markdown into a PDF format. I also used Google docs…
2020 · its alternatives →
- 17SA
Hi HN! I’m Eoin, founder of Sourcetable (https://sourcetable.com). Sourcetable is an AI-native spreadsheet that syncs with all your data. Users pair with an AI copilot that helps them do their spreadsheet work, as well as more database-centric analysis and SQL. Soucetable syncs with databases including Postgres, MySQL, and MongoDB, and over 100+ business applications including Stripe, Zendesk, Hubspot, Quickbooks and Google Analytics. That data is available in a spreadsheet, and any models you build automatically update in near-real-time as new data flows in. The core primitives…
2024 · its alternatives →
- 18
Today I'm open sourcing "Grid studio", a web-based spreadsheet application with full integration of the Python programming language.
2019 · hackernoon.com · its alternatives →
- 19
plait.py - a fake data modeler . Contribute to plaitpy/plaitpy development by creating an account on GitHub.
2018 · github.com · its alternatives →
- 20
- 21
- 22
A container for dataclasses with multi-indexing and bulk operations. - joshlk/dataclassframe
2020 · github.com · its alternatives →
- 23
A grid library for instant big data processing
2022 · datatable.dev · its alternatives →
- 24
Also compare
- Automatically extract data from APIs with dlt and OpenAPI alternatives
- dstack – an open-source tool to build data applications easily alternatives
- We open sourced our entire text-to-SQL product alternatives
- I wrote a RDBMS (SQLite clone) from scratch in pure Python alternatives
- panda{·}etl alternatives
- I built an open-source data pipeline tool in Go alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →