Automatically extract data from APIs with dlt and OpenAPI
Hi Show HN, we are Dave, Marcin, Alena, and Adrian, authors of data load tool (dlt), a Python library that automatically creates datasets from any kind of messy, unstructured data. We launched dlt on HN 7 months ago with a mission to make getting datasets fast and easy. Now dlt helps its users to code around a thousand new data sources each month and to maintain many thousands of live datasets in production. Today we are releasing *dlt-init-openapi,* a Python CLI tool that generates a dlt data pipeline from any OpenAPI spec. It brings the time to create a dataset down to a few minutes.…
Automatically extract data from APIs with dlt and OpenAPI launched on May 29, 2024 with 7 votes, #1,618 of 4,539 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 2 years on.
What it does
In the maker’s words, at launch
Hi Show HN, we are Dave, Marcin, Alena, and Adrian, authors of data load tool (dlt), a Python library that automatically creates datasets from any kind of messy, unstructured data. We launched dlt on HN 7 months ago with a mission to make getting datasets fast and easy. Now dlt helps its users to code around a…
Does a similar job
all alternatives →
Dlt – Python library to automate the creation of datasets2023 · colab.research.google.com · ▲114Hi HN, We're Anna, Adrian, Marcin and Matt, developers of dlt. dlt is an open source library to automatically create datasets out of messy, unstructured data sources. You can use the library to move data from about anywhere into most of well known SQL and vector stores, data lakes, storage buckets, or local engines like DuckDB. It automates many cumbersome data engineering tasks and can by handled by anyone who knows Python. Here’s our Github: https://github.com/dlt-hub/dlt Here’s our Colab demo:…
Tables by Playmaker2024 · portday.com · ▲467Transform unstructured data into actionable data, instantly
- DAdstack – an open-source tool to build data applications easily2020 · ▲134
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
Every open source tool from the "What's HN working on" thread2024 · github.com · ▲169Public datsets made with https://getomni.ai/. Contribute to getomni-ai/datasets development by creating an account on GitHub.
- AGAuto-generate load tests/synthetic test data from OpenAPI spec/HAR file2024 · docs.multiple.dev · ▲33
Hey HN, We just shipped a new AI-powered feature... BUT the "AI" piece is largely in the background. Instead of relying on a chatbot, we've integrated AI (with strict input & output guardrails) into a workflow to handle two specific tasks that would be difficult for traditional programming: 1. Identifying the most relevant base URL from HAR files, since it would be tedious to cover every edge case or scenario to omit analytics, tracking, and other network noise. 2. Generating synthetic data for API requests by passing the API context and faker-js functions to GPT-4. The steps are broken down…
Pipet – CLI tool for scraping and extracting data online, with pipes2024 · github.com · ▲49I often find myself in situations where I need to extract some data from a website, either as one time thing or periodically (and then watch for changes). Maybe I'm tracking stocks, or a delivery, or I want to know when tickets become available to the local sauna; maybe a friend called and asked "can you get me all data from that website into a spreadsheet?" (this happens surprisingly often) I used to write one-off scripts for that, often with Python or JavaScript, but I noticed I spend about one minute getting the right CSS selectors, and then another 10 minutes setting the rest of the…
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 28d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 26d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 24d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 21d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 29d ago · corespeed.io
Source: Show HN post ↗ · launch text quoted in part
