Phidata – Building Blocks for Data Engineering
Hi HN, I’ve been a data engineer for over a decade and have slowly been working on a way for 1 person to run a data platform (mostly to automate my job) When I was starting, I found the wide range of tools required to build data products overwhelming. Infrastructure setup, app deployment, pipeline development, creating metrics, dashboards and finally, integrating ML into product. No wonder companies have 100s of people working on this. So last year I quit the best job in the world to figure this out. I’d like to introduce Phidata: Building Blocks for Data Engineering It works like this: 1.…
Phidata – Building Blocks for Data Engineering launched on January 23, 2023 with 16 votes, #1,306 of 3,901 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 3 years on.
In plain words
Phidata is a framework designed to simplify data platform development by providing pre-configured building blocks for common tools like Airflow, Superset, and Jupyter. It allows data engineers to define infrastructure and applications as Python objects, build data products in Python or SQL, and deploy to AWS. The platform targets small teams or individual data engineers who want to reduce the complexity of managing multiple tools, infrastructure setup, and pipeline development traditionally requiring large teams.
written from the facts on this page · September 2026
From the sources
Build agents and workflows to automate intelligent work.
In the maker’s words, at launch
Hi HN, I’ve been a data engineer for over a decade and have slowly been working on a way for 1 person to run a data platform (mostly to automate my job) When I was starting, I found the wide range of tools required to build data products overwhelming. Infrastructure setup, app deployment, pipeline development…
Does a similar job
all alternatives →
Phidata – Build AI Agents with memory, knowledge, tools and reasoning2024 · github.com · ▲27Build, run, and manage agent platforms. Contribute to agno-agi/agno development by creating an account on GitHub.
I built an open-source data pipeline tool in Go2024 · github.com · ▲200Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
More work this month
the category →
Endless lofi beats composed live in your browser, over pixel-art nights in Paris, Tokyo, New York, Hong Kong, Sydney, San Francisco and more. Free, no sign-up.
Work · 24d ago · loficities.com
Smooth Recorder▲257Mac screen recordings that look edited when you hit stop
Work · 18d ago · smoothrecorder.com
Work · 12d ago · readrare.com
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, January 2023
the whole month →
This works suprisingly well. Just give it instructions like "make it winter" or "remove the cars" and the photo is altered. Here are some examples of transformations it can make: Golden gate bridge: https://raw.githubusercontent.com/brycedrennan/imaginAIry/ma... Girl with a pearl earring: https://raw.githubusercontent.com/brycedrennan/imaginAIry/ma... I integrated this new InstructPix2Pix model into imaginAIry (python library) so it's easy to use for python developers.
AI · 2023 · github.com
HyperSwitch▲715Fast, reliable, and affordable open source payments switch
Dev tools · 2023 · hyperswitch.io
The project is called QueryStorm. It uses Roslyn to offer C# (and VB.NET) support in Excel, as an alternative to VBA. I've posted about it before, but a lot has changed since then so figured I'd share an update. The current version includes a host of new features, namely a C# debugger, support for NuGet packages, and the ability to publish Excel extensions to an "AppStore" (which is essentially a NuGet repository). The AppStore can be used by anyone with the (free) runtime component. Another great addition is the community license, which is a free license for individuals and small companies…
Work · 2023 · querystorm.com
Hey HN! Some more info: I’m an NHS doctor and the founder of Pi-A (https://www.pi-a.io) which developed Lungy (https://www.lungy.app). Lungy is an app (iOS only for now) that responds to breathing in real-time and was designed to make breathing exercises more engaging and beneficial to do. It hopefully has many aspects of interest to the HN community – real-time fluid, cloth and soft body sims running on the phone’s GPU. My background is as a junior surgical trainee and I started building Lungy in 2020 during the first COVID lockdown in London. During COVID, there were…
Life & fun · 2023 · lungy.app
