Phidata – Building Blocks for Data Engineering
Hi HN, I’ve been a data engineer for over a decade and have slowly been working on a way for 1 person to run a data platform (mostly to automate my job) When I was starting, I found the wide range of tools required to build data products overwhelming. Infrastructure setup, app deployment, pipeline development, creating metrics, dashboards and finally, integrating ML into product. No wonder companies have 100s of people working on this. So last year I quit the best job in the world to figure this out. I’d like to introduce Phidata: Building Blocks for Data Engineering It works like this: 1.…
In plain words
Phidata is a framework designed to simplify data platform development by providing pre-configured building blocks for common tools like Airflow, Superset, and Jupyter. It allows data engineers to define infrastructure and applications as Python objects, build data products in Python or SQL, and deploy to AWS. The platform targets small teams or individual data engineers who want to reduce the complexity of managing multiple tools, infrastructure setup, and pipeline development traditionally requiring large teams.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN, I’ve been a data engineer for over a decade and have slowly been working on a way for 1 person to run a data platform (mostly to automate my job) When I was starting, I found the wide range of tools required to build data products overwhelming. Infrastructure setup, app deployment, pipeline development, creating metrics, dashboards and finally, integrating ML into product. No wonder companies have 100s of people working on this. So last year I quit the best job in the world to figure this out. I’d like to introduce Phidata: Building Blocks for Data Engineering It works like this: 1. You start with a codebase that has common data tools like Airflow, Superset and Jupyter pre-configured. Infrastructure and Apps are defined as python objects. 2. Build data products (tables, metrics, models) in python or SQL. Test locally using docker and run production on AWS. 3. Infrastructure, Apps and Data Products live in the same codebase. Teams working together share code and dependencies in a pythonic way. Using phidata, I’ve been running multiple data platforms and have automated most of my boilerplate code using a specially trained GPT-3 data assistant. If you work with data and are looking for a better development experience, checkout [phidata.com](https://www.phidata.com/) or message me, I’d love your feedback. Ashpreet
More work this month
the category →
The app store for voice native apps that lives in your notch
Work · 28d ago · voiceos.com
The New Calendly▲211Handle all of the work before, during, and after meetings
Work · 17d ago · calendly.com
Launched alongside, January 2023
the whole month →- NA
This works suprisingly well. Just give it instructions like "make it winter" or "remove the cars" and the photo is altered. Here are some examples of transformations it can make: Golden gate bridge: https://raw.githubusercontent.com/brycedrennan/imaginAIry/ma... Girl with a pearl earring: https://raw.githubusercontent.com/brycedrennan/imaginAIry/ma... I integrated this new InstructPix2Pix model into imaginAIry (python library) so it's easy to use for python developers.
AI · 2023 · github.com
HyperSwitch▲715Fast, reliable, and affordable open source payments switch
Dev tools · 2023 · hyperswitch.io
- IV
The project is called QueryStorm. It uses Roslyn to offer C# (and VB.NET) support in Excel, as an alternative to VBA. I've posted about it before, but a lot has changed since then so figured I'd share an update. The current version includes a host of new features, namely a C# debugger, support for NuGet packages, and the ability to publish Excel extensions to an "AppStore" (which is essentially a NuGet repository). The AppStore can be used by anyone with the (free) runtime component. Another great addition is the community license, which is a free license for individuals and small companies…
Work · 2023 · querystorm.com
- IM
Hey HN! Some more info: I’m an NHS doctor and the founder of Pi-A (https://www.pi-a.io) which developed Lungy (https://www.lungy.app). Lungy is an app (iOS only for now) that responds to breathing in real-time and was designed to make breathing exercises more engaging and beneficial to do. It hopefully has many aspects of interest to the HN community – real-time fluid, cloth and soft body sims running on the phone’s GPU. My background is as a junior surgical trainee and I started building Lungy in 2020 during the first COVID lockdown in London. During COVID, there were…
Life & fun · 2023 · lungy.app
