Hamilton's UI – observability, lineage, and catalog for data pipelines
Hey HN – Stefan and Elijah here from DAGWorks (http://dagworks.io/, YC W23). If you don’t remember us from our previous HN launch (https://news.ycombinator.com/item?id=35056903), we’re the authors of Hamilton (https://github.com/dagworks-inc/hamilton), an open-source library for building self-documenting, modular dataflows in python that works for data, ML, LLM pipelines, & even web-workflows. We’ve been developing this UI for a while and we’re excited to say we open-sourced it! It comes out of the box with the following capabilities, and…
In plain words
Hamilton's UI is an open-source interface for Python data pipelines that provides observability, data lineage, and artifact cataloging. Built to work with Hamilton's modular dataflow library, it requires minimal code changes to enable automatic execution tracking, code profiling, data summary statistics, and lineage visualization. The tool is designed for data engineers and ML practitioners building data, machine learning, LLM, and web-based workflows in Python.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey HN – Stefan and Elijah here from DAGWorks (http://dagworks.io/, YC W23). If you don’t remember us from our previous HN launch (https://news.ycombinator.com/item?id=35056903), we’re the authors of Hamilton (https://github.com/dagworks-inc/hamilton), an open-source library for building self-documenting, modular dataflows in python that works for data, ML, LLM pipelines, & even web-workflows. We’ve been developing this UI for a while and we’re excited to say we open-sourced it! It comes out of the box with the following capabilities, and only requires a single line code change to get: 1. Execution + metadata capture, e.g. automatic code profiling 2. Data/artifact observability, e.g. summary statistics over dataframes, pydantic objects, etc... 3. Lineage & provenance of data, e.g. quickly see what is upstream & downstream of code/data. 4. Asset/transform catalog, e.g. search & find if feature transforms/metrics/datasets/models exist and where they’re used. While the UI currently only self-populates for Hamilton dataflows, we’re looking to expand to other frameworks (we’d love your feedback!). Check out the following video for an overview: https://www.youtube.com/watch?v=0VIVSeN7Ij8, as well as the documentation: https://hamilton.dagworks.io/en/latest/concepts/ui/. We’re looking for feedback/adopters – feel free to reach out if you have any questions!
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
