Scaling Dataflows with Hamilton
A few months ago we released Hamilton, Stitch Fix’s open-source microframework for managing dataflows. With feedback from the community, we’ve implemented a ...
Scaling Dataflows with Hamilton launched on February 22, 2022 with 5 votes and more than 50% of that year's launches. The site was still online in September 2026, 4 years on.
What it does
A few months ago we released Hamilton , Stitch Fix’s open-source microframework [1] for managing dataflows . With feedback from the community, we’ve implemented a variety of new features that make it more general purpose for Data Science/Data Engineering teams. Importantly, Hamilton now operates over any python object types, and can integrate with a variety of distributed compute platforms. Stitch Fix’s demand forecasting team manages an ever-expanding set of complex, highly-configurable forecasts. In the before-times, these pipelines took the form of an ancient codebase that continually manipulated the same dataframe to transform actuals data into predictions. The systems that massaged…from multithreaded.stitchfix.com
Does a similar job
all alternatives →
Hamilton, a Microframework for Creating Dataframes2021 · github.com · ▲64A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton - stitchfix/hamilton
Hamilton's UI – observability, lineage, and catalog for data pipelines2024 · github.com · ▲40Hey HN – Stefan and Elijah here from DAGWorks (http://dagworks.io/, YC W23). If you don’t remember us from our previous HN launch (https://news.ycombinator.com/item?id=35056903), we’re the authors of Hamilton (https://github.com/dagworks-inc/hamilton), an open-source library for building self-documenting, modular dataflows in python that works for data, ML, LLM pipelines, & even web-workflows. We’ve been developing this UI for a while and we’re excited to say we open-sourced it! It comes out of the box with the following capabilities, and…
Patchwork – Open-source framework to automate development gruntwork2024 · github.com · ▲116Hi HN! We’re Asankhaya and Rohan and we are building Patchwork. Patchwork tackles development gruntwork—like reviews, docs, linting, and security fixes—through customizable, code-first 'patchflows' using LLMs and modular code management steps, all in Python. Here's a quick overview video: https://youtu.be/MLyn6B3bFMU From our time building DevSecOps tools, we experienced first-hand the frustrations our users faced as they built complex delivery pipelines. Almost a third of developer time is spent on code management tasks[1], yet backlogs remain. Patchwork lets you combine…
Open-source real time data framework for LLM applications2024 · getindexify.ai · ▲92Hey HN, I am the founder of Tensorlake. Prototyping LLM applications have become a lot easier, building decision making LLM applications that work on constantly updating data is still very challenging in production settings. The systems engineering problems that we have seen people face are - 1. Reliably process ingested content in real time if the application is sensitive to freshness of information. 2. Being able to bring in any kind of model, and run different parts of the pipeline on GPUs and CPUs. 3. Fault Tolerance to ingestion spike, compute infrastructure failure. 4. Scaling compute,…
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 28d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 26d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 23d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 21d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 28d ago · corespeed.io
Source: Show HN post ↗
Launched alongside, February 2022
the whole month →
Sha256 algorithm explained online step by step visually
Life & fun · 2022 · sha256algorithm.com
EdgeDB 1.0▲947Today, after years of research, development, and refinement EdgeDB graduates to a production-ready database. We are very proud to announce EdgeDB 1.0.
Dev tools · 2022 · edgedb.com
