Version code, models, & datasets together in GitHub
Hi HN! We just launched a GitHub integration that scales your Git repos to handle 100 terabytes of files in a single repo. XetData enables data scientists and machine learning engineers to version code, models, and datasets together. Most teams have glued together clunky workflows using S3, DVC, Git, Git LFS, and other tools and make true reproducibility difficult: https://news.ycombinator.com/item?id=37694701 We instead embrace and extend Git so end-users don’t need to learn a new tool and a new set of commands. Our implementation is similar to Git LFS, where we take over the…
Version code, models, & datasets together in GitHub launched on November 16, 2023 with 19 votes, #1,286 of 4,332 launches that month and more than 75% of that year's launches.
In plain words
XetData is a GitHub integration that allows data scientists and machine learning engineers to version control code, models, and datasets together in a single repository, supporting up to 100 terabytes of files. It extends Git's functionality similarly to Git LFS, eliminating the need to learn additional tools or combine multiple services like S3 and DVC. The platform includes automated pull request comments with dataset views and model comparisons to enhance collaboration and reproducibility.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN! We just launched a GitHub integration that scales your Git repos to handle 100 terabytes of files in a single repo. XetData enables data scientists and machine learning engineers to version code, models, and datasets together. Most teams have glued together clunky workflows using S3, DVC, Git, Git LFS, and…
Does a similar job
all alternatives →- WSWe scaled Git to support 1 TB repos2022 · xethub.com · ▲279
I’ve been in the MLOps space for ~10 years, and data is still the hardest unsolved open problem. Code is versioned using Git, data is stored somewhere else, and context often lives in a 3rd location like Slack or GDocs. This is why we built XetHub, a platform that enables teams to treat data like code, using Git. Unlike Git LFS, we don’t just store the files. We use content-defined chunking and Merkle Trees to dedupe against everything in history. This allows small changes in large files to be stored compactly. Read more here:…
- GFGitHub for ML datasets2018 · dataturks.com · ▲15
I replaced vector databases with Git for AI memory (PoC)2025 · github.com · ▲198Hey HN! I built a proof-of-concept for AI memory using Git instead of vector databases. The insight: Git already solved versioned document management. Why are we building complex vector stores when we could just use markdown files with Git's built-in diff/blame/history? How it works: Memories stored as markdown files in a Git repo Each conversation = one commit git diff shows how understanding evolves over time BM25 for search (no embeddings needed) LLMs generate search queries from conversation context Example: Ask "how has my project evolved?" and it uses git diff to show actual…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, November 2023
the whole month →
Discover & book top creators to promote your product
Growth · 2023 · passionfroot.me
SpeedVitals▲829Hi HN, I’m Kashish, one of the co-founders of SpeedVitals, a platform for testing & monitoring website performance with a major emphasis on Core Web Vitals. Until now, our platform has only focused on Synthetic/Lab testing, such as Lighthouse tests and TTFB tests. But now, we have developed a Real-User Monitoring platform that doubles up as a Web Analytics tool. One major feature we have that most RUM tools lack is Performance Audits. Just like Lighthouse audits work on Lab test data, our audits work across all the real-user visits and gives suggestions on what task to prioritize first.…
Dev tools · 2023 · speedvitals.com