Versatile Data Kit

What it does

An open-source framework that enables anybody to create their own data pipelines, with: - Data SDK for the automation of data extraction, transformation, and loading. - Control Service to deploy, manage and execute Data Jobs in Kubernetes runtime environment.

⛔ [ARCHIVING ON 2026-11-01] One framework to develop, deploy and operate data workflows with Python and SQL. - vmware-archive/versatile-data-kit

• 
  I built an open-source data pipeline tool in Go2024 · github.com · ▲200

  Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…

• 
  Data pipelines for data scientists (new open-source notebook)2022 · mage.ai · ▲36

  Mage AI helps data teams build, orchestrate, monitor, and repair production data pipelines with editable SQL and Python for analytics, applications, and AI.

• 
  All-in-one open source tool for data pipelines2022 · github.com · ▲21

  🧙 Build, run, and manage data pipelines for integrating and transforming data. - mage-ai/mage-ai

• AP
  Automated Pipelines to Your Kubernetes Clusters2016 · distelli.com · ▲81
• 
  Mozart Data2021 · mozartdata.com · ▲112

  The easiest way for teams to build a Modern Data Stack

• 
  Pipelines – Language for scripting parrallel pipelines with Python2019 · github.com · ▲69

  An experimental programming language for data flow - calebwin/pipelines

• 
  /dev/null▲486

  a social network for devs

  Dev tools · 3d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 28d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 26d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 23d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 21d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 28d ago · corespeed.io

• 
  Locofy.ai▲720

  Hey there, I am Honey - Co-founder at Locofy. We, along with other makers, have been building Locofy.ai to help massively accelerate the builders - the designers, developers, product managers & founders, and we're excited to share it today with the HN Community. Locofy.ai simply converts designs into production-ready frontend code for web and mobile apps. It adapts to your preferred design tools, tech stack, and workflows with ease and lets you launch upto 4x faster, while letting designers design the way they want and giving high-quality responsive code that developers love. With this…

  Dev tools · 2022 · locofy.ai

• PO
  Porting OpenBSD Pledge() to Linux▲571

  Life & fun · 2022 · justine.lol

• 
  PocketBase – Open Source realtime backend in one file▲563

  Open Source realtime backend in 1 file. Contribute to pocketbase/pocketbase development by creating an account on GitHub.

  Dev tools · 2022 · github.com

• 
  ruttl▲552

  Visual & Design Feedback Tool that helps you review design, content and report bugs on websites, applications, PDFs and images.

  Dev tools · 2022 · ruttl.com

• 
  Pipes puzzle (a.k.a. Net) on a hexagonal grid▲537

  Hello, HN - I wanted to share this puzzle game I made during my vacation. I'm rather fond of the pipes puzzle where your goal is to restore a scrambled network of connections by rotating tiles. It's usually played on a grid of squares and this all started when I decided to make a programmatic solver for that kind of puzzle. Then I realized that with some minor changes the solver could generate new puzzle instances. I thought about what kind of puzzle to make and someone suggested a hexagonal grid. Adapting the generator wasn't too hard but then I had to create a way to play this variant. So…

  Life & fun · 2022 · hexapipes.vercel.app

• 
  Parsnip – Duolingo for Cooking▲536

  We're building Parsnip to create a "tech tree" of cooking skills that allows anyone to level up on the building blocks of cooking knowledge while tracking their progress over time. It took us a few iterations to figure out the right product; here's the story of our latest pivot: [https://parsnip.substack.com/p/a-new-hope] The goal is to create a personalized way to learn any recipe on the Internet, then use this as a springboard to help home cooks of all levels solve the problem of repeated meal planning in a 10x better way:…

  Work · 2022 · parsnip.ai