SQLMesh – The Future of DataOps
Hey Show HN! I’m Toby and over the last few months, I’ve been working with a team of engineers from Airbnb, Apple, Google, and Netflix, to simplify developing data pipelines with SQLMesh (https://github.com/TobikoData/sqlmesh). We’re tired of fragile pipelines, untested SQL queries, and expensive staging environments for data. Software engineers have reaped the benefits of DevOps through unit tests, continuous integration, and continuous deployment for years. We felt like it was time for data teams to have the same confidence and efficiency in development as their peers.…
In plain words
SQLMesh is an open-source data pipeline tool that applies software engineering practices to data development. It enables data teams to write, test, and deploy SQL queries with unit tests, continuous integration, and continuous deployment workflows. The platform features a CLI, notebook interface, and web-based IDE, and uses virtual data marts to create efficient development and staging environments. SQLMesh is designed for data engineers and analysts who need reliable, tested pipelines without expensive staging infrastructure.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hey Show HN! I’m Toby and over the last few months, I’ve been working with a team of engineers from Airbnb, Apple, Google, and Netflix, to simplify developing data pipelines with SQLMesh (https://github.com/TobikoData/sqlmesh). We’re tired of fragile pipelines, untested SQL queries, and expensive staging environments for data. Software engineers have reaped the benefits of DevOps through unit tests, continuous integration, and continuous deployment for years. We felt like it was time for data teams to have the same confidence and efficiency in development as their peers. It’s time for DataOps! SQLMesh can be used through a CLI/notebook or in our open source web based IDE (in preview). SQLMesh builds efficient dev / staging environments through “Virtual Data Marts” using views, which allows you to seamlessly rollback or roll forward your changes! With a simple pointer swap you can promote your “staging” data into production. This means you get unlimited copy-on-write environments that make data exploration and preview of changes cheap, easy, safe. Some other key features are: Automatic DAG generation by semantically parsing and understanding SQL or Python scripts CI-Runnable Unit and Integration tests with optional conversion to DuckDB Change detection and reconciliation through column level lineage Native Airflow Integration Import an existing DBT project and run it on SQLMesh’s runtime (in preview) We’re just getting started on our journey to change the way data pipelines are built and deployed. We’re huge proponents of open source and hope that we can grow together with your feedback and contributions. Try out SQLMesh by following the quick start guide (https://sqlmesh.readthedocs.io/en/stable/quick_start/). We’d love to chat and hear about your experiences and ideas in our Slack community (https://join.slack.com/t/tobiko-data/shared_invite/zt-1ma66d...).
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, March 2023
the whole month →
- BI
I'm a big fan of the BBC podcast In Our Time -- and (like most people) I've been playing with the OpenAI APIs. In Our Time has almost 1,000 episodes on everything from Cleopatra to the evolution of teeth to plasma physics, all still available, so it's my starting point to learn about most topics. But it's not well organised. So here are the episodes sorted by library code. It's fun to explore. Web scraping is usually pretty tedious, but I found that I could send the minimised HTML to GPT-3 and get (almost) perfect JSON back: the prompt includes the Typescript definition. At the same time I…
AI · 2023 · genmon.github.io