Dev tools · September 26, 2024
Structured, GitHub App for Automated DBT PR Reviews
Scaling data teams today means dealing with the complexity of the modern data stack. While DBT has become a core tool for transforming raw data into structured, analytics-ready tables, most teams are using it in ways that lead to chaos: duplicated models, inconsistent metrics, and inefficient SQL that directly impacts cloud spend. The real issue isn’t with DBT itself—it’s in how it’s applied across teams. Here’s the typical setup: Finance defines a revenue model, Marketing calculates customer lifetime value, and Product defines churn. All in DBT, but all with slightly different logic,…
What it does
In the maker’s words, at launch
Scaling data teams today means dealing with the complexity of the modern data stack. While DBT has become a core tool for transforming raw data into structured, analytics-ready tables, most teams are using it in ways that lead to chaos: duplicated models, inconsistent metrics, and inefficient SQL that directly impacts cloud spend. The real issue isn’t with DBT itself—it’s in how it’s applied across teams. Here’s the typical setup: Finance defines a revenue model, Marketing calculates customer lifetime value, and Product defines churn. All in DBT, but all with slightly different logic, leading to metric fragmentation. This results in data drift, conflicting reports, and a ton of unnecessary engineering time spent reconciling definitions. Worse, engineers end up re-inventing the wheel by duplicating logic that already exists in other models. The inefficiencies don’t stop there: suboptimal SQL patterns (e.g., full-table scans, poor joins) creep into production and drive up cloud costs. We designed our GitHub App to automate the grunt work of DBT model management, focusing on three key areas: preventing redundant logic, maintaining the semantic layer, and optimizing SQL performance. --- (1) Stop Redundant Models: A lot of teams waste time rebuilding models that already exist. Engineers aren’t aware of what’s been built, so they duplicate work. Our app automatically reviews pull requests, flags redundant models, and suggests reusing existing logic. This keeps your key metrics like revenue or churn consistent across teams and prevents conflicting reports. (2) Maintain the Semantic Layer: DBT’s value is in creating a semantic layer—a consistent definition of business metrics. But as teams scale, maintaining this layer gets tricky. People unknowingly break it with small changes, leading to inconsistencies. Our app checks every new model for deviations from the semantic layer, flagging inconsistencies before they go live. This prevents those all-too-common situations where two departments are debating whose revenue number is right. By ensuring everyone’s using the same definitions, you avoid trust issues with the data. (3) SQL Performance = Real Costs: Bad SQL isn’t just a performance problem—it’s a cost problem. Inefficient joins, full-table scans, and poorly written SQL in your DBT models can blow up your cloud bill. Our app reviews SQL in pull requests, flags inefficiencies, and suggests optimizations. Example: An engineer submits a model that joins two large tables without filtering. Our app flags the full-table scan and suggests using indexed columns and adding WHERE filters. This reduces query cost and improves performance before the code hits production. --- Data engineers are already stretched thin with the demands of modern data pipelines. By automating model consistency checks, semantic layer enforcement, and SQL performance reviews, our GitHub App frees up your team to focus on higher-impact work rather than wasting cycles on repetitive tasks or fighting fires caused by bad data logic. The app is live—give it a try, and let us know how it’s improving your workflow. Also, keep an eye out for our upcoming DBT code generation features—we’re automating more of the heavy lifting soon.
Does the same job
all alternatives →
- PAProduct Analytics in SQL with dbt2022 · github.com · ▲8
Hey everyone! Like many data analysts and engineers, I love SQL and the dbt ecosystem. So it bothers me that we have to use separate tools for product analytics. We do our transformations, BI work, and ad-hoc queries in SQL, but when it's time to look at funnels and flows, we have to use (and procure) a separate platform like Mixpanel or Amplitude. This dbt package is a (very rough) start at fixing that. With it, you can create event streams and run funnel analyses via dbt[0]. More features like flows and retention are coming soon! But I'm mostly curious how you all are doing product…
- AOAn Open-Source Collaborative Database Development Tool2023 · github.com · ▲9
A couple of years ago, we had an interesting idea. When a development team within an organization consists of around 10 members or fewer, controlling the risks associated with database changes might be achieved through trust and real-time communication. However, as the team grows, the responsibilities within the team become more specialized. Imagine a scenario where different branches of the business exist, each with its own set of developers, team leads, testers, testing leads, DBAs, and more. Relying solely on traditional communication methods becomes increasingly challenging when it comes…
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, September 2024
the whole month →
BeforeSunset AI 2.0▲1,267Personalized AI daily planning that suits your life
AI · 2024 · beforesunset.ai
