data-diff

What it does

Data Diff is an open-source package that can be run in a CLI or wrapped into any data orchestrator such as Airflow, Dagster, etc. Compare datasets quickly (seconds/minutes) at a large (millions/billions of rows) scale across different databases.

Compare tables within or across databases. Contribute to datafold/data-diff development by creating an account on GitHub.

• DD
  Data Diff – compare tables of any size across databases2022 · ▲127

  Gleb, Alex, Erez and Simon here – we are building an open-source tool for comparing data within and across databases at any scale. The repo is at https://github.com/datafold/data-diff, and our home page is https://datafold.com/. As a company, Datafold builds tools for data engineers to automate the most tedious and error-prone tasks falling through the cracks of the modern data stack, such as data testing and lineage. We launched two years ago with a tool for regression-testing changes to ETL code…

• DI
  Diff.io2015 · diff.io · ▲194
• 
  data-diff2022 · github.com · ▲10

  Efficiently diff data in or across relational databases

• AN
  A new way to diff websites2015 · diffd.com · ▲46
• 
  A benchmark for analytical databases (Snowflake, Druid, Redshift)2022 · benchmark.clickhouse.com · ▲137

  I created a web page to compare different analytical databases (both self-managed and services, open-source and proprietary) on a realistic dataset. It contains 20+ databases, each with installation and data loading scripts. And they can be compared to each other on a set of 43 queries, by data load time or by storage size. There are switches to select different types of databases for comparison - for example, only MySQL compatible or PostgreSQL compatible. If you play with the switches, many interesting details will be uncovered. Full description:…

• DA
  Diffmatic – A daily GitHub digest for dev teams2014 · diffmatic.com · ▲67

• 
  Run full iOS 16.5 firmware on Android Devices▲515

  Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.

  Dev tools · 2d ago · github.com

• 
  /dev/null▲492

  a social network for devs

  Dev tools · 5d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 29d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 25d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 22d ago · talanoa.email

• AF
  A friend and I spent 6 years making a simulation game, finally released▲1,157

  I've seen some interests in (simulation) video games here on HN so I thought I'd share a short version of our story. More than 6 years ago, me and my friend from university were playing around with an idea of making a game we always wanted to play. We worked on it on weekends but the progress was quite slow, especially due to so many dead ends and wasted effort. Eventually however, we solidified our direction and decided to take the risk to resign from our well paid SWE jobs and work on it full time. It took more than a year but yesterday we have finally released it on Steam:…

  Life & fun · 2022

• 
  A central bank simulator game with a realistic economic model▲815

  Life & fun · 2022 · benoitessiambre.com

• 
  I stripped DALL·E Mini to its bare essentials and converted it to Torch▲663

  min(DALL·E) is a fast, minimal port of DALL·E Mini to PyTorch - kuprel/min-dalle

  Life & fun · 2022 · github.com

• 
  Untitled UI Icons▲634

  1,100+ free Figma icons for modern UI design.

  Dev tools · 2022 · untitledui.com

• 
  MakerBox▲621

  600 tools with generous free plans for indie entrepreneurs

  Dev tools · 2022 · makerbox.club

• 
  Typedream CMS▲593

  Display your Notion data beautifully on your site, no-code

  Dev tools · 2022 · typedream.com