Zingg – open-source entity resolution for single source of truth
Hello HN, I am Sonal, a data consultant from India. For the past few months(and years!), I have been working on an entity resolution tool to build a single source of truth for customers, suppliers, products and parts. Here is a short demo of Zingg in action https://www.youtube.com/watch?v=zOabyZxN9b0 As a data consultant, I often struggled to build unified views of core entities on the datalake and the warehouse. Data spread across different systems has variations and consistencies making Customer 360, KYC, AML, segmentation, personalization and other analytics difficult. As I…
In plain words
Zingg is an open-source tool that identifies and merges duplicate records across databases to create unified customer, supplier, product, and parts data. It addresses the problem of inconsistent data spread across multiple systems that complicates analytics, compliance, and personalization efforts. The tool targets data consultants and organizations needing cost-effective entity resolution without expensive commercial master data management solutions.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hello HN, I am Sonal, a data consultant from India. For the past few months(and years!), I have been working on an entity resolution tool to build a single source of truth for customers, suppliers, products and parts. Here is a short demo of Zingg in action https://www.youtube.com/watch?v=zOabyZxN9b0 As a data consultant, I often struggled to build unified views of core entities on the datalake and the warehouse. Data spread across different systems has variations and consistencies making Customer 360, KYC, AML, segmentation, personalization and other analytics difficult. As I talked with different clients facing this issue, I searched for existing solutions which I could use or recommend. Unfortunately, most of them were very expensive MDM solutions like Tamr, or CDP solutions like Amperity. There were many open source libraries, but they did not tie well into the datalake/warehouse scenarios we were working with, did not scale and/or needed a decent bit of programming or did not generalize. I even tried to build something internally and failed miserably, and that got me hooked :-) As I dug deeper into the problem, I realized that there were multiple challenges. Data matching, at its very core, becomes a cartesian join, as you need to compare every pair of records to figure out the matches. With millions of records, this becomes extremely tough to scale. I referred to various research papers and then implemented a blocking algorithm to overcome this. More details at https://docs.zingg.ai/docs/zModels.html The second challenge was to say which pairs are a match. I wanted to have a machine learning-based approach to handle the different types of entities and the variety of differences in real world data. But I also felt that non ML experts should be able to use Zingg easily, hence took the approach of abstracting the feature generation and hyper-parameter tuning for the classifier. Once I settled on the ML approach, the problem of training data quickly arose, which led me to pick up active learning and build an interactive labeler through which sample records can be marked as matches and non matches to build training sets quickly. I still feel that we should have an unsupervised approach as well, but I have not yet figured out the right way to do so. The Zingg repository is hosted at https://github.com/zinggAI/zingg and we have close to 60 members on our Slack(https://join.slack.com/t/zinggai/shared_invite/zt-w7zlcnol-vEuqU9m~Q56kLLUVxRgpOA). We are now two developers working full time on Zingg!!! I am super happy that early users have been able to use Zingg and push us to build more stuff - model documentation, using pre-existing training data, native Snowflake integration etc. I have been an open source consumer all my dev life, and this is the first time I have made a decent contribution. It is my first time trying to build a community as well. Not sure how the future will unfold, but wanted to reach out to the community here and hear what you think about the problem, the approach, any ideas or suggestions. Thanks for reading along, and please do post your thoughts in the comments below.
Does the same job
all alternatives →
- DAdstack – an open-source tool to build data applications easily2020 · ▲134
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
- JAJig – a tool to define, compute and monitor metrics2021 · jigdev.com · ▲74
Hi HN, 8 months ago, I posted “Ask HN: I built it nobody came, what now?” and got a ton of (not very optimistic) feedback [1]. I took away 3 things: 1. The message wasn’t properly targeted 2. The onboarding experience was terrible. 3. Someone posted some advice to find leads which actually worked, yeah! So here we are. 8 months later. I unfortunately didn’t have a lot of time to put in the tool itself. But I improved the website, improved the onboarding, and got a paying customer who seems to really like the software. So here it is in its current form. Let me know what you think would make…
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Launched alongside, February 2022
the whole month →
- S2
Life & fun · 2022 · sha256algorithm.com
- E1EdgeDB 1.0▲947
Life & fun · 2022 · edgedb.com
