Rezolva – Entity resolution for everyone. Minimal. No dependencies
Hello HN! I'm excited to share rezolva, a project I've been hacking on recently. Rezolva is a lightweight library for performing entity resolution - a crucial task in data integration that involves identifying and linking different representations of the same real-world entity across datasets. The goal is to provide a wide range of out-of-the-box functionality (matching algorithms, preprocessing functions, etc.) while also offering a framework for modular/customizable pieces of the process. To ensure easy integration into projects, the base library is built entirely using the Python…
What it does
In the maker’s words, at launch
Hello HN! I'm excited to share rezolva, a project I've been hacking on recently. Rezolva is a lightweight library for performing entity resolution - a crucial task in data integration that involves identifying and linking different representations of the same real-world entity across datasets. The goal is to provide a wide range of out-of-the-box functionality (matching algorithms, preprocessing functions, etc.) while also offering a framework for modular/customizable pieces of the process. To ensure easy integration into projects, the base library is built entirely using the Python standard library. It's still in the early stages, so please proceed with a healthy dose of skepticism. I'd love to hear your thoughts and feedback! Some planned enhancements for the near future include: - More resolution algorithms / methods - Larger list of default preprocessing functions - Examples integrating with other systems for candidate selection (i.e. pgvector, hnswlib, etc.) - Examples integrating with other systems for candidate reranking (i.e. distributing on dask/spark/etc.) - A more functional suite of utilities for comparing resolution implementations against each other - Utilities for auto-tuning resolution parameters You can check out the project here: https://github.com/joshmeek/rezolva I'm particularly interested in hearing about use cases or integration scenarios you'd like to see supported. Any contributions or suggestions are welcome!
Does the same job
all alternatives →- PIParabola.io – Automate your work with visual programming2018 · parabola.io · ▲291
Hi HN! I’m Alex from Parabola (https://parabola.io). Parabola is a visual programming tool for creating functional data flows that everyone can use. It’s entirely drag-and-drop, handles data sizes much larger than a traditional spreadsheet, calculates everything live, and can run your flows on a schedule of your choosing. I used to work in strategy consulting, doing data analytics for SMBs and Fortune 500 companies. The amount of time wasted on menial tasks was astounding. Things like cleaning data, generating custom reports, creating human workflows to solve shortcomings in third…
- ALA labelling tool to easily extract and label Wikipedia data2021 · ▲116
Hi HN! I am Maria, solo founder of DataQA (https://dataqa.ai/), a tool to search and label documents for various NLP tasks (e.g. entity extraction, entity linking, etc). I have worked as a data scientist and ML engineer for the better part of a decade, and over that time have specialised mainly in applications involving natural language processing (NLP). One of the key questions I have always had at the back of my mind is whether my time was well spent. Whenever I spent more time on feature engineering or trying different models, I always wondered whether I would get better…
- VUVisions – User defined data type systems2022 · github.com · ▲45
Visions is a python library for working with user defined data type systems. Out of the box, it provides type inference and automated data cleaning of sequence data with backend specific implementations for pandas, spark, python, and numpy. We often use it as a first pass cleaning step when working with tabular data and to simplify the backend logic of both pandas-profiling[1] and our tabular data compression library compressio[2]. Because data types are user defined, we can build user customizable libraries based around types without adding code complexity. In the case of compressio that…
- ZOZingg – open-source entity resolution for single source of truth2022 · ▲18
Hello HN, I am Sonal, a data consultant from India. For the past few months(and years!), I have been working on an entity resolution tool to build a single source of truth for customers, suppliers, products and parts. Here is a short demo of Zingg in action https://www.youtube.com/watch?v=zOabyZxN9b0 As a data consultant, I often struggled to build unified views of core entities on the datalake and the warehouse. Data spread across different systems has variations and consistencies making Customer 360, KYC, AML, segmentation, personalization and other analytics difficult. As I…
- SVSirixDB – versioning through efficient snapshotting2018 · ▲55
I've already posted yesterday, but I'd really love to get comments, any kind of questions, suggestions and help would be greatly appreciated as it's an Open Source project of mine (and was for others during my studies at the University of Konstanz 6 years ago). Since then I spent countless ours to bring forth the idea of a versioned storage system, especially well suited for analytical tasks for timd-varying data. Especially I'd love to discuss what documentation you need, which next steps are necessary (JSON, Cloud...), API additions or changes... I've updated the README quiet a bit, such…
- SFSmart Fruit – A Python schema-based machine learning library2018 · ▲44
I've made a small Python library, designed for quick-and-easy prototyping of machine learning models. It's built on top of scikit-learn, to serialize and deserialize data from the forms you're likely to have, to the format used in scikit-learn. https://github.com/madman-bob/Smart-Fruit It's pretty bare-bones at the moment, but I thought I'd see if there was any interest before spending too much time on it. Let me know what you think.
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, July 2024
the whole month →
- IC
Many years ago, I made VJ softwares (to mix live visuals in clubs) for unexpected platforms like the Game Boy Advance, the Playstation 2 and the Raspberry Pi. This year, I’m back with a new web-app: Pikimov. Inspired by Photopea (a free Photoshop clone), I created this web-based motion design & video editor as an alternative to After Effects, to fill empty void. It's free, without signup, without cloud uploads (your files stay on your machine), and your projects are not used for AI models training.
AI · 2024 · pikimov.com
