Sliceguard, a Tool for Finding Critical Data Segments in ML Data
Hey HN, We know from experience that a qualitative understanding of critical data segments is necessary when developing ML models. Better tooling makes finding these data slices faster, more systematic and helps to communicate with domain experts. We built sliceguard to go from a raw dataset to an interactive report on critical data slices with just 3 lines of code: https://github.com/Renumics/sliceguard Behind the scenes, we use hierarchical clustering and explainable AI techniques to detect and rank data slices based on features, metadata and embeddings. Here is some…
What it does
In the maker’s words, at launch
Hey HN, We know from experience that a qualitative understanding of critical data segments is necessary when developing ML models. Better tooling makes finding these data slices faster, more systematic and helps to communicate with domain experts. We built sliceguard to go from a raw dataset to an interactive report on critical data slices with just 3 lines of code: https://github.com/Renumics/sliceguard Behind the scenes, we use hierarchical clustering and explainable AI techniques to detect and rank data slices based on features, metadata and embeddings. Here is some more information on Sliceguard: - Works on structured, unstructured data (image, audio, NLP, multimodal) and hybrid datasets. - Directly works on existing Pandas DataFrames. - Automatic computation of embeddings and AutoML functionality to pinpoint problems without any setup. - Interactive GUI for slice inspection supports multimodal data and can be configured with drag-n-drop.
Does the same job
all alternatives →
- ALA labelling tool to easily extract and label Wikipedia data2021 · ▲116
Hi HN! I am Maria, solo founder of DataQA (https://dataqa.ai/), a tool to search and label documents for various NLP tasks (e.g. entity extraction, entity linking, etc). I have worked as a data scientist and ML engineer for the better part of a decade, and over that time have specialised mainly in applications involving natural language processing (NLP). One of the key questions I have always had at the back of my mind is whether my time was well spent. Whenever I spent more time on feature engineering or trying different models, I always wondered whether I would get better…
- DCDatasets.co – Share and discover new machine learning datasets2016 · datasets.co · ▲81
- MAMeTA: A Modern C++ Data Sciences Toolkit2014 · meta-toolkit.github.io · ▲39
- 3I3LC – Illuminate the ML Black Box2024 · pypi.org · ▲8
Hello HackerNews, I am Paul, and I would like to get some feedback on the tool we are releasing as beta today. 3LC is an ML tool that gives detailed insights, real-time data-centric iterative workflows for training/finetuning, and data quality improvements for your Machine Learning datasets and models. 3LC serves as a visualizer, editor, and debugger, focusing on how models learn from the training data. Key Features of 3LC: • Detailed Data Analysis: 3LC enables users to dive into model performance beyond typical labeling errors. It offers the capability to analyze intricate false…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, August 2023
the whole month →
Lottielab▲871Create and ship lottie animations to sites and apps faster
Dev tools · 2023 · lottielab.com
- LC
Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…
AI · 2023 · github.com