Pandera – An open source data testing framework for data science and ML
Hi HN, I'm excited to share Pandera, an open source framework for data testing, built for data scientists and ML engineers: https://www.union.ai/pandera I’ve been working with data and building models for a decade, and one of the biggest pain points for me is working with low-quality data. I got burned by incorrect data types and unexpected values so many times that I built a Pandera to help you safeguard your pipelines from silent data bugs. You can create schemas for your dataframe-like objects, which can be validated at run-time, in your unit tests via property-based…
What it does
In the maker’s words, at launch
Hi HN, I'm excited to share Pandera, an open source framework for data testing, built for data scientists and ML engineers: https://www.union.ai/pandera I’ve been working with data and building models for a decade, and one of the biggest pain points for me is working with low-quality data. I got burned by incorrect data types and unexpected values so many times that I built a Pandera to help you safeguard your pipelines from silent data bugs. You can create schemas for your dataframe-like objects, which can be validated at run-time, in your unit tests via property-based testing, or in your type-linting hooks via mypy. Give it a try here: https://pandera.readthedocs.io/en/stable/try_pandera.html
Does the same job
all alternatives →
- DAdstack – an open-source tool to build data applications easily2020 · ▲134
Dear HN, I am Riwaj, the cofounder of dstack.ai (https://github.com/dstackai). A few months ago, we built an online service that allows users to publish data visualizations from Python or R. The idea was to build a tool that did not require additional programming or front-end development for publishing data visualizations. Such a code can be invoked from either Jupyter notebook, RMarkdown, Python, or R scripts. Once the data is pushed, it can be accessed via a browser. Open-sourcing dstack: During our customer discovery phase, we realized that dstack.ai should integrate a lot…
- DODataHub, open source datasets for Artificial Intelligence2017 · ▲14
Hi HN! I am an undergrad student trying to build interesting things with AI. Recently, I was looking for a dataset I could use for a new project. I realized that it is really frustrating to go through all the government websites (with terrible UX) just to find some usable dataset. I set out to build a GitHub for datasets, named DataHub. Right now, we have more than 1000 datasets from Montréal and New York City, with more cities coming soon (and possible government agencies). All of this is wrapped into a powerful search. It's a breeze to find a dataset to work on. I'd be interested to know…
- SMSyna – Minimal ML and RL Framework Built from Scratch with NumPyOct 2025 · github.com · ▲8
Hello HN, I built Syna to understand how modern ML frameworks like PyTorch actually work — from the ground up. It’s a minimal, define-by-run (dynamic graph) framework inspired by DeZero, written entirely with NumPy. Unlike most libraries, Syna includes a basic reinforcement learning module right inside the same framework — no separate packages. It’s not about speed or GPUs — it’s about clarity, simplicity, and learning the internals of machine learning. Great for students, educators, and anyone curious about what’s really happening under the hood. GitHub:…
- SPScalable pan/zoom with easy authoring, arbitrary datatypes2020 · ▲22
https://github.com/tracyhenry/Kyrix Open source MIT project from Turing award winner Michael Stonebraker. Seeking contributors.
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 16d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, October 2022
the whole month →- I3
Hey HN, I 3d scanned the interior of the Great Pyramid / Khufu's pyramid for the Giza Project this summer and just finished the guided version to share. Would love feedback and/or problems you encounter. I used both a Leica BLK 360 and Matterport Pro 2 to do the scanning and the Matterport SDK for the web viewer. Matterport's web display with Three.js has been the most accessible to a wide audience in the past (previous iterations are in Unity and Unreal, but difficult to download over slower connections). I've been interviewing social studies teachers around the 6th grade level to…
Dev tools · 2022 · giza.mused.org
- RO
Last time I posted this it didn't garner much interest. There have been lots of improvements and fixes since the last release. Quick list of features: - Workspaces - Tabs - Nested Folders - Lots of context menus - Response history - Plugins - Runs fully in the browser and runs offline if necessary - Chrome and Firefox extension to bypass CORS restrictions - Desktop builds for all platforms - GraphQL support - Import collections exported from Postman and Insomnia - Simple user friendly interface I built this because I love Insomnia but wanted a portable version that I could run in the…
Dev tools · 2022 · restfox.dev
Micro SaaS HQ▲672Profitable Micro SaaS ideas, supportive community
Life & fun · 2022 · microsaashq.com
- AR
APIs are a little like the open road—always waiting, full of opportunity, but hardly utilized. So here’s America, composed of several APIs that paint a vivid, real-time picture of a good old-fashioned road trip. Get local classifieds and photos. Tune into local radio stations. Talk wit other drivers. And more. Exploring the country by car was an invaluable experience for me during my time in the states. I’ve since moved to Barcelona, and find myself missing the territory. This is my attempt at recreating the magic.
Life & fun · 2022 · 4m3ric4.com