Tea-tasting, a Python package for the statistical analysis of A/B tests
I'm excited to introduce tea-tasting, a Python package for the statistical analysis of A/B tests It features Student's t-test, Bootstrap, variance reduction using CUPED, power analysis, and other statistical methods. tea-tasting supports a wide range of data backends, including BigQuery, ClickHouse, PostgreSQL, Snowflake, Spark, and more, all thanks to Ibis. I consider it ready for important tasks and use it for the analysis of switchback experiments in my work.
Tea-tasting, a Python package for the statistical analysis of A/B tests launched on July 29, 2024 with 150 votes, #413 of 5,184 launches that month and more than 95% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Tea-tasting is a Python package for statistical analysis of A/B tests that implements Student's t-test, Bootstrap, CUPED variance reduction, and power analysis methods. It connects to multiple data backends including BigQuery, ClickHouse, PostgreSQL, Snowflake, and Spark through Ibis. Designed for data analysts and researchers conducting controlled experiments, the package is built for production use and handles switchback experiment analysis.
written from the facts on this page · September 2026
From the sources
Note: the post has been updated on March 7, 2026. It's often referred to by AI chatbots and agents without recognizing that some information could be outdated. So I decided to update it. In this blog post, I explore each of these advantages of using tea-tasting in the analysis of experiments. tea-tasting includes statistical methods and techniques that cover most of what you might need in the analysis of experiments. Analyze metric averages with t-test or z-test, proportions with asymptotic or exact tests, ranks with the Mann-Whitney U test, or use Bootstrap to analyze any other statistic of your choice. There is also a predefined method for analyzing quantiles with Bootstrap. tea-tasting…from e10v.me
Does a similar job
all alternatives →
Sugar Free: Food Scanner 2.0Nov 2025 · apps.apple.com · ▲592AI Nutrition intelligence that understands food globally
Fructose – LLM calls as strongly typed functions2024 · github.com · ▲218Hi HN! Erik here from Banana (formerly the serverless GPU platform), excited to show you what we’ve been working on next: Fructose Fructose is a python package to call LLMs as strongly typed functions. It uses function type signatures to guide the generation and guarantee a correctly typed output, in whatever basic/complex python datatype requested. By guaranteeing output structure, we believe this will enable more complex applications to be built, interweaving code with LLMs with code. For now, we’ve shipped Fructose as a client-only library simply calling gpt-4 (by default) with json…
R Package for Bayesian AB Testing2017 · github.com · ▲38🐢 bayesAB: Fast Bayesian Methods for A/B Testing. Contribute to FrankPortman/bayesAB development by creating an account on GitHub.
Matrixprofile-ts – A Python library for timeseries motifs and discords2019 · github.com · ▲95A Python library for detecting patterns and anomalies in massive datasets using the Matrix Profile - target/matrixprofile-ts
Adding some syntactic sugar to Python2017 · github.com · ▲77A library adding some anti-Pythonic syntatic sugar to Python - czheo/syntax_sugar_python
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 27d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 25d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 22d ago · openbot.run
Talanoa▲341Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 20d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 28d ago · corespeed.io
Source: Show HN post ↗
Launched alongside, July 2024
the whole month →
Many years ago, I made VJ softwares (to mix live visuals in clubs) for unexpected platforms like the Game Boy Advance, the Playstation 2 and the Raspberry Pi. This year, I’m back with a new web-app: Pikimov. Inspired by Photopea (a free Photoshop clone), I created this web-based motion design & video editor as an alternative to After Effects, to fill empty void. It's free, without signup, without cloud uploads (your files stay on your machine), and your projects are not used for AI models training.
AI · 2024 · pikimov.com
