Python lib to run evals across providers: OpenAI, Anthropic, etc.
Library makes requests asynchronously across models, so you can spend a lot of $$ quickly if you want XD. But seriously I hope this enables folks to create and run evals (especially safety ones) a lot easier than before.
Does the same job
all alternatives →
- CACatsu: A unified Python client for embedding APIsDec 2025 · catsu.dev · ▲7
We open-sourced catsu, a Python client for embedding APIs. The problem: every embedding provider has a different SDK with different bugs. OpenAI has undocumented token limits. VoyageAI's retry logic was broken until September. Cohere breaks downstream libraries every release. LiteLLM's embedding support is minimal. catsu provides: - One API for 11 providers (OpenAI, Voyage, Cohere, Jina, Mistral, Gemini, etc.) - Bundled database of 50+ models with pricing, dimensions, and benchmark scores - Built-in retry with exponential backoff - Automatic cost tracking per request - Full async support…
- ABA batched, concurrent wrapper for OpenAI gym library2020 · ▲12
Hello HN, new user here, so please let me know if I break some rules. Currently I've been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it. I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested. For more details, please visit: https://github.com/Chimpan-Z/agymc Feedback really appreciated! Have a good day everyone!
- OAOpytimizer – A Nature-Inspired Python Optimizer2021 · ▲23
https://github.com/gugarosa/opytimizer Did you ever reach a bottleneck in your computational experiments? Are you tired of selecting suitable parameters for a chosen technique? If yes, Opytimizer is the real deal! This package provides an easy-to-go implementation of meta-heuristic optimizations. From agents to search space, from internal functions to external communication, we will foster all research related to optimizing stuff. Use Opytimizer if you need a library or wish to: - Create your optimization algorithm; - Design or use pre-loaded optimization tasks; -…
- ACAI code reviewer with Python agent called from Rust, streaming works2025 · medium.com · ▲6
Built an AI code reviewer using Letta (Python) that I can call natively from Rust applications. The interesting part: real-time streaming works perfectly across the language boundary with zero hassle using RunAgent. The agent runs in Python with persistent memory, leverages the best in house agentic memory management with Letta (Pythonic AI agent framework), and my rust code just uses it (kinda) natively, though Letta has no Rust bindings. And, streaming works like magic. No FFI, no complex bridges - just native async/streaming that feels like calling any Rust librar, but without…
- SASimilar application in different python frameworks2012 · blog.fruiapps.com · ▲12
More dev tools this month
the category →



Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com

OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com

Launched alongside, June 2024
the whole month →

La Growth Machine▲1,216Create personalized, multi-channel conversations at scale
AI · 2024 · lagrowthmachine.com


PyjamaHR▲1,132Hiring on autopilot. The AI applicant tracking system (ATS).
Work · 2024 · pyjamahr.com