Alternatives
Products that do what Python lib to run evals across providers: OpenAI, Anthropic, etc. does
Library makes requests asynchronously across models, so you can spend a lot of $$ quickly if you want XD. But seriously I hope this enables folks to create and run evals (especially safety ones) a lot easier than before.
- 1
- 2CA
We open-sourced catsu, a Python client for embedding APIs. The problem: every embedding provider has a different SDK with different bugs. OpenAI has undocumented token limits. VoyageAI's retry logic was broken until September. Cohere breaks downstream libraries every release. LiteLLM's embedding support is minimal. catsu provides: - One API for 11 providers (OpenAI, Voyage, Cohere, Jina, Mistral, Gemini, etc.) - Bundled database of 50+ models with pricing, dimensions, and benchmark scores - Built-in retry with exponential backoff - Automatic cost tracking per request - Full async support…
Dec 2025 · catsu.dev
- 3AB
Hello HN, new user here, so please let me know if I break some rules. Currently I've been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it. I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested. For more details, please visit: https://github.com/Chimpan-Z/agymc Feedback really appreciated! Have a good day everyone!
2020
- 4OA
https://github.com/gugarosa/opytimizer Did you ever reach a bottleneck in your computational experiments? Are you tired of selecting suitable parameters for a chosen technique? If yes, Opytimizer is the real deal! This package provides an easy-to-go implementation of meta-heuristic optimizations. From agents to search space, from internal functions to external communication, we will foster all research related to optimizing stuff. Use Opytimizer if you need a library or wish to: - Create your optimization algorithm; - Design or use pre-loaded optimization tasks; -…
2021
- 5AC
Built an AI code reviewer using Letta (Python) that I can call natively from Rust applications. The interesting part: real-time streaming works perfectly across the language boundary with zero hassle using RunAgent. The agent runs in Python with persistent memory, leverages the best in house agentic memory management with Letta (Pythonic AI agent framework), and my rust code just uses it (kinda) natively, though Letta has no Rust bindings. And, streaming works like magic. No FFI, no complex bridges - just native async/streaming that feels like calling any Rust librar, but without…
2025 · medium.com
- 6SA
2012 · blog.fruiapps.com
- 7RA
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
- 8AC
2025 · niquests.readthedocs.io
- 9MM
I kept on reading you guys on how to do it, how not to do it. On January I started with thinkpython, learn python the hard way, mit course on programming, and every other python material I found. You just need to find the best resourse for YOU. I did Udacity's CS253 course, got my certificate and made my MVP! If your one of those guys like me, just get your ideas out there and the project will push you forward. Farm management software in South America is broken. We still have cd and downloadable updates. We've seen www.farmlogs.com and www.farmeron.com for USA and Europe. How about…
2012
- 10PC
Hi HN, We needed a simple way to connect to the top AI models to experiment, prototype and evaluate them. Main features: - Connect to top LLMs in few lines of code (currenly OpenAI, Anthropic and AI21 are supported) - Response meta includes tokens processed, cost and latency standardized across the models - Multi-model support: Get completitions from different models at the same time - LLM benchmark: Eevaluate models on quality, speed and cost The benchmark uses predefine questions to test AI reasoning abilities across a range of "hard" queries. The outputs are then automatically evaulauted…
2023 · github.com
- 11MA
Hi, I'm working on a project that regroups all best AI (AIaaS) from different providers (GCP, AWS, Azure, DeepL, etc.) in one API (https://github.com/edenai/edenai-apis). I've got asked the question : why aren't you regrouping Open Source models (instead of proprietary APIs) into one repo? Well because it doesn't make sens to deploy and maintain large pytorch (or other framework) AI models (especially for document parsing, image and video moderation or speech recognition) in every solution that wants AI capabilities. So using APIs makes way more sens. Deployed OpenSource…
2023 · github.com
- 12UI
Hey everyone! I am excited to share updates on four of my & my teams' open-source projects that take large-scale search systems to the next level: USearch, UForm, UCall, and StringZilla. These projects are designed to work seamlessly together, end-to-end—covering everything from indexing and AI to storage and networking. And yeah, they're optimized for x86 AVX2/512 and Arm NEON/SVE hardware. USearch [1]: Think of it as Meta FAISS on steroids. It's now quicker, supports clustering of any granularity, and offers multi-index lookups. Plus, it's got more native bindings than probably…
2023 · usearch-images.com
- 13AM
I have built many AI agents, and all frameworks felt so bloated, slow, and unpredictable. Therefore, I hacked together a minimal library that works with JSON/dict/kwargs definitions for each step, allowing you a simpler way to define reproducible agents. It supports concurrency for up to 1000 calls/min, giving you speed and predictability in your workflows. Install pip install flashlearn Input is a list of dictionaries Simply take user inputs, API responses, and calculations from other tools and feed them to FlashLearn. user_inputs = [{"query": "When was python launched?"}]…
2025 · github.com
- 14AO
Hi It's Logan (op) and David, we are launching our first open source Python SDK generator with other languages to come :) We built this because we believe that the OpenAPI Generator project is a mess and simply doesn't work for most users (us included). Additionally, proprietary generators miss the mark for complex use cases, and restrict developers from doing what they do best: writing code. On top of all that they force you into proprietary configuration layers for many desirable features which increases vendor lock in and pulls you further away from your API spec as the source of truth.…
2025 · github.com
- 15HA
Demo starts at 50m into the video. This was a bit terrifying to record because 2am the previous night everything was totally broken after a major refactor (so that we could add external LLM support as well as local GPUs). But pressure can be a useful force :-D We start with a stack deployed on my laptop without a GPU, pointing to together.ai so we can run open source LLMs easily without having to have access to a GPU. We show simple inference through the ChatGPT-like web interface (with users, sessions etc) and then simple drag'n'drop RAG. Then we show some helix apps defined as yaml: Marvin…
2024 · youtube.com
- 16LA
Hey everyone, we just released a new version of our Infrastructure as Code Python SDK that now runs entirely on your machine - no account required. LaunchFlow extends OpenTofu to provide features like multi-environment support [1], autoconfigured infrastructure clients [2], and API release management [3]. Our goal is to provide a Python-native infrastructure toolkit that handles everything from automating your infrastructure to using it in your application code. You can use our library of preconfigured OpenTofu modules to add GCP / AWS infrastructure to your app with just a few lines of…
2024 · launchflow.com
- 17OS
Companion is a free, open-source web app, featuring a Python REPL environment with an AI Tutor designed to support one’s learning and problem-solving in programming. I am leveraging the Hermes 3 405B model from Nous Research, hosted on Lambda’s Inference API. It's community-driven, 100% free, and open to all. I’d love your feedback and suggestions. Here's a short video where I demo the tool: https://www.youtube.com/watch?v=4Plt_sh_cIg&ab_channel=Rahul
2024 · companionai.dev
- 18MY
LLM observability is an absolute must-have for anyone running something in prod (or prod-like). While all the observability startups are great, you're essentially sending all your OpenAI usage history - prompts, generations, chats - to a random third party. So this script deploys a basic proxy in your Azure account, catches all incoming OpenAI requests, stores logs in your own resource group, and comes with visualizations premade (charts, timelines, chat history, cost estimation, etc). Thanks for any thoughts and feedback!
2023 · github.com
- 19AE
I've been working on a site [1] to give people control of their LLM workflows through AI evals - automated checks that, once defined, let you move fast without regressions and cut through hype with proof. That one-liner is aimed at software engineers, but I've spent my career helping cross-functional teams collaborate, and that's really what this is about. AI agents make powerful workflows very plausible, but only if teams can grow them incrementally without losing control - no vendor lock-in, no discipline silos, no blind trust in outputs. The site tries to meet different audiences where…
Feb 2026 · ai-evals.io
- 20BO
Read the full blogpost at https://rach.codes/blog/Introducing-Bhumi (click on reader to see the technical breakdown!) AI inference should be fast, but in practice it’s painfully slow. Inference bottlenecks slow down LLM-powered chatbots and AI workflows everywhere. I built Bhumi to fix that. Bhumi is a Python library designed for developers, yet its performance-critical core is implemented in Rust (via PyO3) for near-native speed. This hybrid approach delivers up to 2.5x faster response times across providers like OpenAI, Anthropic, and Gemini—without changing the…
2025 · bhumi.trilok.ai
- 21OA
Scenario: Your company’s IT department says “good news, you have access to azure, aws, openai, mistral, and together AI, here are the API keys”. You think “yippee I can access many models”, but some models like the gpt-oss or Mistral are available on some or all of those platforms? That’s where this app comes in: run it and it will check all the providers that you have configured and then you can search across those providers to see which providers have the model you want available. Built on top of mozilla.ai any-llm library. Check out the link for a GIF showing it in action.
Sep 2025 · github.com
- 22IB
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com
- 23AC
Hi everyone, I've been working on a CLI tool that can help to easily run any model in claude, Codex, Gemini, Pi, and OpenCode. It's also an API keys manager, supports multiple providers or OpenAI/Claude/Gemini accounts. You can add openrouter, poe, Vercel AI gateways etc. It has a built-in provider that is free to all, which is using Deepseek-V4, no login or API key required, add your own when you're ready. After installation you can try claude instantly (No config, no login): aivo claude Hope it's useful to someone.
Apr 2026 · getaivo.dev
- 24IG
As a beginner learning to build ML models, I found it annoying to have to keep printing tensor shapes every other line, having to step through the debugger to check where did I mess up the shapes again. So I built Trickle, it takes the data that flows through your code, caches the types and display them inline (as if you have type annotations). The idea is: "Let types trickle from runtime into your IDE". You get types in Python without having the write them manually. It works by rewriting your Python AST at import time — after every variable assignment, it inserts a lightweight call that…
Mar 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →