nowfound

Dev tools · June 22, 2024

PL

Python lib to run evals across providers: OpenAI, Anthropic, etc.

Library makes requests asynchronously across models, so you can spend a lot of $$ quickly if you want XD. But seriously I hope this enables folks to create and run evals (especially safety ones) a lot easier than before.

Does the same job

all alternatives →
  • Clevrr Computer2024 · ▲195

    Computer use but with OpenAI and Gemini models

  • CA
    Catsu: A unified Python client for embedding APIsDec 2025 · catsu.dev · ▲7

    We open-sourced catsu, a Python client for embedding APIs. The problem: every embedding provider has a different SDK with different bugs. OpenAI has undocumented token limits. VoyageAI's retry logic was broken until September. Cohere breaks downstream libraries every release. LiteLLM's embedding support is minimal. catsu provides: - One API for 11 providers (OpenAI, Voyage, Cohere, Jina, Mistral, Gemini, etc.) - Bundled database of 50+ models with pricing, dimensions, and benchmark scores - Built-in retry with exponential backoff - Automatic cost tracking per request - Full async support…

  • AB
    A batched, concurrent wrapper for OpenAI gym library2020 · ▲12

    Hello HN, new user here, so please let me know if I break some rules. Currently I've been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it. I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested. For more details, please visit: https://github.com/Chimpan-Z/agymc Feedback really appreciated! Have a good day everyone!

  • OA
    Opytimizer – A Nature-Inspired Python Optimizer2021 · ▲23

    https://github.com/gugarosa/opytimizer Did you ever reach a bottleneck in your computational experiments? Are you tired of selecting suitable parameters for a chosen technique? If yes, Opytimizer is the real deal! This package provides an easy-to-go implementation of meta-heuristic optimizations. From agents to search space, from internal functions to external communication, we will foster all research related to optimizing stuff. Use Opytimizer if you need a library or wish to: - Create your optimization algorithm; - Design or use pre-loaded optimization tasks; -…

  • AC
    AI code reviewer with Python agent called from Rust, streaming works2025 · medium.com · ▲6

    Built an AI code reviewer using Letta (Python) that I can call natively from Rust applications. The interesting part: real-time streaming works perfectly across the language boundary with zero hassle using RunAgent. The agent runs in Python with persistent memory, leverages the best in house agentic memory management with Letta (Pythonic AI agent framework), and my rust code just uses it (kinda) natively, though Letta has no Rust bindings. And, streaming works like magic. No FFI, no complex bridges - just native async/streaming that feels like calling any Rust librar, but without…

  • SA
    Similar application in different python frameworks2012 · blog.fruiapps.com · ▲12

More dev tools this month

the category →
  • Dograh592

    The open source VAPI alternative

    Dev tools · 25d ago · dograh.com

  • Meridian530

    Don't let your work go unnoticed. Get promoted!

    Dev tools · 20d ago · meridiona.com

  • x1516

    Lovable for iPhone apps go from idea to App Store

    Dev tools · 11d ago · x1.new

  • Open-source GTM skills for technical founders

    Dev tools · 29d ago · gtmcofounder.com

  • OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.

    Dev tools · 1d ago · opentrailpaper.com

  • Nuphos380

    The AI-Native DevOps Workspace.

    Dev tools · 24d ago · nuphos.ai

Launched alongside, June 2024

the whole month →
  • Pygma1,409

    AI copilot for Instagram

    AI · 2024 · pygma.me

  • Cello1,347

    Easiest way to add a referral program to your SaaS

    Growth · 2024 · cello.so

  • Create personalized, multi-channel conversations at scale

    AI · 2024 · lagrowthmachine.com

  • FlowMapp 3.01,147

    Visual website planning in the most powerful way

    Work · 2024 · flowmapp.com

  • Revolutionary AI email, reimagined for teams

    AI · 2024 · superhuman.com

  • PyjamaHR1,132

    Hiring on autopilot. The AI applicant tracking system (ATS).

    Work · 2024 · pyjamahr.com