Alternatives
Products that do what Relai-SDK – simulate → evaluate → optimize AI agents does
What relai-sdk is an open-source toolkit for making AI agents reliable via a complete learning loop: simulate → evaluate → optimize. Why Agent runs are stochastic; tool-calls fail; hard to reproduce, measure, and fix at scale. It’s also hard to align behavior with goals across output quality/format, cost, and latency. We need a loop that integrates user feedback and LLM evaluators directly into the agent code (prompts, configs, models, graphs) without overfitting. How - Simulation: LLM personas, mocked MCP servers/tools, synthetic data; can condition on real traces - Evaluation:…
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9AJ
Hi HN! I’m Tony, co-founder of Inngest. I wanted to share AgentKit, our Typescript multi-agent library we’ve been cooking and testing with some early users in prod for months. Although OpenAI’s Agents SDK has been launched since, we think an Agent framework should offer more deterministic and flexible routing, work with multiple model providers, embrace MCP (for rich tooling), and support the unstoppable and growing community of TypeScript AI developers by enabling a smooth transition to production use cases. This is why we are building AgentKit, and we’re really excited about it for a few…
2025 · github.com
- 10DA
Write a task in plain English. An AI agent runs it on a simulator on your Mac and tells you if a real user could complete it. Save the successful run as a regression check you can replay later.
23d ago · app.deltix.ai
- 11PR
2025 · temporal.io
- 12AA
Hey HN! Previous CERN physicist turned hacker here. We've developed a way to make AI coding actually work by systematically identifying and fixing places where LLMs typically fail in full-stack development. Today we're launching as Lovable (previously gptengineer.app) since it's such a big change. The problem? AI writing code typically make small mistakes and then get stuck. Those who tried know the frustration. We fixed most of this by mapping out where LLMs fail in full-stack dev and engineering around those pitfalls with prompt chains. Thanks to this, in all comparisons I found with: v0,…
2024 · lovable.dev
- 13NA
Hi, I'm a Dapr CNCF project maintainer. We've recently released Dapr Agents which provides agentic AI features together with built-in durable execution to guarantee statefulness and reliable agentic workflows that run to completion and retry upon failure. It runs natively on Kubernetes, has built-in OTEL integration and uses a lightweight architecture where agents scale to zero, allowing you to run thousands of agents on commodity hardware. It'd be great if you can test it out and give us feedback.
2025 · github.com
- 14RG
Hey HN! I'm excited to share Reagent, an open source Javascript framework to build UI rich AI agents using graph based workflow. Github: https://github.com/reagentai/reagent There are already amazing tools for building agents but most of them seem to be focused on building the AI workflow on the backend with markdown as final output. There is generative UI in AI SDK from vercel but it seems to work only with server components afaik. So, I created a framework for building AI agents with interactive UI components; you can render any component from LLM tool, update it's…
2024 · github.com
- 15MA
We built meta-agent: an open-source library that automatically and continuously improves agent harnesses from production traces. Point it at an existing agent, a stream of unlabeled production traces, and a small labeled holdout set. An LLM judge scores unlabeled production traces as they stream. A proposer reads failed traces and writes one targeted harness update at a time, such as changes to prompts, hooks, tools, or subagents. The update is kept only if it improves holdout accuracy. On tau-bench v3 airline, meta-agent improved holdout accuracy from 67% to 87%. We open-sourced meta-agent.…
Apr 2026 · github.com
- 16RT
Mar 2026 · github.com
- 17CA
Hi all, I'm the creator and maintainer of Dapr. Today we announced an agentic AI framework that allows developers to run thousands of agents on a single core that can scale to/from zero with minimal latency, with a durable execution engine that supports automatic retries. Us maintainers would very much appreciate your feedback
2025 · github.com
- 18SS
Hi HN! We’re a team of ML validation specialists and we’ve been building /Spec27, a tool for testing whether AI agents still do their job safely and reliably as models, prompts, tools, and surrounding systems change. We started working on this because a lot of current LLM evaluation work seems aimed at scoring general model behavior, while many teams are deploying systems that have a specific mission to fulfill. Many of the tools also assume you have full access to the agent stack and traces so you can place SDKs and Gateways, but a lot of agents are being created on vendor platforms…
Apr 2026 · spec27.ai
- 19OS
GitHub: https://github.com/ClioAI/kw-sdk Most AI agent frameworks target code. Write code, run tests, fix errors, repeat. That works because code has a natural verification signal. It works or it doesn't. This SDK treats knowledge work like an engineering problem: Task → Brief → Rubric (hidden from executor) → Work → Verify → Fail? → Retry → Pass → Submit The orchestrator coordinates subagents, web search, code execution, and file I/O. then checks its own work against criteria it can't game (the rubric is generated in a separate call and the executor never sees it…
Feb 2026 · github.com
- 20MD
We’re excited to share ML-Dev-Bench, a new open-source benchmark that tests AI agents on real-world ML development tasks. Unlike typical coding challenges or Kaggle-style competitions, our benchmark simulates end-to-end ML workflows including: - Dataset handling and preprocessing - Debugging model and code failures - Implementing new model architectures - Fine-tuning and improving existing models With 30 diverse tasks, ML-Dev-Bench evaluates agents across critical stages of ML development. To complement this, we built Calipers, a framework that provides systematic performance evaluation and…
2025 · github.com
- 21OA
Steve from Temporal here. Temporal is an MIT open source project for reliable execution at scale. I adapted+extended some of OpenAI's Agents SDK samples to integrate with Temporal. These demo agents can survive process crashes, scale to millions of executions in parallel and have easy-to-implement human interactivity. Just add a couple of Python decorators to your OpenAI agent code, run Temporal workers and you're ready to go. Check the video I did with OpenAI showing this in action (it's linked in the repo). OpenAI actually use us for ChatGPT Images and also their Codex code writing agent…
2025 · github.com
- 22BA
I'm one of the creators of The Edge Agent (TEA). We built this because we needed a way to deploy agents that was verifiable and robust enough for production/edge cases, moving away from loose scripts. The architecture aims to solve critical gaps in deterministic orchestration identified by *Prof. Claudionor Coelho Jr. (Stanford alum, ML/DL Faculty at Santa Clara Univ., and Senior Fellow for AI at Majestic Labs)* during our work on the Kiroku project. *Key Technical Features:* * *Neurosymbolic Native:* We integrated Prolog to logically validate LLM outputs. This combines neural…
Jan 2026 · fabceolin.github.io
- 23WI
At Laminar (https://github.com/lmnr-ai/lmnr) we're building open source AI observability platform in Rust. We obsess over instrumentation DX for our Python and TS SDKs and in this new blog we outline how we made the most seamless way of instrumenting recently released claude agent sdk
Dec 2025 · laminar.sh
- 24RR
Hi HN, We built something new! It is an AI agent that can: - iterate through up to a dozen sources (web or GDrive) - reason between hops in natural language (exposed in the trace) - generate structured JSON so downstream code can consume results deterministically It not only delivers state-of-the-art performance on both SimpleQA and Reka Research-Eval, but also shatters the notion that cutting-edge AI must come at a premium, being significantly more affordable than alternatives. Tech stack highlights - Base model: Reka Flash 3.1 (trained from scratch, post-trained with RL on verifiable…
2025 · reka.ai
Ranked by how close each launch is in meaning, then by votes. Refine with a description →