AI · alternatives · 2026
24 alternatives to Pipevals
Evaluation pipelines for every LLM application
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
Hey HN! Pipevals is early and rough (this is a learning project), but usable. It currently lets you: - build evaluation pipelines as graphs - run them against datasets - track how output quality changes over time
Mar 2026 · github.com · its alternatives →
- 2
- 3
We’re Robin, Louis, and Thomas. Pipelex is a DSL and a Python runtime for repeatable AI workflows. Think Dockerfile/SQL for multi-step LLM pipelines: you declare steps and interfaces; any model/provider can fill them. Why this instead of yet another workflow builder? - Declarative, not glue code: you state what to do; the runtime figures out how. - Agent-first: each step carries natural-language context (purpose, inputs/outputs with meaning) so LLMs can follow, audit, and optimize. Our MCP server enables agents to run pipelines but also to build new pipelines on demand. - Open…
Oct 2025 · github.com · its alternatives →
- 4
Get and manipulate data with a visual programming interface.
2017 · pipes.digital · its alternatives →
- 5
Evidently AIfree▲382Open-source evaluations and observability for LLM apps
2024 · evidentlyai.com · its alternatives →
- 6
I've been obsessed for the past ~year with the possibilities of talking to LLMs. I built a bunch of one-off prototypes, shared code on X, started a Meetup group in SF, and co-hosted a big hackathon. It turns out that there are a few low-level problems that everybody building conversational/real-time AI needs to solve on the way to building/shipping something that works well: low-latency media transport, echo cancellation, voice activity detection, phrase endpointing, pipelining data between models/services, handling voice interruptions, swapping out different…
2024 · github.com · its alternatives →
- 7
buildpipe▲116Compose, run and automate multi step AI developer workflows
May 2026 · buildpipe.com · its alternatives →
- 8
- 9
Hey HN! We’re Kevin and Steve. We’re building PromptTools (https://github.com/hegelai/prompttools): open-source, self-hostable tools for experimenting with, testing, and evaluating LLMs, vector databases, and prompts. Evaluating prompts, LLMs, and vector databases is a painful, time-consuming but necessary part of the product engineering process. Our tools allow engineers to do this in a lot less time. By “evaluating” we mean checking the quality of a model's response for a given use case, which is a combination of testing and benchmarking. As examples: - For generated…
2023 · github.com · its alternatives →
- 10
- 11
Validate, monitor, and safeguard LLM-based apps
2023 · deepchecks.com · its alternatives →
- 12
- 13
Ludwig 0.8▲109Build and fine-tune custom LLMs on your private data
2023 · ludwig.ai · its alternatives →
- 14
APIEval-20▲121An open benchmark for AI agents that test APIs
May 2026 · resources.kusho.ai · its alternatives →
- 15PB
2014 · pipesql.com · its alternatives →
- 16
Mage AI helps data teams build, orchestrate, monitor, and repair production data pipelines with editable SQL and Python for analytics, applications, and AI.
2022 · mage.ai · its alternatives →
- 17
- 18
Our goal with this project is to build a completely open source, state of the art turn detection model that can be used in any voice AI application. I've been experimenting with LLM voice conversations since GPT-4 was first released. (There's a previous front page Show HN about Pipecat, the open source voice AI orchestration framework I work on. [1]) It's been almost two years, and for most of that time, I've been expecting that someone would "solve" turn detection. We all built initial, pretty good 80/20 versions of turn detection on top of VAD (voice activity detection) models. And…
2025 · github.com · its alternatives →
- 19
- 20
Hey HN! We are building *open source infrastructure for deploying customer-facing data pipelines.* Here’s our repo https://github.com/pipebird/pipebird and website https://pipebird.com/. Pipebird (YC W22) is designed to enable companies that generate important data to offer secure data pushes to their customers’ warehouses, directly from their products. Our team was previously building in fintech, where we heard from many of our peers that their customers wanted data pushed directly to their warehouses. Customers wanted to bring data into their source of…
2022 · github.com · its alternatives →
- 21
- 22
Seeknal▲57Data & AI/ML CLI for pipelines and NL queries
Apr 2026 · seeknal.exe.xyz · its alternatives →
- 23
Agent evals and guardrails in one request. Built on Jev, Kev and Laya. - openlayer-ai/jevals
Sep 2026 · github.com · its alternatives →
- 24AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
Also compare
- Pipevals – a visual pipeline builder for evaluation-driven AI alternatives
- Superpipe alternatives
- Pipelex – Declarative language for repeatable AI workflows alternatives
- Early-stage Yahoo Pipes spiritual successor alternatives
- Evidently AI alternatives
- An open source framework for voice assistants alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →