Inference API that adapts to your SLA and quality constraints
Hi HN, I'm one of the creators of Exosphere. Think of us like a reliability lab for agents. Today we are launching Exosphere Flex Inference APIs: Inference APIs should adapt to your constraints, not the other way around. Usually, when you need to run inference at scale, you are forced into rigid boxes: 1. "Real-time" APIs (Expensive, optimized for <1s latency, prone to 429s). 2. "Batch" APIs (Cheaper, but often force 24-hour windows and rigid file formats). 3. "Self-hosted" (Total control, but high ops overhead). We built a flexible inference engine that sits in the middle. You define the…
What it does
In the maker’s words, at launch
Hi HN, I'm one of the creators of Exosphere. Think of us like a reliability lab for agents. Today we are launching Exosphere Flex Inference APIs: Inference APIs should adapt to your constraints, not the other way around. Usually, when you need to run inference at scale, you are forced into rigid boxes: 1. "Real-time" APIs (Expensive, optimized for <1s latency, prone to 429s). 2. "Batch" APIs (Cheaper, but often force 24-hour windows and rigid file formats). 3. "Self-hosted" (Total control, but high ops overhead). We built a flexible inference engine that sits in the middle. You define the constraints—SLA (time), Cost, and Quality and the system handles the execution. Here is how it works under the hood: 1. Flexible SLAs (The "Time" Constraint): Instead of just "now" or "tomorrow," you pass an `sla` parameter (e.g., 60 minutes, 4 hours). Our scheduler bins these requests to optimize GPU saturation across our provider mesh. You trade strict immediacy for up to ~70% lower cost. 2. Reliability Layer (The "Ops" Constraint): We abstract away the error handling. If a provider throws a 429 or 503, you shouldn't have to write a retry loop with backoff jitter. Our infrastructure absorbs these failures and retries internally. We guarantee the request eventually succeeds (within your SLA) or we don't charge you. 3. Built-in Quality Gates (The "Accuracy" Constraint): This is the feature I’m most excited about. You can define an "eval" config in the request (using LLM-as-a-Judge or python scripts). If the output doesn't meet your criteria, our system automatically feeds the failure back into the model and retries it. This moves the "validation loop" from your client code into the infrastructure. I’d love to hear your thoughts on this approach—specifically, does moving the "retry/eval" loop into the API layer simplify your backend, or do you prefer keeping that logic client-side? Playground: https://models.exosphere.host/ More Details: https://exosphere.host/flex-inference
Does the same job
all alternatives →
- FSFull Stack Starter Repo Optimized2025 · abeahmed.com · ▲6
Hi everyone! I've been loving building with AI, and over the past few years I've been leaning more and more into Typescript (and bun). My team at inference.net is constantly trying to get more leverage out of AI and find ways to setup our codebase to be able to increase the level of correctness that our AI is able to write code at. This starter repo is a very opinionated way to lay out a repo to lean into AI heavily. It leverages Cloudflare Workers as a deployment target for the API (my goal is to never have to deploy an API on a AWS/Azure/GCP server ever again unless I get to a…
- CMComposable middleware for LLM inference Optimization PassesMar 2026 · github.com · ▲7
Hey HN, I've been building AutoAgents, an AI agent framework in Rust. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines. The problem Every agent framework lets you swap LLM providers. Almost none of them give you a structured way to enforce safety, caching, or data sanitization in the inference path itself. You end up with guardrails as application-level if-statements, caching bolted on as a separate service, and PII handling as a "we'll add it later" TODO that never ships. This gets worse with local models. Cloud APIs…
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
