Open-source load balancer for llama.cpp

Stateful load balancer customized for llama.cpp (with a reverse proxy).

Visit github.com ↗Alternativestop 19% of June 2024

In plain words

Open-source load balancer for llama.cpp is a stateful load balancer designed specifically for llama.cpp that includes a reverse proxy. It helps distribute requests across multiple instances while maintaining state, making it suitable for developers working with llama.cpp inference servers who need to manage traffic across multiple backend instances efficiently.

written from the facts on this page · September 2026

More dev tools this month

the category →

Dograh▲592
The open source VAPI alternative
Dev tools · 25d ago · dograh.com
Meridian▲530
Don't let your work go unnoticed. Get promoted!
Dev tools · 20d ago · meridiona.com
x1▲516
Lovable for iPhone apps go from idea to App Store
Dev tools · 11d ago · x1.new
The GTM Co-Founder▲402
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
Open-Source eInk Bike Computer▲385
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Nuphos▲380
The AI-Native DevOps Workspace.
Dev tools · 24d ago · nuphos.ai

Launched alongside, June 2024

the whole month →

Pygma▲1,409
AI copilot for Instagram
AI · 2024 · pygma.me
Cello▲1,347
Easiest way to add a referral program to your SaaS
Growth · 2024 · cello.so
La Growth Machine▲1,216
Create personalized, multi-channel conversations at scale
AI · 2024 · lagrowthmachine.com
FlowMapp 3.0▲1,147
Visual website planning in the most powerful way
Work · 2024 · flowmapp.com
Superhuman 2.0▲1,143
Revolutionary AI email, reimagined for teams
AI · 2024 · superhuman.com
PyjamaHR▲1,132
Hiring on autopilot. The AI applicant tracking system (ATS).
Work · 2024 · pyjamahr.com