Open-source load balancer for llama.cpp

In plain words

Open-source load balancer for llama.cpp is a stateful load balancer designed specifically for llama.cpp that includes a reverse proxy. It helps distribute requests across multiple instances while maintaining state, making it suitable for developers working with llama.cpp inference servers who need to manage traffic across multiple backend instances efficiently.

written from the facts on this page · September 2026

From the sources

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU. - intentee/paddler

• 
  Llama2024 · developer.meta.com · ▲312

  3.1-405B: an open source model to rival GPT-4o / Claude-3.5

• 
  Llama2021 · github.com · ▲91

  A terminal file manager

• 
  LlamaCoder 2024 · ai.meta.com · ▲373

  AI-powered coding assistant by Meta

• 
  PgDogJul 2026 · pgdog.dev · ▲227

  Scale PostgreSQL without changing your app

• 
  Open-source LLM provider price comparison2024 · github.com · ▲125

  Looking for the cheapest place to deploy llama 3.1 model? Don't worry we have found it so you don't have to.

• 
  Llama 22023 · developer.meta.com · ▲263

  The next generation of Meta's open source LLM

• 
  /dev/null▲468

  a social network for devs

  Dev tools · 2d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 25d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 22d ago · openbot.run

• 
  Talanoa▲340

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 20d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 27d ago · corespeed.io

• 
  Pygma▲1,412

  AI copilot for Instagram

  AI · 2024 · pygma.me

• 
  Cello▲1,347

  Easiest way to add a referral program to your SaaS

  Growth · 2024 · cello.so

• 
  La Growth Machine▲1,216

  Create personalized, multi-channel conversations at scale

  AI · 2024 · lagrowthmachine.com

• 
  Superhuman 2.0▲1,142

  Revolutionary AI email, reimagined for teams

  AI · 2024 · superhuman.com

• 
  FlowMapp 3.0▲1,142

  Visual website planning in the most powerful way

  Work · 2024 · flowmapp.com

• 
  PyjamaHR▲1,132

  Hiring on autopilot. The AI applicant tracking system (ATS).

  Work · 2024 · pyjamahr.com