Dev tools · alternatives · 2026

24 alternatives to Open-source load balancer for llama.cpp

Stateful load balancer customized for llama.cpp (with a reverse proxy).

Open-source load balancer for llama.cpp is a stateful load balancer designed specifically for llama.cpp that includes a reverse proxy. It helps distribute requests across multiple instances while maintaining state… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Open-source load balancer for llama.cpp launched in 2024; newer entries below may have overtaken it.

1. 1
   Llama▲312

   3.1-405B: an open source model to rival GPT-4o / Claude-3.5

   2024 · developer.meta.com · its alternatives →

2. 2
   Llama▲91

   A terminal file manager

   2021 · github.com · its alternatives →

3. 3
   Alpaca.cpp – Run an Instruction-Tuned Chat-Style LLM on a MacBook▲673

   Locally run an Instruction-Tuned Chat-Style LLM . Contribute to antimatter15/alpaca.cpp development by creating an account on GitHub.

   2023 · github.com · its alternatives →

4. 4
   LlamaCoder ▲373

   AI-powered coding assistant by Meta

   2024 · ai.meta.com · its alternatives →

5. 5
   PgDog▲227

   Scale PostgreSQL without changing your app

   Jul 2026 · pgdog.dev · its alternatives →

6. 6
   Llama 3.3 70B▲347

   Llama 405B-level performance, at a fraction of the cost

   2024 · developer.meta.com · its alternatives →

7. 7
   Open-source LLM provider price comparison▲125

   Looking for the cheapest place to deploy llama 3.1 model? Don't worry we have found it so you don't have to.

   2024 · github.com · its alternatives →

8. 8
   Llama 2▲263

   The next generation of Meta's open source LLM

   2023 · developer.meta.com · its alternatives →

9. 9
   Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model▲343

   High-speed download of LLaMA, Facebook's 65B parameter GPT model - shawwn/llama-dl

   2023 · github.com · its alternatives →

10. 10
    Llamao▲496

    Private & offline alternative to ChatGPT on your device

    2025 · llamao.app · its alternatives →

11. 11
    Llama Stack▲169

    Build Once and Deploy Anywhere

    2025 · github.com · its alternatives →

12. 12
    Llama Life▲284

    A fun, flexible task manager for desktop web

    2021 · llamalife.co · its alternatives →

13. 13
    PgDog – Scale Postgres without changing the app▲326

    Hey HN! Lev and Justin here, authors of PgDog (https://pgdog.dev/), a connection pooler, load balancer and database sharder for PostgreSQL. If you build apps with a lot of traffic, you know the first thing to break is the database. We are solving this with a network proxy that works without requiring application code changes or database migrations. Our post from last year: https://news.ycombinator.com/item?id=44099187 The most important update: we are in production. Sharding is used a lot, with direct-to-shard queries (one shard per query) working pretty much…

    Feb 2026 · github.com · its alternatives →

14. 14
    PgCat, Postgres pooler with sharding, load balancing and failover▲233

    PostgreSQL pooler with sharding, load balancing and failover support. - postgresml/pgcat

    2022 · github.com · its alternatives →

15. 15
    Get a Kubernetes LoadBalancer where you never thought it was possible▲61

    Get public TCP LoadBalancers for local Kubernetes clusters - inlets/inlets-operator

    2020 · github.com · its alternatives →

16. 16
    Llama▲163

    A fun, flexible, task manager for desktop web

    2020 · llamalife.co · its alternatives →

17. 17
    LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2▲225

    A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support! - getumbrel/llama-gpt

    2023 · github.com · its alternatives →

18. 18RL
    Run Llama.cpp In-Process from Java with Project Panama FFM▲6

    A week of benchmarks on a Xeon CPU box. TurboQuant ≠ 8× faster, speculative decoding ≠ universal win, ik_llama.cpp breaks parallel calls. Gemma-4-E4B-it on stock llama.cpp wins. Eleven cells, full source, MIT.

    Jun 2026 · deemwar-products.github.io · its alternatives →

19. 19
    Grammar Generator App for Llama.cpp▲19

    llama.cpp added context-free grammar guided generation functionality. It requires passing a file in a derivative of BNF notation, which gets messy very quickly for things like JSON. To improve the experience, we built a small compiler from TypeScript interfaces to the grammar file format and have it hosted in a little browser app. See more in discussion at https://github.com/ggerganov/llama.cpp/discussions/2494

    2023 · grammar.intrinsiclabs.ai · its alternatives →

20. 20
    A fully open-source (Apache 2.0)implementation of llama▲158

    We believe that AI should be fully open source and part of the collective knowledge. The original LLaMA code is GPL licensed which means any project using it must also be released under GPL. This "taints" any other code and prevents meaningful academic and commercial use. Lit-LLaMA solves that for good.

    2023 · github.com · its alternatives →

21. 21
    Llama – Terminal File Manager▲48

    Terminal file manager. Contribute to antonmedv/walk development by creating an account on GitHub.

    2022 · github.com · its alternatives →

22. 22
    Simple Load Balancer – Golang▲16

    Simple Load Balancer. Contribute to JackyChiu/slb development by creating an account on GitHub.

    2018 · github.com · its alternatives →

23. 23
    Llama▲65

    An online IDE for testing code 🛠️

    2019 · its alternatives →

24. 24
    A tiny TCP load balancer▲22

    A tiny TCP load balancer 🍸. Contribute to inlets/mixctl development by creating an account on GitHub.

    2022 · github.com · its alternatives →

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →