Distributed Llama – Run LLMs on multiple devices in parallel
Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference. - b4rtaz/distributed-llama
Distributed Llama – Run LLMs on multiple devices in parallel launched on January 25, 2024 with 12 votes, #1,193 of 4,438 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Distributed Llama is a tool that enables large language models to run across multiple devices simultaneously. It allows users to parallelize LLM inference, making it possible to process requests faster by distributing the computational load. The software is designed for developers and users who want to improve performance when working with large language models. It's built as an open-source project available on GitHub.
written from the facts on this page · September 2026
Does a similar job
all alternatives →
Llama 3.3 70B2024 · developer.meta.com · ▲347Llama 405B-level performance, at a fraction of the cost
Llama-dl – high-speed download of LLaMA, Facebook's 65B GPT model2023 · github.com · ▲343High-speed download of LLaMA, Facebook's 65B parameter GPT model - shawwn/llama-dl
Finetune LLaMA-7B on commodity GPUs using your own text2023 · github.com · ▲449I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU. To prepare the data, simply separate your text with two blank lines. There's an inference tab, so you can test how the tuned model behaves. This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio. Enjoy!
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 28d ago · world.pixverse.video
Source: Show HN post ↗
Launched alongside, January 2024
the whole month →
Amie▲1,334Turn meeting notes into automated workflows. Handle meetings, summaries, todos and emails (soon) with your AI personal assistant.
Work · 2024 · amie.so
- IM
Sit in Shade - Discover the optimal seat to stay in the shade during your journey. Sit In Shade provides real-time sun exposure data and route visualization.
Life & fun · 2024 · sitinshade.com
