Alternatives
Products that do what Compare Groq and Llama-3-70B with GPT-4 Turbo does
Just added support for Llama-3 models to our AI app platform Promptly. We decided to try Groq cloud for powering these models and the results have so far been pretty good comparing Llama-3-70B with GPT-4 Turbo. Put an app together to compare these models. Check it out at https://trypromptly.com/a/groq-llama-3-70b-vs-gpt-4-turbo. https://trypromptly.com/s/iQG7EoJ4Pm is a sample output comparison between Llama-3-70B and GPT-4 turbo. https://youtu.be/1UChY6EDwFA shows the inference speed of Groq compared to GPT-4.
- 1
- 2FL
I've been playing around with https://github.com/zphang/minimal-llama/ and https://github.com/tloen/alpaca-lora/blob/main/finetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU. To prepare the data, simply separate your text with two blank lines. There's an inference tab, so you can test how the tuned model behaves. This is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio. Enjoy!
2023 · github.com
- 3
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…
Jul 2026 · github.com
- 48F
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com
- 5
- 6LD
2023 · github.com
- 7TL
Hey HN, we wanted to share our repo where we fine-tuned Llama 3.1 on Google TPUs. We’re building AI infra to fine-tune and serve LLMs on non-NVIDIA GPUs (TPUs, Trainium, AMD GPUs). The problem: Right now, 90% of LLM workloads run on NVIDIA GPUs, but there are equally powerful and more cost-effective alternatives out there. For example, training and serving Llama 3.1 on Google TPUs is about 30% cheaper than NVIDIA GPUs. But developer tooling for non-NVIDIA chipsets is lacking. We felt this pain ourselves. We initially tried using PyTorch XLA to train Llama 3.1 on TPUs, but it was rough: xla…
2024 · github.com
- 8L3
Hi everyone, I'm kinda involved in some retrogaming and with some experiments I ran into the following question: "It would be possible to run transformer models bypassing the cpu/ram, connecting the gpu to the nvme?" This is the result of that question itself and some weekend vibecoding (it has the linked library repository in the readme as well), it seems to work, even on consumer gpus, it should work better on professional ones tho
Feb 2026 · github.com
- 9SU
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org
- 10
- 11IR
2023 · sagittarius.greg.technology
- 12
General Compute▲315AI models that run on an inference cloud optimized for speed
May 2026 · generalcompute.com
- 13
Fast and efficient models optimized for coding and subagents
Mar 2026 · openai.com
- 14LS
2023 · github.com
- 15OS
Looking for the cheapest place to deploy llama 3.1 model? Don't worry we have found it so you don't have to.
2024 · github.com
- 16
- 17
- 18WM
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com
- 19TC
Hi HN, I spent my easter weekend stuck in the house with COVID and I decided to play with llama.cpp [1] and fauxpilot [2] to see if I could get LLM code assist working on pure CPU. As a proof of concept I'd say I've proven that it's possible. However there's still a lot to do. The auto complete is quite slow at the moment. PRs welcome. [1] https://github.com/ggerganov/llama.cpp [2] https://github.com/fauxpilot/fauxpilot
2023 · github.com
- 20
- 21LL
Hey, recently I took inspiration from llama.cpp, ollama, and many other similar tools that enable inference of LLMs locally, and I just finished building a Llama inference engine for the 8B model in CUDA C. I recently wanted to explore my newly founded interest in CUDA programming and my passion for machine learning. This project only makes use of the native CUDA runtime api and cuda_fp16. The inference takes place in fp16, so it requires around 17-18GB of VRAM (~16GB for model params and some more for intermediary caches). It doesn’t use cuBLAS or any similar libraries since I wanted to be…
2025 · github.com
- 22FL
2024 · colab.research.google.com
- 23CA
Hey folks, we created a ChatGPT alternative for anyone to try out LLaMA models and benchmark responses across OpenAI's models and other open source models. Would love some feedback! No data is being used for retraining models.
2023 · chat.nbox.ai
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →