An LLM Running on a PS Vita
Hello all, a couple of moons ago I ported karpathy's llama2.c code to run inference on the TinyStories 260K & 15M checkpoints on the on the PS Vita with the ability to download/delete the models on device. Runs showed that the 260K model ran at ~120 tok/s and at 15M ran at 1.8 tok/s, which probably could be a bit higher if it weren't a single threaded application. Had fun working on it as a weekend project, check it out for yourselves: https://github.com/callbacked/psvita-llm
An LLM Running on a PS Vita launched on June 15, 2025 with 5 votes and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
A small storytelling LLM running on the PS Vita. Contribute to callbacked/psvita-llm development by creating an account on GitHub.
“Your scientists were so preoccupied with whether or not they could, they didn't stop to think if they should.” Turns out it can! PSVita-LLM runs a modified version of llama2.c to load and infer the TinyStories 260K and 15M checkpoints right on the Vita. To build this project yourself, you will need a working VitaSDK installation. cmake . This project would not be possible without Andrej Karpathy's foundational work on llama2.c . What was done was more of a port job if anything, but a great way to learn! Breaking up some code: Having everything in a single file like the original llama2.c file is pretty cool, but I should have seperated the networking code as I feel like that could be used…from github.com
Does a similar job
all alternatives →
Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
I've built a locally running Perplexity clone2024 · github.com · ▲669The video demo runs a 7b Model on a normal gaming GPU. I think it already works quite well (accounting for the limited hardware power). :)
80% faster, 50% less memory, 0% loss of accuracy Llama finetuning2023 · github.com · ▲385Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
NobodyWhoAug 2026 · github.com · ▲106Hey, Hello, we are the team behind NobodyWho; an open-source plugin that embeds large language models functionality inside Unity and Godot games - completely locally and offline. We try to make it as easy and user friendly as possible to use our plugin, so feel free to create an issue on our GitHub if you have any inconveniences - and we will try to improve it! So why might this interest you: - Written in Rust with llama.cpp bindings and optional GPU back-ends (Vulkan / Metal) for fast inference. - Drop-in nodes / components: add a model asset and a chat or embedding node and get a…
KVSplit – Run 2-3x longer contexts on Apple Silicon2025 · github.com · ▲272I discovered that in LLM inference, keys and values in the KV cache have very different quantization sensitivities. Keys need higher precision than values to maintain quality. I patched llama.cpp to enable different bit-widths for keys vs. values on Apple Silicon. The results are surprising: - K8V4 (8-bit keys, 4-bit values): 59% memory reduction with only 0.86% perplexity loss - K4V8 (4-bit keys, 8-bit values): 59% memory reduction but 6.06% perplexity loss - The configurations use the same number of bits, but K8V4 is 7× better for quality This means you can run LLMs with 2-3× longer…
Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗
Launched alongside, June 2025
the whole month →- IM
Hey HN! Pilots everywhere are required to keep a logbook of all their flying hours, aircraft, airports, and so on. Since I track everything digitally (some people still just use paper logbooks!), I put together some data visualizations and a few 3D globes to show my flying history. This globe is probably my favourite so far: https://jameshard.ing/pilot/globes/all If you’ve got ideas for other graphs or ways to show this kind of data, I’d love to hear them!
Life & fun · 2025 · jameshard.ing
Stunning presentations with AI. No design skills required.
AI · 2025 · chroniclehq.com
Build native mobile apps for iOS and Android without code.
Dev tools · 2025 · bble.io
All your connections, across all your socials.
AI · 2025 · pally.com
I was the main contributor to workout.lol, an open-source fitness app to easily build a workout routine. The project had traction (1.4k GitHub stars, 95 forks, ~20K visits/month), but was eventually sold due to video licensing hurdles. The new owner stopped maintaining it, and the repo went abandoned. Over the next 9 months, I sent 15 emails to try to save it : no replies. Feature requests & issues were ignored. The community was left with a "broken" tool let's say. I couldn't just let it die So I built the new version from scratch with the same open-source spirit, but a better…
Dev tools · 2025 · github.com