We made a 2.5GB Offline disaster AI assistant [video]
It is a prototype for Gemma 3n Impact Challenge hosted by DeepMind. We don't have experience on local LLM before, so it is a pretty fun learning experience. Hope to see more lightweight llm model in the future!
Does the same job
all alternatives →- PSPacific Slate: a self-hosted, model-agnostic multi-agent AI assistant27d ago · pacslate.com · ▲5
I didn't want to buy a standalone computer or repurpose a laptop to run constantly so I could maintain a system to sync my LLMs, so I built this. It's a simple overview of my system, laid out in a way easy to unpack and replicate for yourself. The project is meant to be configured individually, and uniquely, since one solution might not be what's best for another. If anything, maybe it gives you some ideas on how to implement things for your own project. Best wishes, Ryan.
- RARun any Llama model finetune and more, instantly2024 · featherless.ai · ▲7
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
- IMI made AI agent lib that you will understand2025 · github.com · ▲7
Every time I wanted to use LLMs in my existing pipelines the integration was very bloated, complex, and too slow. This is why I created a lightweight library that works just like scikit-learn, the flow generally follows a pipeline-like structure where you “fit” (learn) a skill from sample data or an instruction set, then “predict” (apply the skill) to new data, returning structured results. High-Level Concept Flow Your Data --> Load Skill / Learn Skill --> Create Tasks --> Run Tasks --> Structured Results --> Downstream Steps And the bast part: Every step can be saved and reused as…
- ATA tiny reasoning layer that steadies LLM outputs (MIT; +22.4% accuracy)2025 · github.com · ▲6
We kept shipping “simple” LLM features that were fluent-but-wrong. After too many postmortems we wrote down the failure patterns and added a small reasoning layer in front of the model. It’s model-agnostic, sits beside your existing stack, and you can implement it from a single PDF (MIT). What’s inside the PDF A problem map of 16 failure modes we kept hitting in real systems (OCR/layout drift, table-to-question mismatches, embedding≠meaning, pre-deploy collapse, etc.). Four lightweight gates you can add today: Knowledge-boundary canaries (empty/adversarial/known-fact probes).…
- LCLocal Coding Agent with LLMs to Delegate Tool Calls to Small AI ModelsMay 2026 · github.com · ▲9
- HPHorizon – Programmatic Prompt Generation and LLM Configurations2023 · gethorizon.ai · ▲7
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, August 2025
the whole month →
- IS
I built the world's most impractical 1000-pixel display and anyone in the world can draw on it. It draws a single pixel at a time and takes 30-60 minutes to complete a single image. Anyone can participate in the project by voting for the next image to be drawn, and submitting images. https://kilopx.com/
Work · 2025 · benholmen.com
- KT
Kitten TTS is an open-source series of tiny and expressive text-to-speech models for on-device applications. We are excited to launch a preview of our smallest model, which is less than 25 MB. This model has 15M parameters. This release supports English text-to-speech applications in eight voices: four male and four female. The model is quantized to int8 + fp16, and it uses onnx for runtime. The model is designed to run literally anywhere eg. raspberry pi, low-end smartphones, wearables, browsers etc. No GPU required! We're releasing this to give early users a sense of the latency and voices…
Dev tools · 2025 · github.com
- IW
I was wondering how I can arrange objects along a spherical helix path, and read some articles on it. I ended up learning about parametric equations again, and make this visualization to document what I learned: https://visualrambling.space/moving-objects-in-3d/ feel free to visit and let me know what you think!
Life & fun · 2025 · visualrambling.space
- TC
For HTML Day 2025 [1], I made a web service that displays the current sky at your approximate location as a CSS gradient. Colours are simulated on-demand using atmospheric absorption and scattering coefficients. Updates every minute, without the use of client-side JavaScript. Source code and additional information is available on GitHub: https://github.com/dnlzro/horizon [1] https://html.energy/html-day/2025/index.html
Dev tools · 2025 · sky.dlazaro.ca