Pruna AI – Inference Optimization Engine
Hello Hacker News! I am Bertrand from Pruna AI. With my associates, John, Rayan, and Stephan, we are fellow researchers in AI efficiency and reliability coming from TUM. We are building an optimization engine that combines compression methods (e.g. quantization, pruning, compilation, batching…) in the aim of saving compute power when running AI models. This optimization engine take one base model as input and returns a compressed model as output. It aims to help for two things: - Make various AI models faster and/or smaller for various hardware (because they can require significant…
Pruna AI – Inference Optimization Engine launched on December 9, 2024 with 6 votes and more than 50% of that year's launches.
What it does
In the maker’s words, at launch
Hello Hacker News! I am Bertrand from Pruna AI. With my associates, John, Rayan, and Stephan, we are fellow researchers in AI efficiency and reliability coming from TUM. We are building an optimization engine that combines compression methods (e.g. quantization, pruning, compilation, batching…) in the aim of saving…
Does a similar job
all alternatives →
We made our own inference engine for Apple Silicon2025 · github.com · ▲186We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
I ran a language model on a PS2Mar 2026 · github.com · ▲46The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
Serve 100 Large AI models on a single GPU with low impact to TTFTNov 2025 · github.com · ▲7I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm. I started experimenting with Serverless AI inference, but found out that coldstarts were huge. I went deep into the research and put together an engine that loads large models from SSD to VRAM up to ten times faster than alternatives. It works with vLLM, and transformers, and more coming soon. With this project you can hot-swap entire large models (32B) on demand. Its great for: Serverless AI Inference Robotics On Prem deployments Local Agents And Its open source. Let me know if anyone…
- CACoding Agents swarming your codebase2025 · infrastructureas.ai · ▲9
Imagine you have to make a little change in all your microservices, like changing a line in Dockerfile, but you have 50 microservices, ouch right! The PRs, pipelines, reviews... I have created a dashboard with a chatbot where you can simply ask "Update all repositories with alpine base image version X to Y", and then click next a bunch of time and see a list of PRs with their pipeline status. Hi friends I'm Daniel, the creator of Infra as AI. I was laid off from my SRE role a couple of months ago, and decided to build something that could always help me at work. My job was to keep the lights…
I built an open-source AI system for drones2023 · github.com · ▲5Hey HN! I just released a suite of AI models for deployment on UAV and other "overhead" devices to provide some understanding of the world below. The objective is to empower all sorts of open-source use cases around search and rescue, wildfire prevention, ground risk mitigation for flight over populated areas etc... The neural networks are trained for a bunch of different devices from big GPUs to tiny edge AI cameras like the Luxonis OAK, with some optimised ones for Nvidia TensorRT and other cool bits and pieces too. The main release package also includes some boilerplate code for running…
We built a powerful and free AI research assistant2024 · join.modularmind.app · ▲7Hi everyone, We have been developing a platform to enable professionals to build AI assistants to help them through their work. After a few months, we realized people are trying to sell basic functionalities that can be built from scratch in a couple of hours. Due to this, individuals who are not familiar with the current SOTA are misinformed about the potential of generative models. So, we decided to open up some of our most popular templates as standalone tools for free to empower individuals and set a solid standard for what people should expect. We believe the barrier to accessing…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
