Alternatives
Products that do what Pruna AI – Inference Optimization Engine does
Hello Hacker News! I am Bertrand from Pruna AI. With my associates, John, Rayan, and Stephan, we are fellow researchers in AI efficiency and reliability coming from TUM. We are building an optimization engine that combines compression methods (e.g. quantization, pruning, compilation, batching…) in the aim of saving compute power when running AI models. This optimization engine take one base model as input and returns a compressed model as output. It aims to help for two things: - Make various AI models faster and/or smaller for various hardware (because they can require significant…
- 1
- 2S1
I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm. I started experimenting with Serverless AI inference, but found out that coldstarts were huge. I went deep into the research and put together an engine that loads large models from SSD to VRAM up to ten times faster than alternatives. It works with vLLM, and transformers, and more coming soon. With this project you can hot-swap entire large models (32B) on demand. Its great for: Serverless AI Inference Robotics On Prem deployments Local Agents And Its open source. Let me know if anyone…
Nov 2025 · github.com
- 3IB
Hey HN! I just released a suite of AI models for deployment on UAV and other "overhead" devices to provide some understanding of the world below. The objective is to empower all sorts of open-source use cases around search and rescue, wildfire prevention, ground risk mitigation for flight over populated areas etc... The neural networks are trained for a bunch of different devices from big GPUs to tiny edge AI cameras like the Luxonis OAK, with some optimised ones for Nvidia TensorRT and other cool bits and pieces too. The main release package also includes some boilerplate code for running…
2023 · github.com
- 4WB
Hi everyone, We have been developing a platform to enable professionals to build AI assistants to help them through their work. After a few months, we realized people are trying to sell basic functionalities that can be built from scratch in a couple of hours. Due to this, individuals who are not familiar with the current SOTA are misinformed about the potential of generative models. So, we decided to open up some of our most popular templates as standalone tools for free to empower individuals and set a solid standard for what people should expect. We believe the barrier to accessing…
2024 · join.modularmind.app
- 5MA
I've been working on training this small vision language model for the last month - excited to release the first prototype today! It is based on SigLIP (image encoder), Phi-1.5 (text model) and trained using the LLaVa-1.5 training dataset. It runs reasonably fast on CPU with ~8GB of RAM in full 32-bit precision. There's plenty of room to speed it up and reduce memory consumption by quantizing the model. I posted a video of it running on my M2 Macbook Air (on CPU not MPS, so performance should be comparable on other hardware) on Twitter to demonstrate inference speed:…
2023 · github.com
- 6AT
I have a favour to ask. I’ve been working for a while on Kalavai, a project to make distributed AI easy. There are brilliant tools out there to help AI hobbyists and devs on the software layer (shout out to vLLM and llamacpp amongst many others!) but it’s a jungle out there when it comes to procuring and managing the necessary hardware resources and orchestrating them. This has always led me to compromise on the size of the models I end up using (quantized versions, smaller models) to save cost or to play within the limits of my rig. Today I am happy to share the first public version of our…
2024 · github.com
- 7BO
Read the full blogpost at https://rach.codes/blog/Introducing-Bhumi (click on reader to see the technical breakdown!) AI inference should be fast, but in practice it’s painfully slow. Inference bottlenecks slow down LLM-powered chatbots and AI workflows everywhere. I built Bhumi to fix that. Bhumi is a Python library designed for developers, yet its performance-critical core is implemented in Rust (via PyO3) for near-native speed. This hybrid approach delivers up to 2.5x faster response times across providers like OpenAI, Anthropic, and Gemini—without changing the…
2025 · bhumi.trilok.ai
- 8TN
Hi guys, I’m excited to share an update on ReproModel, an open-source toolbox designed to streamline the testing and reproduction of machine learning models. I, like many of you, have really struggled with benchmarking and comparing models, from missing code, to opaque experiment parameters slowing the process. I decided to take matters into my own hands, and created a mini-toolbox in my free time to streamline the process. The goal is to reduce the time and effort spent on replicating experiments, enabling researchers to focus on innovation rather than setup. Knowing this task is not an…
2024 · github.com
- 9CM
Hey HN, I've been building AutoAgents, an AI agent framework in Rust. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines. The problem Every agent framework lets you swap LLM providers. Almost none of them give you a structured way to enforce safety, caching, or data sanitization in the inference path itself. You end up with guardrails as application-level if-statements, caching bolted on as a separate service, and PII handling as a "we'll add it later" TODO that never ships. This gets worse with local models. Cloud APIs…
Mar 2026 · github.com
- 10FA
Founder here. I built NEO, an AI agent designed specifically for AI and ML engineering workflows, after repeatedly hitting the same wall with existing tools: they work for short, linear tasks, but fall apart once workflows become long-running, stateful, and feedback-driven. In real ML work, you don’t just generate code and move on. You explore data, train models, evaluate results, adjust assumptions, rerun experiments, compare metrics, generate artifacts, and iterate; often over hours or days. Most modern coding agents already go beyond single prompts. They can plan steps, write files, run…
Jan 2026 · marketplace.visualstudio.com
- 11FS
Hi everyone! I've been loving building with AI, and over the past few years I've been leaning more and more into Typescript (and bun). My team at inference.net is constantly trying to get more leverage out of AI and find ways to setup our codebase to be able to increase the level of correctness that our AI is able to write code at. This starter repo is a very opinionated way to lay out a repo to lean into AI heavily. It leverages Cloudflare Workers as a deployment target for the API (my goal is to never have to deploy an API on a AWS/Azure/GCP server ever again unless I get to a…
2025 · abeahmed.com
- 12RA
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
- 13PA
Hey HN, we’re the team at Morph Labs and we’re excited to release Phorm (https://phorm.ai), a fast, simple, and SOTA codebase answer engine. You can search over up to 8 repositories in almost any language, and Phorm can comfortably handle repositories up to ~200K LOC each. It is free during our initial research preview. Phorm’s Advanced Indexing combines synthetic data with static analysis of the code graph to improve the relevancy of search results by up to 3X. We’re proud to launch with featured Advanced Indexing support for a select group of leading open-source projects: - Nomic…
2024 · phorm.ai
- 14YA
Hey folks! I'm a founding engineer at Yorph AI, an agentic data platform, built using ADK, that helps users (starting with product managers and analysts) join data from different sources (upload or sync), build version-controlled and reliable data workflows, and clean, analyze, and visualize data — all in one place. We're also releasing semantic layer creation later this week. The beta is live at yorph.ai/login — would love to hear your thoughts and feedback! (FYI: We're still waiting on Google app verification — you'll see a warning for a few days. Dropbox shows a similar one since…
Nov 2025 · yorph.ai
- 15VA
VibeSolve is an open-source tool that turns a plain-English description of an optimisation problem into Timefold code. Mathematical optimisation is a branch of mathematics and computer science that searches for the minimum/maximum of objective functions, and has applications in transport, logistics, scheduling etc. We are exploring where LLMs can add value in optimisation algorithm development, and where they get in the way. Right now, it works well for rapid prototyping. It does not create production-ready code and requires technical skills to use. It is noticeably better at creating…
Jun 2026 · vibesolve.ai
- 16FC
Hi there, I've created this side project to make it easier to find interesting repositories using AI. There's still a lot of work to be done to improve it, so any suggestions for enhancements would be greatly appreciated. Thank you!
2024 · awesome-repositories.com
- 171P
A few weeks ago I posted about GoodToGo https://news.ycombinator.com/item?id=46656759 - a tool that gives AI agents a deterministic answer to "is this PR ready to merge?" Several people asked about the larger orchestration system I mentioned. This is that system. I got tired of being a project manager for Claude Code. It writes code fine, but shipping production code is seven or eight jobs — research, planning, design review, implementation, code review, security audit, PR creation, CI babysitting. I was doing all the coordination myself. The agent typed fast. I was still the…
Feb 2026 · github.com
- 18FC
Hi HN,I am Anubhav from RamanLabs.We have been developing dedicated modules based on deep-learning for purposes like face-detection,object-detection,pose-estimation etc. We hope to make it easy for developers,hobbyists to integrate such functionalities into their existing app/pipeline at the cost of a few milliseconds.All our modules run end to end in super-realtime even on consumer-grade CPUs[0]. For now we provide only Python based API. We provide Demo for each of the modules to allow testing for your desired data distribution.We also have a blog[1] where we hope to add more technical…
2022 · ramanlabs.in
- 19MA
Hey HN! I built a thing and I'm really excited to share it. EDIT: I meant to link to the github, not the website: https://github.com/max-hq/max Like many of us here, I've been commonly reaching for a pattern of "pull data into db; give it to claude" for a while, whilst doing data spelunking or building tooling - for the same reasons mentioned by thellimist over here [1] and a few other recent "CLI vs MCP" posts. To that end, about a month ago I started building a project called `max` - its goal is to cut the middleman and schematise any data source for you. Essentially,…
Mar 2026 · max.cloud
- 20FA
Hi HN, We're excited to introduce Fixstars AIBooster, our new performance engineering tool designed to significantly accelerate AI model training while optimizing GPU utilization. AIBooster provides: Real-time monitoring of GPU, CPU, memory, and power consumption. Clear visibility into performance bottlenecks, helping developers optimize AI workloads. Proven acceleration of AI training processes—users commonly achieve up to 2-3x speed improvements. Significant cost savings by maximizing infrastructure efficiency. It's free to try, requires minimal setup, and integrates seamlessly into your…
2025 · fixstars.com
- 21IB
Disclaimer it is a heavily AI assisted project. The goal was not to be the most performative but the kind that's easier to learn from. I wanted to share this in case there are people who had the same idea or wanted to see something like this.
Jun 2026 · github.com
- 22ST
Hi HN! I’m a founder at Nextmv (YC 20) [1] We’ve been building out optimization algorithm testing capabilities (acceptance tests, etc.) and just shipped our first pass at shadow testing [2, 3]. In our experience, tools like shadow testing save time and build confidence in decision models, but tools also take time to build and maintain. We’ve seen shadow testing tools in the machine learning and MLOps space [4], but not so much in the operations research community. A lot of folks here [5] seem experienced with optimization models and we’d love to have your feedback! What do you like? What…
2023 · nextmv.io
- 23DA
Hi HN, Today I'd like to present the results of my weekend project of the last year or so. Given there are many posts on HN about LLMs and Prolog, I thought that this would be of interest. DeepClause is my own (possibly misguided :-) attempt at combining LLMs with Logic Programming, ultimately hoping to establish a foundation for building more reliable agents, that produce reproducible and fully traceable result. At the heart of DeepClause is a DSL called "DeepClause Meta Language" (DML) which can be used to encode agent behaviors as executable logic programs. DML is executed by a…
Nov 2025 · github.com
- 24PS
I didn't want to buy a standalone computer or repurpose a laptop to run constantly so I could maintain a system to sync my LLMs, so I built this. It's a simple overview of my system, laid out in a way easy to unpack and replicate for yourself. The project is meant to be configured individually, and uniquely, since one solution might not be what's best for another. If anything, maybe it gives you some ideas on how to implement things for your own project. Best wishes, Ryan.
27d ago · pacslate.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →