Alternatives
Products that do what Launch StableStudio local inference in one commmand does
Happy to announce integration between imaginAIry and StableStudio. This is the easiest way to try out StableStudio with local image generation. A local instance can be launched simply with `pip install imaginairy --upgrade && aimg server` As a reminder StableStudio is the new open-source version of DreamStudio from stability.ai. https://github.com/Stability-AI/StableStudio
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9IA
I like the idea of taking one thing and turning it into another—very much inspired by NotebookLM and wondered what it might take to generate full graphic novels, with consistent characters, narrative flow, story arc, etc. Developed a 7-pass scripting enrichment system (beat analysis, adaptation filtering, character deep dives) before generating any images. Dual backend: Google Gemini for scripting (2M context window) and either Gemini or OpenAI for image generation with 3-tier model fallback (comparing the performance of both). It's not great. Would love feedback on the pipeline.
Mar 2026 · arv.in
- 10IB
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com
- 11OS
Triton Co-Pilot: A quick way to write glue code to make deploying with NVIDIA Triton Inference Server easier. It's a cool CLI tool that we created as part of an internal team hackathon. Earlier, deploying a model to Triton was very tough. You had to navigate through the documentation for the Python backend, figure out how to get your inputs and outputs right, write a bunch of glue code, create a config.pbtxt file with all the correct parameters, and then package everything up. It could easily take a couple of hours. But with Triton Co-Pilot, all that hassle is gone. Now, you just write your…
2024 · github.com
- 12S1
I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm. I started experimenting with Serverless AI inference, but found out that coldstarts were huge. I went deep into the research and put together an engine that loads large models from SSD to VRAM up to ten times faster than alternatives. It works with vLLM, and transformers, and more coming soon. With this project you can hot-swap entire large models (32B) on demand. Its great for: Serverless AI Inference Robotics On Prem deployments Local Agents And Its open source. Let me know if anyone…
Nov 2025 · github.com
- 13RA
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
- 14IM
I made my first macOS utility app that ships with a bundled Gemma 4 model, specifically the Gemma E4B one. It made my app DMG have 5.3 GB in size, but I think it is a small size for the power that this free local model can provide. It runs fine on CPU, but can also run on Apple Silicon GPU, although I did not notice any performance improvements with GPU (tested on a M5 chip). I think these local lightweight and multimodal models will open multiple possibilities for new software tools where privacy is essential.
May 2026 · snapname.app
- 15CT
I had been looking to try <500M parameter language models but you wouldn't find an API to try them anywhere, so I built this cloudflare hosted static website that hosts weights and built an inference runtime for these models that uses WebGPU and runs inference from your browser. These are only so useful in a multi-turn conversation but it's still interesting to see what you can pack in a <250mb model. I tried using ONNX versions earlier, but there were too many quirks of using them with language models and the TPS wasn't too impressive. Inspired by svenflow/webgpu-gemma, I put my codex…
May 2026 · chonklm.com
- 16IE
Quick note on how it works and how I've done my batch embedding engine IgniteMS. The whole thing runs as one process using Rust, reading input, tokenizing, packing batches, keeping the queue full. TensorRT handles inference. Python is only as a wrapper. I built it this way because when you use more than couple of GPUs, the GPUs stop being the problem. CPU cannot feed them fast enough. One A100 can go through batches faster than Python can tokenize and feed, so the GPU just sits there idle waiting for work. Most of my time went into optimizing this. At 8 GPUs that was basically the entire…
Jun 2026 · github.com
- 17FM
As the title and description of the GitHub repo suggest, I’m working on a small project for purely educational purposes, with the goal of implementing generative model inference (small models capable of modeling 2D distributions) based on the Flow Matching paradigm in C. I’ve worked on generative AI models based on Flow Matching from a more “abstract” perspective, using frameworks like PyTorch, and I wanted to understand what goes on behind the scenes. The repository is still a work in progress and is also one of my first "serious" projects in C.
Jul 2026 · github.com
- 18DI
Hi HN! I’m so excited to show my another open-source project here. It is a PoC project. Distributed Inference is a project to demonstrate an approach to designing cross-language and distributed pipeline in deep learning/machine learning domain, using WebRTC and Redis Streams. This project consists of multiple services, which are written in Go, Python, and TypeScript, running on Docker. It allows setting up multiple inference services in multiple host machines, in a distributed manner. It does RPC-like calls and service discovery via my other open-source projects, go-inventa and…
2023 · github.com
- 19OC
I built an open source version of OpenAI Chronicle. Some design decisions I made: 1. Local first: OCR uses Apple Vision, summarization supports local AI providers via Vercel AI SDK. Nothing leaves your computer. 2. Multiple Provider: exposes MCP so any coding agents can use it. 3. Swift menubar app: efficient, low-footprint 4. Blacklist apps: password managers, messaging apps (Slack, WhatsApp, Messenger), mail clients are on default blocklist. Current Limitations: 1. Mac only. Mac-first is a feature. 2. Small local models with weak structured-output support will fail on generateObject. 3.…
Apr 2026 · github.com
- 20AC
Built an AI code reviewer using Letta (Python) that I can call natively from Rust applications. The interesting part: real-time streaming works perfectly across the language boundary with zero hassle using RunAgent. The agent runs in Python with persistent memory, leverages the best in house agentic memory management with Letta (Pythonic AI agent framework), and my rust code just uses it (kinda) natively, though Letta has no Rust bindings. And, streaming works like magic. No FFI, no complex bridges - just native async/streaming that feels like calling any Rust librar, but without…
2025 · medium.com
- 21LT
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com
- 22
- 23HA
Demo starts at 50m into the video. This was a bit terrifying to record because 2am the previous night everything was totally broken after a major refactor (so that we could add external LLM support as well as local GPUs). But pressure can be a useful force :-D We start with a stack deployed on my laptop without a GPU, pointing to together.ai so we can run open source LLMs easily without having to have access to a GPU. We show simple inference through the ChatGPT-like web interface (with users, sessions etc) and then simple drag'n'drop RAG. Then we show some helix apps defined as yaml: Marvin…
2024 · youtube.com
- 24KA
Hi Indie Hackers We’re building Kinkora, a creative platform that brings multiple image and video AI models into one place for experimentation and creation. Like many builders, we found ourselves constantly switching tools just to test different models or creative directions. Each platform felt limited to a single workflow or use case. So we decided to build a space that’s more modular, exploratory, and creator-first. Kinkora focuses on: Supporting popular generative models Making experimentation fast and enjoyable Laying the foundation for a creative community, not just a generator Our…
Dec 2025 · kinkora.fun
Ranked by how close each launch is in meaning, then by votes. Refine with a description →