Alternatives
Products that do what Red Candle – Run LLMs Natively in Ruby with Rust does
I've been working on Red Candle, a Ruby gem that runs LLMs (Llama, Mistral, Gemma, Phi) directly in your Ruby process through Rust bindings (based on the candle crate from Hugging Face). No Python, no servers - just FFI with Metal/CUDA acceleration. It's been useful for adding AI features to Rails apps without the complexity of managing separate services. Would love feedback from anyone working with LLMs in Ruby.
- 1
- 2
- 3
- 4
- 5PR
TL;DR - http://www.gemfury.com - pre-launch feedback appreciated Hello HN, In true hacker tradition, a tool built for ourselves often grows into a service of its own. In the past few months, we've been polishing our private RubyGem server into something that other Ruby developers can use. Today, I'd like to ask for your feedback on where we stand and whether we're "launch-ready". The service is called Gemfury and offers customers a private RubyGem repository in the cloud. The story is that many RubyGems are built as open source and solid infrastructure exists to develop and distribute those.…
2012
- 6RT
2012 · rubymotion-tutorials.com
- 7
- 8XR
Hi HN, We built Xybrid, a Rust library for running LLM + speech pipelines directly inside your app, no server, no daemon, just one binary. We started building it while working on a privacy-focused LLM app with Tauri and realized there wasn’t a straightforward way to embed models directly into shipped applications without relying on a separate server process. Xybrid links into your process like any other library. It supports GGUF / ONNX / CoreML and integrates with Flutter, Swift, Kotlin, Unity, and Tauri, letting you run pipelines like speech → LLM → speech in a single call. On…
Mar 2026 · github.com
- 9VA
2012 · vesperapps.com
- 10
- 11LT
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com
- 12AO
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
2024 · github.com
- 13TI
Hello HN, While browsing the Python docs yesterday, I discovered that the latest 3.12 version has added support for a `python3 -m sqlite3` interactive shell. I looked into the source code, and its implementation was simple, giving me an idea: Why not hook the beautiful llm library by simonw into such an interactive shell, and thus have direct LLM support in SQLite? Without writing a C extension, build a shared object and all that fuss. Well, now you can `pip install tsellm` and do just that. demo gif:…
2024 · github.com
- 14UA
Hey HN! After using a combination of Unsloth and Axolotl a lot, and finding it generally painful to figure out the right performance tuning for things like batch sizing and multi-GPU sharding, I wrote a small Python lib that sets up known-good LoRA training configurations for Llama 3.1 8B and 70B Instruct, and includes helpers for distilling from larger models or training on serverless finetuning platforms, and includes a walkthrough for distilling DeepSeek-R1 into a Llama 3.1 8B LoRA... But you can use it for pretty much any finetuning task, not just distilling large models!
2025 · github.com
- 15RD
Hi everyone, I've been working on this thing for a while. It transpiles Haml into Ruby, and then it streams patches to the browser which update the DOM. There is a memory leak in s.rb, which I'm not really sure how to solve. Seems like objects stay alive for longer than they should because of circular references or something but I'm not sure what's going on to be honest. I think this approach has potential though.
2023 · github.com
- 16GE
2019 · fiddle.dragonruby.org
- 17RL
Deep Chat has just received a huge update! You can now host entire LLMs on the browser. No servers, no connections, run it all in the comfort of your browser. Supported models include popular LLaMA and Mistral LLMs. Check out the Open Source project to add it to your website: https://github.com/OvidijusParsiunas/deep-chat Try it out live in the Deep Chat playground: https://deepchat.dev/playground
2024 · deepchat.dev
- 18IB
hey hn, I built an open-source Perplexity clone that can run local LLMs and cloud LLMs. It's fully self-hostable through Docker and uses ollama to support local LLMs. The demo video in the repository shows me running it locally with llama3 on my M1 Macbook Pro. I'm open to any suggestions or feedback, thanks!
2024 · github.com
- 19RG
Hi HN, I've just made a ruby gem for parsing Mermaid syntax, and it's an extension of kramdown parser. I'd love to get feedbacks and or problems you encounter.
2023 · github.com
- 20PB
Started this a few months ago because I wanted a better test runner for an entirely different LLM-based project, then got completely nerdsniped by making LLMs as easy as possible to hotswap into other projects. It's still on the early side but it's finally at the point where I would happily use it for my original project, so I figured I'd post it. It's also—almost by accident—fully remote compatible (permissioning system included!), so you can host it on a box and then connect a program to it remotely.
2023 · github.com
- 21BA
2014 · github.com
- 22GA
2024 · github.com
- 23FR
2019 · github.com
- 24OB
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →