Relari – Auto Prompt Optimizer as Lightweight Alternative to Finetuning
Hi HN, we are the founders of Relari (https://www.relari.ai). We launched our LLM evaluation stack on HN a few months ago (https://news.ycombinator.com/item?id=39641105), which is now used in production by AI teams at companies like Vanta and PwC. We have since expanded to directly optimizing parts of an LLM pipeline using a data-driven approach. In particular, we see a lot of potential in the Auto Prompt Optimization—which could be an attractive alternative to fine-tuning in many cases—to use data to align LLMs for domain-specific tasks. Here’s a demo video:…
In plain words
Relari provides an auto prompt optimizer that improves large language models for specific tasks using data-driven methods. Designed for AI teams building LLM applications, it offers an alternative to fine-tuning that requires less overhead. The tool automatically develops and refines prompts across different use cases and models, helping teams adapt to model updates and changing requirements without manual iteration.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN, we are the founders of Relari (https://www.relari.ai). We launched our LLM evaluation stack on HN a few months ago (https://news.ycombinator.com/item?id=39641105), which is now used in production by AI teams at companies like Vanta and PwC. We have since expanded to directly optimizing parts of an LLM pipeline using a data-driven approach. In particular, we see a lot of potential in the Auto Prompt Optimization—which could be an attractive alternative to fine-tuning in many cases—to use data to align LLMs for domain-specific tasks. Here’s a demo video: https://www.loom.com/share/4ad30bf1053e46a3846fc5a07495c486 We started working on the auto prompt optimizer because of our own frustration with developing, iterating, and maintaining prompts across different use cases and models. A minor update to the underlying LLM, a change in user requirements, or a shift in application infrastructure can render a carefully crafted prompt useless. As one user put it, “Prompt engineering is not software engineering; it’s wishful thinking.” We tried prompt optimization tools like DSPy and TextGrad, but realized they require you to adopt new frameworks, craft custom metrics from scratch, and offer limited visibility into the optimization process (or even the final optimized prompt). This lack of transparency left us guessing whether the new prompts are genuinely better or just different. Our Auto Prompt Optimizer aims to be an easy-to-use yet robust alternative, with maximum visibility into the optimization process and final results. It takes two inputs: a dataset with inputs and expected outputs for a given LLM task, and a target metric (we have 30+ out-of-the-box metrics). The optimizer then starts from your initial prompt and uses the dataset to align the LLM output with your desired outcomes. It does this iteratively, mutating the prompt based on feedback from the target metric. The optimizer automatically selects the examples from the datasets to create few-shot prompts and bake in common techniques such as chain of thought when appropriate. Here are two examples of the results that include the initial prompt, each version of the new prompt, and its performance on the target metric - Drug Review Prompt: https://app.relari.ai/demo/prompt/drug-review (a non-standard task where the optimizer created sophisticated instructions with detailed rating rubric and corner case handling) - Summarization Prompt: https://app.relari.ai/demo/prompt/cnn-highlights (a simple task where the optimizer added more straightforward instructions on styling) We see the prompt optimizer as a lightweight and practical alternative for adapting LLMs for domain-specific tasks. It can deliver high-quality prompts with as few as 100 data points. Try it yourself (https://app.relari.ai/). You can upload your dataset or generate a simple synthetic dataset to start the optimization process. It is recommended to use a dataset with at least 30 samples. The optimization process can take up to an hour depending on the size of the dataset and metrics, so we ask you to create an account so we can keep track of each optimization run and will send you an email notice once it’s completed. What’s next? We’re currently working on support for more advanced features like prompt chaining and agent tool call use cases. For power users, we offer custom metrics and multi-objective optimization to address the most complex use cases. What’s been your biggest challenge with prompt engineering? Would a dataset-driven approach could improve your prompt workflow? We’d love to hear your thoughts and feedback on our approach.
Does the same job
all alternatives →
- HPHorizon – Programmatic Prompt Generation and LLM Configurations2023 · gethorizon.ai · ▲7
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
- PRPrompt-refiner – Lightweight optimization for LLM inputs and RAGDec 2025 · github.com · ▲7
Hi HN, While building RAG agents, I noticed a lot of token budget was wasted on formatting overhead (HTML tags, JSON structure, whitespace). Existing solutions felt too heavy (often requiring torch/transformers), so I wrote this lightweight, zero-dependency library to solve it. It includes strategies for context packing, PII redaction, and tool output compression. Benchmarks show it can save ~15% of tokens with negligible latency overhead (<0.5ms). Happy to answer any questions!
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, August 2024
the whole month →
- IY
Life & fun · 2024 · ytch.xyz
- IA
Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…
Dev tools · 2024 · github.com