Standardizing build, experiment, and deployment for LLM Development
*Motivation* Hi hackers, I'm Asif. I know we dislike premature standardization, but hear me out. LLM Application development is extremely iterative, more so than most other types of application development. We need a process that allows us to iterate faster. LLM Development is highly iterative due to the activities that come with regular software development, as well as the need to make the LLM Application accurate and reduce hallucination. To improve hallucination, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt…
What it does
In the maker’s words, at launch
*Motivation* Hi hackers, I'm Asif. I know we dislike premature standardization, but hear me out. LLM Application development is extremely iterative, more so than most other types of application development. We need a process that allows us to iterate faster. LLM Development is highly iterative due to the activities that come with regular software development, as well as the need to make the LLM Application accurate and reduce hallucination. To improve hallucination, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, and possibly try multi-agent architecture. There are thousands of permutations to try, and we want to be able to easily experiment with different permutations and have a process to objectively judge LLM performance so we can iteratively move towards accuracy goals. *Solution* I have been working in AI since 2021 - first at FAANG with ML, then with LLM in start-ups since early 2023. I have had the chance to talk with many different companies that have been successful and unsuccessful with AI development. Using my learnings, I am working on an Open Source framework to standardize the build, experiment, and deploy process for LLM Development. The goal of this framework is to optimize for rapid iteration. We are doing this by enforcing a modular LLM application layer build, allowing for easy testing of different configurations of your application. We provide maximum flexibility for using any external tools you want for building your application. We also have tools to benchmark your accuracy and improve the performance of your application in a data-driven way. Finally, everything is deployable through a Docker image. *Getting Involved* If you're curious, check us out on Github. You can get fully set up with a single command. Stars for better visibility
Does the same job
all alternatives →- ILImprove LLM Performance by Maximizing Iterative Development2024 · github.com · ▲104
I have been working in AI space for a while now, first at FAANG with ML since 2021, then with LLM in start-ups since early 2023. I think LLM Application development is extremely iterative, more so than any other types of development. This is because to improve an LLM application performance (accuracy, hallucinations, latency, cost), you need to try various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, different agent architecture, and more. There are thousands of possible combinations and you need a process…
- ILImprove LLM Performance by Maximizing Iteration Speed2024 · palico.ai · ▲5
LLM Application development is extremely iterative, more so than any other types of development. This is because in addition to all the activities involved in regular application development, we also need to make the LLM Application accurate and reduce hallucination. To improve performance, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, try different agent architecture, and more. There are thousands of permutations to try. We need to be able to easily experiment with these…
- WWWhy write code if the LLM can just do the thing? (web app experiment)Nov 2025 · github.com · ▲436
I spent a few hours last weekend testing whether AI can replace code by executing directly. Built a contact manager where every HTTP request goes to an LLM with three tools: database (SQLite), webResponse (HTML/JSON/JS), and updateMemory (feedback). No routes, no controllers, no business logic. The AI designs schemas on first request, generates UIs from paths alone, and evolves based on natural language feedback. It works—forms submit, data persists, APIs return JSON—but it's catastrophically slow (30-60s per request), absurdly expensive ($0.05/request), and has zero UI…
- HPHorizon – Programmatic Prompt Generation and LLM Configurations2023 · gethorizon.ai · ▲7
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
- IBI built a toy music controller for my 5yo with a coding agent2025 · github.com · ▲37
The HN community may find the context of the prompts, organized by each turn in each session, the most useful. See the website/docs/prompts.md and session-X.md files. I also started exploring some workflows for the LLM to execute, organized in the website/docs/tasks/ folder. I found it pretty handy to have the LLM document our work as we went and simply embedded the static site into the executable, along with all the music and logic. The whole project took me about a day for the backend. The C++ controller itself took only a few turns. I enjoyed focusing on my son's…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, July 2024
the whole month →
- IC
Many years ago, I made VJ softwares (to mix live visuals in clubs) for unexpected platforms like the Game Boy Advance, the Playstation 2 and the Raspberry Pi. This year, I’m back with a new web-app: Pikimov. Inspired by Photopea (a free Photoshop clone), I created this web-based motion design & video editor as an alternative to After Effects, to fill empty void. It's free, without signup, without cloud uploads (your files stay on your machine), and your projects are not used for AI models training.
AI · 2024 · pikimov.com
