Alternatives
Products that do what Arch-Function: 3B parameter LLM that beats GPT-4o on function calling does
Hi HN! My name is Salman Paracha. I aam the the Founder/CEO of Katanemo - the organization behind the open source Arch GW (an intelligent gateway for prompts - https://github.com/katanemo/arch). Today, we are making the (SOTA) LLMs engineered in Arch GW for function calling scenarios available under an OSS license that borrows from Llama's community license. What is function calling? Function calling helps developers personalize apps by calling application-specific operations via user prompts. This involves any predefined functions or APIs you want to expose to…
- 1AR
Hi HN — we're the team behind Arch (https://github.com/katanemo/archgw), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router (https://huggingface.co/katanemo/Arch-Router-1.5B), a 1.5B router model for preference-based routing, now integrated into the proxy. As teams integrate multiple LLMs - each with different strengths, styles, or cost/latency profiles — routing the right prompt to the right model becomes a critical part of the application design. But it's still an open problem. Most routing systems fall into two…
2025
- 2AA
Hi HN! My name is Adil Hafeez, and I am the Co-Founder at Katanemo and the lead developer behind Arch - an open source project for developers to build faster, generative AI apps. Previously I worked on Envoy at Lyft. Engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting jailbreak attempts, intelligently calling “backend” APIs to fulfill the user’s request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability…
2024 · github.com
- 3
- 4
- 5WW
I spent a few hours last weekend testing whether AI can replace code by executing directly. Built a contact manager where every HTTP request goes to an LLM with three tools: database (SQLite), webResponse (HTML/JSON/JS), and updateMemory (feedback). No routes, no controllers, no business logic. The AI designs schemas on first request, generates UIs from paths alone, and evolves based on natural language feedback. It works—forms submit, data persists, APIs return JSON—but it's catastrophically slow (30-60s per request), absurdly expensive ($0.05/request), and has zero UI…
Nov 2025 · github.com
- 6AA
Hi HN! My name is Adil Hafeez, and I am the Co-Founder at Katanemo and the lead developer behind Arch - an open source project for developers to build faster, generative AI apps. Previously I worked on Envoy at Lyft. Engineered with purpose-built LLMs, Arch handles the critical but undifferentiated tasks related to the handling and processing of prompts, including detecting and rejecting jailbreak attempts, intelligently calling “backend” APIs to fulfill the user’s request represented in a prompt, routing to and offering disaster recovery between upstream LLMs, and managing the observability…
2024 · github.com
- 7LB
For the past few months I've been building a lot of things with LLMs (GPT-3, Codex, etc.) as I've been trying to push them to their limits (especially towards applying them to the tabular data domain) When working on this, I've found there are some common patterns for solving problems (templating, chaining, functional-programming style operations, etc.) As I've iterated, I've come to believe that a functional style interface is likely going to power a new wave of systems I'm calling "prompt-machines"(systems where the core new unit of work is a "named" LLM prompt, extending the "function"…
2022 · github.com
- 8AG
Hi HN My name is Salman and I work on Arch GW - the intelligent gateway designed to protect, observe, and personalize LLM applications with your APIs. https://github.com/katanemo/arch Our team built Envoy Proxy at Lyft, and re-imagined it with the belief that: Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – all outside business logic. Engineered with purpose-built LLMs, Arch…
2024 · docs.archgw.com
- 9AA
Hi HN! This is Salman, Adil, Shuguang and Co working on ArchGW[1] - an open-source lightweight proxy server for prompts - written in Rust and built on top of Envoy[2]. Arch moves the critical but pesky handling and processing of prompts: task understanding, prompt routing, safety, and observability - outside business logic. Its an edge and egress proxy for agentic apps. We've talked to 100s of developers at places like Twilio, GE Healthcare, Redhat, Square, etc and there was a consistent theme in building AI apps: to move past a nascent demo they are left to their own devices in building out…
2025 · github.com
- 10AO
Hi HN! This is Adil, Salman, Co and Shuguang and we're excited to introduce archgw [1], an open source intelligent proxy for agents built on Envoy [2]. Arch moves the critical but crufty work around safety, observability, and routing of prompts outside business logic. Arch is a uniquely intelligent infrastructure primitive, engineered with purpose-built fast LLMs [3] for tasks like intent detection over multi-turn, parameter identification and extraction, triggering single/multiple function calls, and offers convenience features to auto dispatch LLM calls for summarization based on data…
2024 · github.com
- 11PE
Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…
Jan 2026 · github.com
- 12LT
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com
- 13LF
Hey HN, I built SWE-Kit, LLM toolkit (Function callable tools) which makes building agents specialised in coding like Devin very easy. I noticed a typical pattern while building local agents: creating & perfecting LLM tools to interact with system or codebase was the repeated and time-consuming. We created a layer that simplifies building agents that can interact with code, file system, git, shell and allows you to quickly solve for a wide variety of coding agent use cases. Aren’t there open coding agents already? Well, yes, but most folks would want to solve their specific use case like a…
2024 · swekit.dev
- 14TO
Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2024 · github.com
- 15FT
I wrote a small local tool to transcribe audio notes (Whisper/Parakeet). Code: https://github.com/bilawalriaz/lazy-notes I wanted to process raw transcripts locally without OpenRouter. Llama 3.2 3B with a prompt was decent but incomplete, so I tried SFT. I fine-tuned Llama 3.2 3B to clean/analyze dictation and emit structured JSON (title, tags, entities, dates, actions). Data: 13 real memos → Kimi K2 gold JSON → ~40k synthetic + gold; keys canonicalized. Chutes.ai (5k req/day). Training: RTX 4090 24GB, ~4h, LoRA (r=128, α=128, dropout=0.05), max seq 2048,…
2025 · bilawal.net
- 16PA
Hey HN! We just launched PromptL: a templating language built to simplify writing complex prompts for LLMs like GPT-4 and Claude. Why PromptL? Creating dynamic prompts for LLMs can get tricky, even with standardized APIs that use lists of messages and settings. While these formats are consistent, building complex interactions with custom logic or branching paths can quickly become repetitive and hard to manage as prompts grow. PromptL steps in to make this simple. It allows you to define and manage LLM conversations in a readable, single-file format, with support for control flow and…
2024 · promptl.ai
- 17HP
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
2023 · gethorizon.ai
- 18
- 19EL
Hey HN! I built Experiment to solve a common frustration in LLM development: the lack of proper tools for prompt engineering experimentation. Here's what makes it different: Key Features: - Load and edit chat completion logs from CSV files - Fork and modify specific conversation entries - Run inference via Anthropic, Mistral, and OpenAI - Define custom tools using JSONSchema format - Visual tool usage analysis with collapsible, sorted key-value pairs - Full mobile support and available as installable PWA Technical Highlights: - Built with React using custom isomorphic architecture -…
2025 · github.com
- 20SL
For speech-to-text, large-language-model inference and text-to-speech I created three wrapper libraries in C/C++ (using Whisper.cpp, Llama.cpp and Piper). Follow the URL to see an example that shows how to use these libraries for a speech-to-text, LLM inference, text-to-speech pipeline. Windows and Linux are supported.
Sep 2025 · github.com
- 21OS
Hi HN, Matvey, Ildar, Joey, and Dominik here. If you're building LLM agents that use tools, you're probably worried about prompt injection attacks that can hijack those tools. We were too, and found that solutions like prompt-based filtering or secondary "guard" LLMs can be unreliable. Our thesis is that agent security should be handled at the network level between the agent and the LLM, just like a traditional web application firewall. So we built Archestra Platform: an open-source gateway that acts as a secure proxy for your AI agents. It's designed to be a deterministic firewall against…
Oct 2025 · archestra.ai
- 22RO
Hi HN! RΞASON is an OSS Typescript framework for developing LLM apps that uses Typescript's interfaces to get structured output from an LLM. While there are other TS LLM frameworks, I think RΞASON fills a unique space in the market: it's laser-focused on only three areas and, most importantly, actively stays away from pre-made prompting & retrieval. I've been in the LLM space since GPT-3 originally came out, and I've always had problems with other frameworks, such as LangChain. I dislike that they focus a ton on out-of-the-box prompting & pre-made agents — I, as the dev, should be the one in…
2023 · github.com
- 23LO
Needed a simple way to call multiple LLM providers. LiteLLM provides 2 functions - `completion` and `embedding`; and guarantees consistent input/output formats across all providers. That's it!
2023 · litellm.ai
- 24HA
Demo starts at 50m into the video. This was a bit terrifying to record because 2am the previous night everything was totally broken after a major refactor (so that we could add external LLM support as well as local GPUs). But pressure can be a useful force :-D We start with a stack deployed on my laptop without a GPU, pointing to together.ai so we can run open source LLMs easily without having to have access to a GPU. We show simple inference through the ChatGPT-like web interface (with users, sessions etc) and then simple drag'n'drop RAG. Then we show some helix apps defined as yaml: Marvin…
2024 · youtube.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →