Arch-Function: 3B parameter LLM that beats GPT-4o on function calling
Hi HN! My name is Salman Paracha. I aam the the Founder/CEO of Katanemo - the organization behind the open source Arch GW (an intelligent gateway for prompts - https://github.com/katanemo/arch). Today, we are making the (SOTA) LLMs engineered in Arch GW for function calling scenarios available under an OSS license that borrows from Llama's community license. What is function calling? Function calling helps developers personalize apps by calling application-specific operations via user prompts. This involves any predefined functions or APIs you want to expose to…
What it does
In the maker’s words, at launch
Hi HN! My name is Salman Paracha. I aam the the Founder/CEO of Katanemo - the organization behind the open source Arch GW (an intelligent gateway for prompts - https://github.com/katanemo/arch). Today, we are making the (SOTA) LLMs engineered in Arch GW for function calling scenarios available under an OSS license that borrows from Llama's community license. What is function calling? Function calling helps developers personalize apps by calling application-specific operations via user prompts. This involves any predefined functions or APIs you want to expose to perform tasks, gather information, or manipulate data - via prompts. With function calling, you get to support agentic workflows tailored to domain-specific use cases - from updating insurance claims to creating ad campaigns. Arch-Function analyzes prompts, extracts critical information from prompts, engages in lightweight conversations with the user to gather any missing parameters and makes API calls so that you can focus on writing business logic. Arch-Function is an auto-regressive model that if run on the NVIDIA A100 GPUs using vLLM offers throughput of ~1900/output tokens per second, and a output token price of $0.10/M token. This is ~12x faster and 44x cheaper than GPT-4o.
Does the same job
all alternatives →- LBlambdaprompt – build, compose and call templated LLM prompts2022 · github.com · ▲9
For the past few months I've been building a lot of things with LLMs (GPT-3, Codex, etc.) as I've been trying to push them to their limits (especially towards applying them to the tabular data domain) When working on this, I've found there are some common patterns for solving problems (templating, chaining, functional-programming style operations, etc.) As I've iterated, I've come to believe that a functional style interface is likely going to power a new wave of systems I'm calling "prompt-machines"(systems where the core new unit of work is a "named" LLM prompt, extending the "function"…
- AGArch GW – Distributed gateway for agents, engineered with small LLMs2024 · docs.archgw.com · ▲7
Hi HN My name is Salman and I work on Arch GW - the intelligent gateway designed to protect, observe, and personalize LLM applications with your APIs. https://github.com/katanemo/arch Our team built Envoy Proxy at Lyft, and re-imagined it with the belief that: Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – all outside business logic. Engineered with purpose-built LLMs, Arch…
- PEPlano – Edge and service proxy with orchestration for AI agentsJan 2026 · github.com · ▲8
Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…
- LFLLM Function Calling Library to Interact with File, Shell, Git and Code2024 · swekit.dev · ▲5
Hey HN, I built SWE-Kit, LLM toolkit (Function callable tools) which makes building agents specialised in coding like Devin very easy. I noticed a typical pattern while building local agents: creating & perfecting LLM tools to interact with system or codebase was the repeated and time-consuming. We created a layer that simplifies building agents that can interact with code, file system, git, shell and allows you to quickly solve for a wide variety of coding agent use cases. Aren’t there open coding agents already? Well, yes, but most folks would want to solve their specific use case like a…
- PAPromptL, a templating language designed for LLM prompting2024 · promptl.ai · ▲7
Hey HN! We just launched PromptL: a templating language built to simplify writing complex prompts for LLMs like GPT-4 and Claude. Why PromptL? Creating dynamic prompts for LLMs can get tricky, even with standardized APIs that use lists of messages and settings. While these formats are consistent, building complex interactions with custom logic or branching paths can quickly become repetitive and hard to manage as prompts grow. PromptL steps in to make this simple. It allows you to define and manage LLM conversations in a readable, single-file format, with support for control flow and…
- HPHorizon – Programmatic Prompt Generation and LLM Configurations2023 · gethorizon.ai · ▲7
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, October 2024
the whole month →
Softr for Notion▲925Turn Notion databases into portals & apps with no code
Dev tools · 2024 · softr.io
One inbox for all your work discussions
Work · 2024 · generalcollaboration.com