Litellm – Simple library to standardize OpenAI, Cohere, Azure LLM I/O
I built this library because langchain was too bloated and I needed a simple abstraction to call multiple LLM APIs. litellm has two functions - completion(), embedding()
In plain words
Litellm is a lightweight library that provides a simplified interface for calling multiple large language model APIs, including OpenAI, Cohere, and Azure. It offers two core functions: completion() for generating text and embedding() for creating embeddings. Designed as a leaner alternative to more complex frameworks, litellm enables developers to work with different LLM providers through a standardized, straightforward abstraction layer without unnecessary overhead.
written from the facts on this page · September 2026
Does the same job
all alternatives →
- LOLiteLLM -Open-Source Library for Anthropic,Azure,OpenAI, etc. API Calls2023 · litellm.ai · ▲5
Needed a simple way to call multiple LLM providers. LiteLLM provides 2 functions - `completion` and `embedding`; and guarantees consistent input/output formats across all providers. That's it!
- LPliteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching2023 · github.com · ▲140
Hello hacker news, I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: https://github.com/BerriAI/litellm/ We’re open sourcing our implementation of liteLLM proxy: https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-... TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming What can liteLLM proxy do? - It’s a central place to…
- WTWordllama – Things you can do with the token embeddings of an LLM2024 · github.com · ▲370
After working with LLMs for long enough, I found myself wanting a lightweight utility for doing various small tasks to prepare inputs, locate information and create evaluators. This library is two things: a very simple model and utilities that inference it (eg. fuzzy deduplication). The target platform is CPU, and it’s intended to be light, fast and pip installable — a library that lowers the barrier to working with strings semantically. You don’t need to install pytorch to use it, or any deep learning runtimes. How can this be accomplished? The model is simply token embeddings that are…
- LCLLMs can generate valid JSON 100% of the time2023 · github.com · ▲854
Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, July 2023
the whole month →
- WL
Hey everyone, I here is a small open-source project I've been working on lately. I'd love to hear your thoughts and improvement ideas :) GitHub: [github.com/Vincenius/workout-lol](https://github.com/Vincenius/workout-lol)
Dev tools · 2023 · workout.lol
- HN
I saw this [0] pretty cool thread by user revskill, and wanted a quicker way to search through it, but also to keep them all in one place so I can read them at my leisure whenever I get time. Right now is like 60 lines of Ruby using Nokogiri, but I will certainly look into it further down the line and improve the list. There's a cronjob checking the thread every 12 hours but I will eventually shut that down and it will become static after that. There are some really awesome blogs in there. I really recommend going through the list, it made my day. [0] "Could you share your personal blog…
Life & fun · 2023 · dm.hn
A safe space for your thoughts, private, local, p2p & open
Commerce · 2023 · anytype.io
