Alternatives
Products that do what Catsu: A unified Python client for embedding APIs does
We open-sourced catsu, a Python client for embedding APIs. The problem: every embedding provider has a different SDK with different bugs. OpenAI has undocumented token limits. VoyageAI's retry logic was broken until September. Cohere breaks downstream libraries every release. LiteLLM's embedding support is minimal. catsu provides: - One API for 11 providers (OpenAI, Voyage, Cohere, Jina, Mistral, Gemini, etc.) - Bundled database of 50+ models with pricing, dimensions, and benchmark scores - Built-in retry with exponential backoff - Automatic cost tracking per request - Full async support…
- 1WT
After working with LLMs for long enough, I found myself wanting a lightweight utility for doing various small tasks to prepare inputs, locate information and create evaluators. This library is two things: a very simple model and utilities that inference it (eg. fuzzy deduplication). The target platform is CPU, and it’s intended to be light, fast and pip installable — a library that lowers the barrier to working with strings semantically. You don’t need to install pytorch to use it, or any deep learning runtimes. How can this be accomplished? The model is simply token embeddings that are…
2024 · github.com
- 2LS
I built this library because langchain was too bloated and I needed a simple abstraction to call multiple LLM APIs. litellm has two functions - completion(), embedding()
2023 · github.com
- 3LE
Author here. I just wanted a quick and easy way to easily submit strings to a REST API and get back the embedding vectors in JSON using Llama2 and other similar LLMs, so I put this together over the past couple days. It's very quick and easy to set up and totally self-contained and self-hosted. You can easily add new models to it by simply adding the HuggingFace URL to the GGML format model weights. Two models are included by default, and these are automatically downloaded the first time it's run. It lets you not only submit text strings and get back the embeddings, but also to compare two…
2023 · github.com
- 4FB
2019 · fastapi.tiangolo.com
- 5IS
Everything that would be here is in the README. I hope this gets big, it has tons of potential.
2013 · github.com
- 6AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 7AR
2017 · github.com
- 8AO
Hi, I'm Ben, the co-creator of Embedbase. Embedbase lets you use OpenAI Embeddings and Pinecone seamlessly. For example, you can add Embedbase to your app and pair it with GPT3 to allow people to search using natural language (e.g. How many workouts did I complete last week?), or simply expanding your current search experience beyond full-text search (e.g. looking for "similar" documents in Notion to find other related information) Managing embeddings is uncharted territory, we needed to discover the best practices ourselves. Now we're happy to share our learnings with Embedbase. Shoot if…
2023 · embedbase.xyz
- 9AU
2014 · embedkit.com
- 10BO
2022 · usebruno.com
- 11AE
2015 · apiembed.com
- 12PL
Library makes requests asynchronously across models, so you can spend a lot of $$ quickly if you want XD. But seriously I hope this enables folks to create and run evals (especially safety ones) a lot easier than before.
2024 · github.com
- 13AC
2025 · niquests.readthedocs.io
- 14SH
Drop in replacement for OpenAI's embedding API. Can be used with official OpenAPI libraries. Written in python3
2023 · github.com
- 15RC
Hey, OpenAI recently released "assistants" - These have RAG built into the API, which means that you can provide up to 10.000 files to the assistant without the overhead of dealing with vector databases, splitting files into chunks and more. They work surprisingly well, so I've built a free simple tool to embed them on websites. It has no dependencies and simply uses fetch to communicate to my backend which proxies openai. In the future I plan to add more widget embedding options (currently only chat is available). Let me know if you have any specific questions about either the tool or…
2024 · rispose.com
- 16IB
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com
- 17PA
2018 · github.com
- 18LT
I wanted to share a project I've been working on for the past few weeks: llgtrt. It's a Rust implementation of a HTTP REST server for hosting Large Language Models using llguidance library for constrained output with NVIDIA TensorRT-LLM. The server is compatible with the OpenAI REST API and supports structured JSON schema enforcement as well as full context-free grammars (via Guidance). It's similar in spirit to the Python-based TensorRT-LLM OpenAI server example but written entirely in Rust and built with constraints in mind. No Triton Inference Server involved. This also serves as a demo…
2024 · github.com
- 19IE
Quick note on how it works and how I've done my batch embedding engine IgniteMS. The whole thing runs as one process using Rust, reading input, tokenizing, packing batches, keeping the queue full. TensorRT handles inference. Python is only as a wrapper. I built it this way because when you use more than couple of GPUs, the GPUs stop being the problem. CPU cannot feed them fast enough. One A100 can go through batches faster than Python can tokenize and feed, so the GPU just sits there idle waiting for work. Most of my time went into optimizing this. At 8 GPUs that was basically the entire…
Jun 2026 · github.com
- 20PA
Hi HN, I’m experimenting with a small Python library called PicoFlow for building LLM agent workflows using a lightweight DSL. I’ve been using tools like LangChain and CrewAI, and wanted to explore a simpler, more function-oriented way to compose agent logic, closer to normal Python control flow and async functions. PicoFlow focuses on: - composing async functions with operators - minimal core and few concepts to learn - explicit data flow through a shared context - easy embedding into existing services A typical flow looks like: flow = plan >> retrieve >> answer await flow(ctx) Patterns…
Jan 2026
- 21PF
Introducing embeds.ai: an embedding playground to compare how embedding models work on a real world use case (retrieval augmented generation for Wikipedia articles + Elad Gil's High growth handbook) A few weeks ago, Shreyan and I were looking for an embedding model to use for RAG. We eventually came across the MTEB leaderboard, but we struggled to understand the benchmark scores. We wanted a tool to test various embedding models with example queries on real-world datasets. After unsuccessfully looking for such a “playground”, we decided to just build one ourselves! We embedded HuggingFace’s…
2023 · embeds.ai
- 22OM
Hi all, so I've been working on this low to no code platform that allows you to spin up deep learning workloads(I'm talking LLM's, Huggingface models, etc), interconnect a bunch of them, and deploy them as API's. The idea essentially came up early in September, when experimenting with combining a Huggingface based BERT model with an LLM at work, and I realized it would be cool if I could do that instantly(especially since it was a prototype). At the time, I was considering a platform that could essentially help you train deep learning models without any code. It was my observation that much…
2024 · github.com
- 23AB
Hello HN, new user here, so please let me know if I break some rules. Currently I've been working on training reinforcement learning agents, and OpenAI gym, while is great, runs only one agent at a time. Hence I decided to extend it. I built a wrapper around OpenAI gym, such that it now runs several environments concurrently. All while (mostly) having the same call signature as OpenAI gym. And it is published to PyPI for anyone interested. For more details, please visit: https://github.com/Chimpan-Z/agymc Feedback really appreciated! Have a good day everyone!
2020
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →