Dev tools · alternatives · 2026
24 alternatives to Llama2 Embeddings FastAPI Server
Author here. I just wanted a quick and easy way to easily submit strings to a REST API and get back the embedding vectors in JSON using Llama2 and other similar LLMs, so I put this together over the past couple days. It's very quick and easy to set up and totally self-contained and self-hosted. You can easily add new models to it by simply adding the HuggingFace URL to the GGML format model weights. Two models are included by default, and these are automatically downloaded the first time it's run. It lets you not only submit text strings and get back the embeddings, but also to compare two…
Llama2 Embeddings FastAPI Server is a self-hosted REST API that converts text strings into embedding vectors using Llama2 and similar language models. Users can submit individual strings to receive JSON-formatted… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier. Llama2 Embeddings FastAPI Server launched in 2023; newer entries below may have overtaken it.
- 1
This project originally started out with a focus on easily generating embeddings from Llama2 and other llama_cpp (gguf) models and storing them in a database, all exposed via a convenient REST api (hence why it was originally called the not-very-catchy name of "llama_embeddings_fastapi_service" when I submitted it a few weeks ago). But since then, I've added a lot more functionality: 1) New endpoint for generating text completions (including specifying custom grammars, like JSON). 2) Get all the embeddings for an entire document--can be any kind of document (plaintext, PDFs, doc/.docx,…
2023 · github.com · its alternatives →
- 2
After working with LLMs for long enough, I found myself wanting a lightweight utility for doing various small tasks to prepare inputs, locate information and create evaluators. This library is two things: a very simple model and utilities that inference it (eg. fuzzy deduplication). The target platform is CPU, and it’s intended to be light, fast and pip installable — a library that lowers the barrier to working with strings semantically. You don’t need to install pytorch to use it, or any deep learning runtimes. How can this be accomplished? The model is simply token embeddings that are…
2024 · github.com · its alternatives →
- 3
Llama 2▲263The next generation of Meta's open source LLM
2023 · developer.meta.com · its alternatives →
- 4
Hey HN, we’re building an open specification that lets agents discover and invoke APIs with natural language, built on the OpenAPI standard. agents.json clearly defines the contract between LLMs and API as a standard that's open, observable, and replicable. Here’s a walkthrough of how it works: https://youtu.be/kby2Wdt2Dtk?si=59xGCDy48Zzwr7ND. There’s 2 parts to this: 1. An agents.json file describes how to link API calls together into outcome-based tools for LLMs. This file sits alongside an OpenAPI file. 2. The agents.json SDK loads agents.json files as tools for an LLM that…
2025 · github.com · its alternatives →
- 5
A few months ago, I benchmarked FastAPI on an i9 MacBook Pro. I couldn't believe my eyes. A primary REST endpoint to `sum` two integers took 6 milliseconds to evaluate. It is okay if you are targeting a server in another city, but it should be less when your client and server apps are running on the same machine. FastAPI would have bottleneck-ed the inference of our lightweight UForm neural networks recently trending on HN under the title "Beating OpenAI CLIP with 100x less data and compute". (Thank you all for the kind words!) So I wrote another library. It has been a while since I have…
2023 · github.com · its alternatives →
- 6
- 7L3
2024 · goodfire.ai · its alternatives →
- 8
- 9
Llama 3.3 70B▲347Llama 405B-level performance, at a fraction of the cost
2024 · developer.meta.com · its alternatives →
- 10
Hello HN! I've been thinking about the idea of a LLM thats a clone of me - instead of generating replies to be a helpful assistant, it generates replies that are exactly like mine. The concept's appeared in fiction numerous times (the talking paintings in Harry Potter that mimic the person painted, the clones in The Prestige), and I think with LLMs, there might actually be a possibility of us doing something like this! I've just released a fork of the facebookresearch/llama-recipes which allows you to fine-tune a Llama model on your personal WhatsApp conversations. This adaptation can…
2023 · github.com · its alternatives →
- 11
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support! - getumbrel/llama-gpt
2023 · github.com · its alternatives →
- 12
Google EmbeddingGemma 2free▲318An open, lightweight multimodal embedding model
16d ago · blog.google · its alternatives →
- 13
- 14
Hey hacker news, We’re the cofounders at Psychic.dev (http://psychic.dev) where we help companies connect LLMs to private data. With the launch of Llama 2, we think it’s finally viable to self-host an internal application that’s on-par with ChatGPT, so we did exactly that and made it an open source project. We also included a vector DB and API server so you can upload files and connect Llama 2 to your own data. The RAG in RAGstack stands for Retrieval Augmented Generation, a technique where the capabilities of a large language model (LLM) are augmented by retrieving information…
2023 · github.com · its alternatives →
- 15
- 16
We’ve just launched Gradient — an API that helps you build private LLMs that you own. We simplify inference and fine-tuning on open-source LLMs such as llama2, and you only pay by the token. Our API platform makes it possible for you to create private models with a single API call. Run inference on your fine tuned model instantly with no cold boot (and no need to pay for compute costs). The product is truly on demand - when you run fine tuning and inference on our platform, there's nearly 0 startup latency for these API calls. And you're not paying for the compute, you just pay for the…
2023 · gradient.ai · its alternatives →
- 17
Hi Hacker News! We’re Vadim and Chris from Highlight.io [1]. We do web app monitoring and are working on using LLMs/embeddings to add new functionality to our error monitoring product. Given that there’s a lot of founders/engineers using LLMs in their products, we figured we’d share how we built the new functionality, their impact on our workflows, and how you can try it out. Our goal was to build two features: (1) tagging errors (e.g. deeming an error as “authentication error” or a “database error”); and (2) grouping similar errors together (e.g. two errors that have a different…
2023 · github.com · its alternatives →
- 18
JS tokenizer for LLaMA 1 and 2. Contribute to belladoreai/llama-tokenizer-js development by creating an account on GitHub.
2023 · github.com · its alternatives →
- 19AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
- 20
I built LlamaPReview to solve a common frustration: most AI code reviewers either require complex setup or don't truly understand project context. Key differentiators: 1. One-click installation through GitHub Marketplace - no configuration needed 2. Analyzes your entire codebase first to understand: - Project structure - Coding patterns - Naming conventions - Architecture decisions 3. Completely free with no usage limits 4. Fully automated PR reviews with zero human intervention required Technical implementation: - Built on top of llama-github (my open source project) - Focuses on deep code…
2024 · github.com · its alternatives →
- 21
Hello hacker news, I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: https://github.com/BerriAI/litellm/ We’re open sourcing our implementation of liteLLM proxy: https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-... TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming What can liteLLM proxy do? - It’s a central place to…
2023 · github.com · its alternatives →
- 22
A generator for OpenAPI 3. Contribute to egorsmkv/openapi3-generator development by creating an account on GitHub.
2020 · github.com · its alternatives →
- 23
Provides a Kotlin/Java library to create a random json string - mangatmodi/RandomJson
2019 · github.com · its alternatives →
- 24
Auto-generated code snippets in many programming languages for your website, blog or API documentation. Made with Love by Kong.
2015 · apiembed.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →