Llama2 Embeddings FastAPI Server

In plain words

Llama2 Embeddings FastAPI Server is a self-hosted REST API that converts text strings into embedding vectors using Llama2 and similar language models. Users can submit individual strings to receive JSON-formatted embeddings, compare two strings for similarity scores, or upload plaintext and PDF files to generate embeddings for every sentence. The server comes with two pre-configured models that download automatically on first run, and supports easy addition of custom models via HuggingFace URLs. It requires minimal setup and is fully self-contained.

written from the facts on this page · September 2026

From the sources

A FastAPI service for semantic text search using precomputed embeddings and advanced similarity measures, with built-in support for various file types through textract. - Dicklesworthstone/swiss_army_llama

In the maker’s words, at launch

Author here. I just wanted a quick and easy way to easily submit strings to a REST API and get back the embedding vectors in JSON using Llama2 and other similar LLMs, so I put this together over the past couple days. It's very quick and easy to set up and totally self-contained and self-hosted. You can easily add new…

• 
  Swiss Army Llama2023 · github.com · ▲6

  This project originally started out with a focus on easily generating embeddings from Llama2 and other llama_cpp (gguf) models and storing them in a database, all exposed via a convenient REST api (hence why it was originally called the not-very-catchy name of "llama_embeddings_fastapi_service" when I submitted it a few weeks ago). But since then, I've added a lot more functionality: 1) New endpoint for generating text completions (including specifying custom grammars, like JSON). 2) Get all the embeddings for an entire document--can be any kind of document (plaintext, PDFs, doc/.docx,…

• 
  Wordllama – Things you can do with the token embeddings of an LLM2024 · github.com · ▲370

  After working with LLMs for long enough, I found myself wanting a lightweight utility for doing various small tasks to prepare inputs, locate information and create evaluators. This library is two things: a very simple model and utilities that inference it (eg. fuzzy deduplication). The target platform is CPU, and it’s intended to be light, fast and pip installable — a library that lowers the barrier to working with strings semantically. You don’t need to install pytorch to use it, or any deep learning runtimes. How can this be accomplished? The model is simply token embeddings that are…

• 
  Llama 22023 · developer.meta.com · ▲263

  The next generation of Meta's open source LLM

• 
  Agents.json – OpenAPI Specification for LLMs2025 · github.com · ▲212

  Hey HN, we’re building an open specification that lets agents discover and invoke APIs with natural language, built on the OpenAPI standard. agents.json clearly defines the contract between LLMs and API as a standard that's open, observable, and replicable. Here’s a walkthrough of how it works: https://youtu.be/kby2Wdt2Dtk?si=59xGCDy48Zzwr7ND. There’s 2 parts to this: 1. An agents.json file describes how to link API calls together into outcome-based tools for LLMs. This file sits alongside an OpenAPI file. 2. The agents.json SDK loads agents.json files as tools for an LLM that…

• L3
  Llama 3.3 70B Sparse Autoencoders with API access2024 · goodfire.ai · ▲201
• 
  Embed Anything2025 · vercel.com · ▲13

  a cool product to embed, search, for RAG

• 
  /dev/null▲468

  a social network for devs

  Dev tools · 2d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 25d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 22d ago · openbot.run

• 
  Talanoa▲340

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 20d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 27d ago · corespeed.io

• 
  Resend▲1,325

  An entire collection of pre-built components with a much faster development experience.

  Dev tools · 2023 · resend.com

• 
  STORI AI▲1,316

  Your ideas become visually compelling branded social posts

  AI · 2023 · storiai.com

• 
  Kombai▲1,079

  A new AI model that you can prompt with UI designs

  AI · 2023 · kombai.com

• 
  User Persona Generator▲1,014

  Understand your audience without 50 interviews

  AI · 2023 · founderpal.ai

• 
  Lottielab▲869

  Create and ship lottie animations to sites and apps faster

  Dev tools · 2023 · lottielab.com

• 
  LLMs can generate valid JSON 100% of the time▲854

  Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…

  AI · 2023 · github.com