LLaMA tokenizer that runs in browser

In plain words

LLaMA tokenizer that runs in browser is a web-based tool that tokenizes text using Meta's LLaMA model vocabulary directly in the browser without requiring a server. It's designed for developers and researchers working with LLaMA language models who need to understand how their text gets converted into tokens. The tool runs locally in the browser, making it fast and private for experimentation and development purposes.

written from the facts on this page · September 2026

• 
  LLaMA 3 tokenizer runs in the browser2024 · belladoreai.github.io · ▲10

  Web site created using create-react-app

• 
  TokenDagger – A tokenizer faster than OpenAI's Tiktoken2025 · github.com · ▲281

  TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.). It’s written in C++ 17 with thin Python bindings, keeps the exact same BPE vocab/special-token rules, and focuses on raw speed. I’m teaching myself LLM internals by re-implementing the stack from first principles. Profiling TikToken’s Python/Rust implementation showed a lot of time was spent doing regex matching. Most of my perf gains come from a) using a faster jit-compiled regex engine; and b) simplifying the algorithm to forego regex matching special tokens at all.…

• 
  I wrote an open-source browser alternative for Computer Use for any LLM2024 · github.com · ▲180

  Hey HN, I made Browser-Use, an open-source tool that lets (all Langchain supported) LLMs execute tasks directly in the browser just with function calling. It allows you to build agents that interact with web elements using natural language prompts. We created a layer that simplifies website interaction for LLMs by extracting xPaths and interactive elements like buttons and input fields (and other fancy things). This enables you to design custom web automation and scraping functions without manual inspection through DevTools. Hasn't this been done a lot of times? Good question, as a general…

• 
  Run Llama 3.1 8B in the browser2024 · app.wiz.chat · ▲23
• 
  Llama2 Embeddings FastAPI Server2023 · github.com · ▲178

  Author here. I just wanted a quick and easy way to easily submit strings to a REST API and get back the embedding vectors in JSON using Llama2 and other similar LLMs, so I put this together over the past couple days. It's very quick and easy to set up and totally self-contained and self-hosted. You can easily add new models to it by simply adding the HuggingFace URL to the GGML format model weights. Two models are included by default, and these are automatically downloaded the first time it's run. It lets you not only submit text strings and get back the embeddings, but also to compare two…

• 
  Llama 22023 · developer.meta.com · ▲263

  The next generation of Meta's open source LLM

• 
  /dev/null▲484

  a social network for devs

  Dev tools · 3d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 28d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 26d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 23d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 20d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 28d ago · corespeed.io

• 
  Non.io, a Reddit-like platform Ive been working on for the last 4 years▲1,943

  Heya HN, I've been working on a reddit-like platform as my primary side project for the last few years. Doing a (very) soft launch today, mainly because I want to use it to encourage discussion of alternatives. How non.io works: 1. Free to browse, paid to interact. 2. Minimum subscription is $2 (though you can choose more). I take $1 to run the servers, everything left gets split evenly between everything you upvote that month. It's a simple model, but I hope it's a better one than the freemium model we've been relying on for the last few years. Fundamentally I feel like any ad-supported…

  Commerce · 2023 · non.io

• 
  Sivi AI▲938

  Generative AI to magically turn text to visual designs

  AI · 2023 · sivi.ai

• 
  Typefully 2.0▲931

  Effortlessly publish on Twitter and LinkedIn, now with AI

  Growth · 2023 · typefully.com

• 
  Olvy 2.0▲826

  Analyze user feedback 10x faster with AI

  AI · 2023 · olvy.co

• 
  SlashPage▲755

  Create websites in seconds to build your first 1K community

  Growth · 2023 · slashpage.com

• 
  Unicorn Platform AI▲751

  Easy to use AI-powered, drag-and-drop website, SAAS landing, directory, and blog builder for startups, mobile apps, and SaaS. 100% no code, no design skills needed.

  AI · 2023 · unicornplatform.com