Alternatives
Products that do what MemStitch – Zero-copy context bridging for vLLM (25x TTFT speedup) does
- 1MM
Hi HN! Erik here from Pig.dev, and today I'd like to share a new project we've just open sourced: Muscle Mem is an SDK that records your agent's tool-calling patterns as it solves tasks, and will deterministically replay those learned trajectories whenever the task is encountered again, falling back to agent mode if edge cases are detected. Like a JIT compiler, for behaviors. At Pig, we built computer-use agents for automating legacy Windows applications (healthcare, lending, manufacturing, etc). A recurring theme we ran into was that businesses already had RPA (pure-software scripts), and…
2025 · github.com
- 2
- 3
- 4
- 5TA
TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.). It’s written in C++ 17 with thin Python bindings, keeps the exact same BPE vocab/special-token rules, and focuses on raw speed. I’m teaching myself LLM internals by re-implementing the stack from first principles. Profiling TikToken’s Python/Rust implementation showed a lot of time was spent doing regex matching. Most of my perf gains come from a) using a faster jit-compiled regex engine; and b) simplifying the algorithm to forego regex matching special tokens at all.…
2025 · github.com
- 6MB
Hey HN! We're Deshraj and Taranjeet. We've been building working on a startup called Mem0, building an open-source memory layer for AI apps and agents (https://news.ycombinator.com/item?id=41447317). We also kept running into our own daily frustrations with AI assistants forgetting everything between conversations. Over a weekend, we decided to hack together a Chrome extension to solve this for ourselves. The problem was simple: we were constantly re-explaining our context across platforms when switching between ChatGPT, Claude, and Perplexity. Start a coding discussion in…
2024 · github.com
- 7ST
2024 · supermaven.com
- 8MC
Hi HN, Jack here! I'm one of the creators of MonkeyPatch, an easy tool that helps you build LLM-powered functions and apps that get cheaper and faster the more you use them. For example, if you need to classify PDFs, extract product feedback from tweets, or auto-generate synthetic data, you can spin up an LLM-powered Python function in <5 minutes to power your application. Unlike existing LLM clients, these functions generate well-typed outputs with guardrails to mitigate unexpected behavior. After about 200-300 calls, these functions will begin to get cheaper and faster. We've seen 8-10x…
2023 · github.com
- 9WT
After working with LLMs for long enough, I found myself wanting a lightweight utility for doing various small tasks to prepare inputs, locate information and create evaluators. This library is two things: a very simple model and utilities that inference it (eg. fuzzy deduplication). The target platform is CPU, and it’s intended to be light, fast and pip installable — a library that lowers the barrier to working with strings semantically. You don’t need to install pytorch to use it, or any deep learning runtimes. How can this be accomplished? The model is simply token embeddings that are…
2024 · github.com
- 10TV
May 2026 · github.com
- 11MO
Hey HN! We're Taranjeet and Deshraj, the founders of Mem0 (https://mem0.ai). Mem0 adds a stateful memory layer to AI applications, allowing them to remember user interactions, preferences, and context over time. This enables AI apps to deliver increasingly personalized and intelligent experiences that evolve with every interaction. There’s a demo video at https://youtu.be/VtRuBCTZL1o and a playground to try out at https://app.mem0.ai/playground. You'll need to sign up to use the playground – this helps ensure responses are more tailored to you by…
2024 · github.com
- 12
- 13TW
Apr 2026 · github.com
- 14BM
2016 · github.com
- 15MA
I've invented a new alternative to forking / vendoring / monkey-patching packages in Python. It's a bit like OverlayFS for Python modules - it allows you write modifications for a target module (lower) in a new module (upper), and have these combined in a new virtual module (mount). It works by rewriting imports using AST transformations, then running both the lower and upper module's code in the new Python module. This prevents polluting the global namespace when monkey-patching, and means if you want to make changes to a third-party package, you don't have to take on the…
Oct 2025 · github.com
- 16TS
Hi everyone, I just released an open source load testing tool for LLMs: https://github.com/twerkmeister/tokenflood === What is it and what problems does it solve? === Tokenflood is a load testing tool for instruction-tuned LLMs hat can simulate arbitrary LLM loads in terms of prompt, prefix, and output lengths and requests per second. Instead of first collecting prompt data for different load types, you can configure the desired parameters for your load test and you are good to go. It also let's you assess the latency effects of potential prompt parameter changes before…
Nov 2025 · github.com
- 17BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
Oct 2025 · docs.butter.dev
- 18MI
Hi HN, TL;DR Memex is a cross-platform desktop app for vibe coding. Think ChatGPT + Claude Code rolled into one. Why we built it: We love chat tools like Perplexity and ChatGPT. We also love coding agents, like in Cursor and Windsurf. We don’t like that web-based app builders are opinionated about tech stack and we can’t run them locally. So, we built Memex to be a chat tool + coding agent that supports any tech stack. What it can do today: Claude Code-like coding. Agentic web search / research. Pre-built templates (e.g. fullstack, iOS, python + modal, etc). Inline data analysis + viz.…
2025 · memex.tech
- 19LM
Feb 2026 · marketplace.visualstudio.com
- 20MA
May 2026 · github.com
- 21SF
We’ve just open-sourced SemHash, a lightweight package for semantic text deduplication. It lets you effortlessly clean up your datasets and avoid pitfalls caused by duplicate samples in semantic search, RAG, and machine learning. Main Features: - Fast and hardware friendly: Deduplicate datasets with millions of records in minutes, on a CPU. - Flexible: Works on single or multiple datasets (e.g., train/test deduplication), and multi-column data (e.g., Question-Answering datasets). - Lightweight: Minimal dependencies (largest is NumPy). - Explainable: Easily inspect duplicates and what…
2025 · github.com
- 22FM
2015 · github.com
- 23OL
I've been working on Fast LiteLLM - a Rust acceleration layer for the popular LiteLLM library - and I had some interesting learnings that might resonate with other developers trying to squeeze performance out of existing systems. My assumption was that LiteLLM, being a Python library, would have plenty of low-hanging fruit for optimization. I set out to create a Rust layer using PyO3 to accelerate the performance-critical parts: token counting, routing, rate limiting, and connection pooling. The Approach - Built Rust implementations for token counting using tiktoken-rs - Added lock-free data…
Nov 2025 · github.com
- 24LP
I built llmswap to solve a problem I kept hitting in hackathons - burning through API credits while testing the same prompts repeatedly during development. It's a simple Python package that provides a unified interface for OpenAI, Anthropic, Google Gemini, and local models (Ollama), with built-in response caching that can cut API costs by 50-90%. Key features: - Intelligent caching with TTL and memory limits - Context-aware caching for multi-user apps - Auto-fallback between providers when one fails - Zero configuration - works with environment variables from llmswap import LLMClient client…
2025 · pypi.org
Ranked by how close each launch is in meaning, then by votes. Refine with a description →