Advanced Chunking in JavaScript/TypeScript with Chonkie
Hi HN, We’re Shreyash and Bhavnick. We built Chonkie, an open-source library for advanced chunking and embedding of text and code. It was previously Python-only, but we just released a TypeScript version: https://github.com/chonkie-inc/chonkie-ts Many AI projects in JS/TS (like those using Vercel's AI SDK or Mastra) rely on basic text splitters. But better chunking = better retrieval = better performance. That’s what Chonkie is built for. Current native chunkers (in TS): - Code Chunker – handles Python, TypeScript, etc. - Recursive Chunker – rule-based, hierarchical…
In plain words
Chonkie is an open-source TypeScript library for splitting and preparing text and code for AI applications. It provides multiple chunking strategies including code-aware, recursive, token-based, and sentence-level splitting, with support for custom tokenizers and delimiters across languages. Designed for JavaScript and TypeScript projects using AI frameworks, Chonkie aims to improve retrieval quality and AI performance compared to basic text splitters.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi HN, We’re Shreyash and Bhavnick. We built Chonkie, an open-source library for advanced chunking and embedding of text and code. It was previously Python-only, but we just released a TypeScript version: https://github.com/chonkie-inc/chonkie-ts Many AI projects in JS/TS (like those using Vercel's AI SDK or Mastra) rely on basic text splitters. But better chunking = better retrieval = better performance. That’s what Chonkie is built for. Current native chunkers (in TS): - Code Chunker – handles Python, TypeScript, etc. - Recursive Chunker – rule-based, hierarchical splitting - Token Chunker – split by token count (fully customizable) - Sentence Chunker – split on sentence boundaries. Delimiters are customizable, so it works for multiple languages. All chunkers support custom tokenizers, chunk overlap, delimiters, and more. Coming soon in native TS (already available via the API client): - Semantic Chunker – splits texts wherever it detects a shift in meaning. - SDPM Chunker – merges semantically similar disjoint chunks - Late Chunker – generates context-aware embeddings for each chunk - Slumber Chunker – LLM-refined recursive chunks. Significantly reduces token usage (and thus cost) while maximizing chunk quality. - Embeddings Refinery - Embed chunks with any embedding model - Overlap Refinery – Create overlaps between consecutive chunks for better context preservation. Chonkie is free, open-source, and MIT licensed. GitHub: https://github.com/chonkie-inc/chonkie-ts We’d love your feedback, ideas, or contributions. Thanks!
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, May 2025
the whole month →
- C9
Life & fun · 2025 · felixrieseberg.github.io
