Faking SIMD to Search and Sort Strings 5x Faster

In plain words

This CPython library accelerates string searching and sorting operations, achieving up to 5x performance improvements for large text processing tasks. Built for developers working with massive text datasets like multi-terabyte newline-delimited files, it uses a portable bit-level heuristic that simulates SIMD behavior without requiring actual SIMD instructions. The approach maintains consistent code across different hardware architectures while leveraging 64-bit words to operate at 8-bit granularity, making it particularly useful for handling CommonCrawl and similar large-scale textual data.

written from the facts on this page · September 2026

From the sources

StringZilla uses SIMD tricks to hit 16 GB/s substring search, beating standard libraries by 5-10x and parsing multi-terabyte files Python couldn

A few years back, I found a simple trick in tandem with SIMD intrinsics to truly unleash the power of contemporary CPUs. I put the strstr of LibC and the std::search of the C++ Standard Templates Library to the test and hit a throughput of around 1.5 GB/s for substring search on a solitary core. Not too shabby, right? But imagine, that the memory bandwidth could theoretically reach a striking 10-15 GB/s per core. Going on this wild hypothesis that if the first 4 characters of a string are a match, it’s highly probable that the rest will match too, I brought this idea to life for x86 SSE, AVX, AVX-512, and Arm Neon. The result? I achieved memory bandwidth that outdid the standard by a…from ashvardanian.com

In the maker’s words, at launch

I want to share a really dumb, but very practical project I have packaged this summer, to perform operations on strings much faster. I was using Python to work with a multi-terabyte newline-delimited file. Reading, splitting, and shuffling it was a nightmare. So, I wrapped a trivial hardware-friendly heuristic I've…

• 
  I built a hardware processor that runs Python2025 · runpyxl.com · ▲983

  Hi everyone, I built PyXL — a hardware processor that executes a custom assembly generated from Python programs, without using a traditional interpreter or virtual machine. It compiles Python -> CPython Bytecode -> Instruction set designed for direct hardware execution. I’m sharing an early benchmark: a GPIO test where PyXL achieves a 480ns round-trip toggle — compared to 14-25 micro seconds on a MicroPython Pyboard - even though PyXL runs at a lower clock (100MHz vs. 168MHz). The design is stack-based, fully pipelined, and preserves Python's dynamic typing without static type restrictions.…

• 
  RapidFuzz – A fast string matching library for Python2020 · github.com · ▲134

  Rapid fuzzy string matching in Python using various string metrics - rapidfuzz/RapidFuzz

• FS
  Fast suffix arrays in Python2017 · louisabraham.github.io · ▲72
• 
  TurboRLE – Turbo Run Length Encoding using SIMD2015 · github.com · ▲23

  TurboRLE-Fastest Run Length Encoding. Contribute to powturbo/Turbo-Run-Length-Encoding development by creating an account on GitHub.

• 
  A SIMD-accelerated noise library in Rust2018 · github.com · ▲73

  SIMD noise library for Rust. Contribute to verpeteren/rust-simd-noise development by creating an account on GitHub.

• 
  Gsubpy, an interpreter for subset of Python, written in Go2023 · github.com · ▲65

  Recently, I write an interpreter for subset of Python using Go. I haven't seen source code of CPython, all is just written according to my understanding of Python so there may be some unusual things. Hope you can play fun with it. Any suggestion is welcome, glad to get some feedback.

• 
  /dev/null▲468

  a social network for devs

  Dev tools · 2d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 25d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 22d ago · openbot.run

• 
  Talanoa▲340

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 20d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 27d ago · corespeed.io

• 
  Resend▲1,325

  An entire collection of pre-built components with a much faster development experience.

  Dev tools · 2023 · resend.com

• 
  STORI AI▲1,316

  Your ideas become visually compelling branded social posts

  AI · 2023 · storiai.com

• 
  Kombai▲1,079

  A new AI model that you can prompt with UI designs

  AI · 2023 · kombai.com

• 
  User Persona Generator▲1,014

  Understand your audience without 50 interviews

  AI · 2023 · founderpal.ai

• 
  Lottielab▲869

  Create and ship lottie animations to sites and apps faster

  Dev tools · 2023 · lottielab.com

• 
  LLMs can generate valid JSON 100% of the time▲854

  Outlines is a Python library that focuses on text generation with large language models. Brandon and I are not LLM experts and started the project a few months ago because we wanted to understand better how the generation process works. Our original background is probabilistic, relational and symbolic programming. Recently we came up with a fast way to generate text that matches a regex (https://blog.normalcomputing.ai/posts/2023-07-27-regex-guide...). The basic idea is simple: regular expressions have an equivalent Deterministic-Finite Automaton (DFA) representation. We…

  AI · 2023 · github.com