Predibase Inference Engine

What it does

The Predibase Inference Engine, powered by LoRA eXchange, Turbo LoRA, and seamless GPU autoscaling, serves fine-tuned SLMs at speeds 3-4 times faster than traditional methods and confidently handles enterprise workloads of 100s of requests per second.

• 
  Groq®2024 · groq.com · ▲237

  Hyperfast LLM running on custom built GPUs

• 
  Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series MacJul 2026 · github.com · ▲919

  Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…

• 
  We made our own inference engine for Apple Silicon2025 · github.com · ▲186

  We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.

• 
  Tiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205

  Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm

• 
  General ComputeMay 2026 · generalcompute.com · ▲315

  AI models that run on an inference cloud optimized for speed

• 
  Speeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419

  Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 17d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 16d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 18d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 20d ago · stillwet.art

• 
  Onepin▲384

  AI voiceover with every name, price and date said right

  AI · 30d ago · onepin.ai

• 
  PixVerse R2▲381

  A real-time world model you can explore and change

  AI · 28d ago · world.pixverse.video

• 
  buzzabout▲1,267

  Audience insights from 1B+ online discussions in 2 mins

  AI · 2024 · buzzabout.ai

• 
  bolt.new▲1,230

  Build and scale high-performing websites & apps using your words. Join millions and start building today.

  AI · 2024 · bolt.new

• 
  Feta▲1,217

  Run smarter stand-ups, build better products

  AI · 2024 · feta.io

• 
  Trag▲955

  AI code review companion

  AI · 2024 · usetrag.com

• 
  Softr for Notion▲923

  Turn Notion databases into portals & apps with no code

  Dev tools · 2024 · softr.io

• 
  General Collaboration▲910

  One inbox for all your work discussions

  Work · 2024 · generalcollaboration.com