Alternatives
Products that do what Swift-powered AI apps on iOS: Real-time multimodal semantic search does
I was recently playing with Apple's CoreML and had several painful observations on tooling. It's not enough for a long read but should be for an HN post. In short, you can take a simple BERT-like encoder model in PyTorch, convert it into an f32 CoreML checkpoint, and run it on CPU or GPU, but not NPU. Let's unpack this. Having a simple and extensible format to exchange common ANN architectures is a big issue for anyone who uses more than one framework or programming language to run the same model. ONNX is the closest we have to that standard, but it's hard to call anything Protobuf-related…
- 1OS
Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…
Jul 2026 · github.com
- 2
- 3
First TTS model to support all 22 Indic languages + English
2024
- 4WM
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com
- 5RA
Aug 2026 · github.com
- 6RT
I ported Microsoft's TRELLIS.2 (4B parameter image-to-3D model) to run on Apple Silicon via PyTorch MPS. The original requires CUDA with flash_attn, nvdiffrast, and custom sparse convolution kernels: none of which work on Mac. I replaced the CUDA-specific ops with pure-PyTorch alternatives: a gather-scatter sparse 3D convolution, SDPA attention for sparse transformers, and a Python-based mesh extraction replacing CUDA hashmap operations. Total changes are a few hundred lines across 9 files. Generates ~400K vertex meshes from single photos in about 3.5 minutes on M4 Pro (24GB). Not as fast as…
Apr 2026 · github.com
- 7AS
Hi HN, author here. SHARP is Apple's recent single-image 3D Gaussian splatting model (https://arxiv.org/abs/2512.10685). Their reference code is PyTorch + a pretty heavy pipeline; I wanted to see if it could run in a browser with no server hop, so I exported the predictor to ONNX and ran it via onnxruntime-web with the WebGPU EP. What works: drop in an image, get a .ply you can download or preview live, all on your machine — your image never leaves the tab. The model is large (~2.4 GB sidecar) so first load is slow on a cold cache, but inference itself is a few seconds on…
May 2026 · github.com
- 8SS
I wrote this tool to get familiar with CLIP model, I know many people have written similar tools with CLIP before, but I'm new to machine learning and writing a classic tool helps my study. The unusual thing with my version is, it is in pure Node.js, with the power of node-mlx, a Node.js machine learning framework. The repo in the link is mostly about implementing indexing and CLI, the code of the model implementation lives as a Node.js module: https://github.com/frost-beta/clip . Hope this helps other learners!
2024 · github.com
- 9SO
We built SwiftAI, an open-source Swift library that lets you use Apple’s on-device LLMs when available (Apple opened access in June), and fall back to a cloud model when they aren’t available — all without duplicating code. SwiftAI gives you: - A single, model-agnostic API - An agent/tool loop - Strongly-typed structured outputs - Optional chat state Backstory: We started experimenting with Apple’s local models because they’re free (no API calls), private, and work offline. The problem: not all devices support them (older iPhones, Apple Intelligence disabled, low battery, etc.). That…
2025 · github.com
- 10PO
Our company Vertex.AI has been working on this for a while but this is the first public release. We're starting with using PlaidML to bring OpenCL support to Keras and more frameworks, platforms, etc are coming. Yes, this means you can use use your AMD GPU for deep learning dev. Sorry, no Mac or Windows support yet although the brave can try building from source (it should work). http://vertex.ai/blog/announcing-plaidml https://github.com/plaidml/plaidml
2017
- 11
- 12EL
2023 · github.com
- 13MA
I've been exploring the (not so=) amazing potential of AI in coding and have compiled a list of tools. From AI-powered IDEs to code generators, this resource is my contribution to the community. I'm still on the fence about including txt2sql projects, as their functionality seems too basic to me. And I'm personally maintaining this, so your feedback is wellcome.
2025 · aicode.danvoronov.com
- 14ML
We’ve recently open-sourced Model2vec, a method to distill sentence transformers into static embeddings that outperform all previous approaches by a large margin on MTEB. Our new models set a new state-of-the-art for static embeddings. Main features: - Our best model (potion-base-8M) has only 8M parameters, which is ~30mb on disk - Inference is ~500x faster than the distilled base model (bge-base), on a CPU - New models can be distilled in 30 seconds on a CPU without requiring a dataset - just a vocabulary - Numpy-only inference: The packaged can be install the package with minimal…
2024 · github.com
- 15GC
Go bindings for CoreML that let you build and run ML models on Apple Silicon from Go, no Python required. You construct models programmatically using CoreML's MIL intermediate language, then run them on the Neural Engine, GPU, or CPU. Also works as a GoMLX backend.
Jan 2026 · github.com
- 16LA
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com
- 17RP
The excitement surrounding PrismML’s 1-bit/ternary Bonsai models has the industry closely watching how smartphone giants, particularly Apple, will implement LLMs on edge devices. Moving AI on-device is a brilliant and necessary strategy. It ensures absolute user privacy in alignment with EU regulations, fundamentally shifts the economics away from costly cloud inference, and paves the way for a significant hardware upgrade supercycle as users seek true AI-capable silicon. To create a smart on-device "Semantic Router," models need to reach the 27B+ parameter scale. Achieving this on a…
Jul 2026
- 18MA
Hey HN, We’ve been heads-down building MOSS - a semantic memory layer that brings AI-powered search and personalization fully on-device (No cloud | No latency | No data leaving the user’s device) We just launched a live demo showing MOSS running entirely in-browser, performing lightning-fast semantic search over local in-browser VectorDB. This unlocks a new class of privacy-first, hybrid AI experiences that work even without a server connection. If you’re curious about: - how to run AI search right inside the browser - the technical challenges behind on-device vector search - why we believe…
2025 · twitter.com
- 19UV
I want to share the most recent model release we have prepared. It's a Vision-Language understanding Transformer. It has 40% fewer parameters than vanilla CLIP while performing much better on text-to-image retrieval, where it's also beneficial that our output embeddings have 2x fewer dimensions (256 vs. 512). Moreover, it supports 21 languages, including popular English, Hindi, Chinese, Arabic, and lower-resource languages like Ukrainian, Hebrew, and Armenian. We have packed the library into ONNX and CoreML, providing PyTorch inference code for CPUs and GPUs and PopTorch code for Graphcore…
2023 · github.com
- 20OS
We're a group of engineers, AI/ML enthusiasts, and author of this paper https://openreview.net/forum?id=0pxiMpCyBtr who saw a closed door in AI/ML and decided to open it. This project is a PyTorch reimagining of Google's TensorFlow Lattice models, which despite being labeled open-source, were previously open in name only (you have to be a Googler to contribute). Also, side point…TensorFlow is dying https://thenextweb.com/news/why-tensorflow-for-python-is-dyi... Here's the deal: Lattice models excel in making AI interpretable—key for sectors where…
2023 · github.com
- 21FA
Apr 2026 · brilliant.design
- 22VA
Dear HN Community, I am a long time fan and first-time contributor. I just launched a developer focused semantic search platform and wanted to share it with the community. The idea is simple: upload structured or unstructured documents, select the fields you want to index and tag as metadata, and instantly get a clean search API you can use in your own app. Here is what it currently supports: - Manage your own tenants and projects - Upload .json and .txt files (support for .pdf, .docx, .xlsx, .yml, etc. coming soon) - Expose 3 APIs: search, upload document (embeddings), and delete document -…
2025 · aisearch.vpuna.com
- 23
- 24AA
Hi guys, For a few months now I've been working on a web GUI to build, visualise, train and share deep neural models. It's currently reaching a state where opening it for Beta release make sense. Currently the tool support: - Fully connected and Convolutional architecture - Cloud and local, saving / loading of models - Edit / delete layers - Visualise Convolutional layers filters - Freeze / Unfreeze layers - More datasets: Fashion MNIST, QuickDraw(10 and 30) The editor can be found here: https://aifiddle.io. Your feedback, ideas, suggestions are greatly useful, so…
2019
Ranked by how close each launch is in meaning, then by votes. Refine with a description →