Mixlayer – code and deploy LLM prompts using JavaScript
Hi HN, I'm excited to introduce Mixlayer, a platform I've been working on over the past 6 months that allows you to code and deploy prompts using simple JavaScript functions. Mixlayer recreates the developer experience of using LLMs locally without having to do all of the local setup yourself. I originally came up with this idea when using LLMs on my MacBook and thought it’d be cool to build a product that makes it easy for everyone. It compiles your code to a WASM binary and runs it alongside a custom inference stack I wrote in Rust. When you integrate LLMs in this way, your code and the…
What it does
In the maker’s words, at launch
Hi HN, I'm excited to introduce Mixlayer, a platform I've been working on over the past 6 months that allows you to code and deploy prompts using simple JavaScript functions. Mixlayer recreates the developer experience of using LLMs locally without having to do all of the local setup yourself. I originally came up with this idea when using LLMs on my MacBook and thought it’d be cool to build a product that makes it easy for everyone. It compiles your code to a WASM binary and runs it alongside a custom inference stack I wrote in Rust. When you integrate LLMs in this way, your code and the model share a common context window that stays open for the duration of your program’s execution. I find many common prompting patterns become much simpler when applied in this way versus using a generic OpenAI-style inference API. Some cool features: * Tool calling: LLM has direct access to your code, just pass objects containing functions and their descriptions * Hidden tokens: Mark certain tokens as "hidden" to recreate long-running reasoning and iterative refinement operations like gpt-4o. * Output constraints: Use regular expressions to constrain the generated text * Instant deployment: we can host your prompts behind an API that we scale for you Tech details: * Built on Huggingface's candle crate * Supports continuous batching and multi-GPU for larger models * WASM allows me to support for more prompt languages easily in the future Models: * Free tier: Llama 3.1 8b (on NVIDIA L4s, shared resources) * Paid tier: Faster models on A100s (soon H100 SXMs) * Llama 3.1 70b (currently gated due to resource constraints, requires 8xH100 SXMs) Future: * Vision models * More elaborate decoding methods (e.g. beam) * Multiple model prompts (routing/spawning/forking/joining) I’m happy to discuss any of the internal/technical details around how I built this. Thank you for your time and feedback!
Does the same job
all alternatives →- LBlambdaprompt – build, compose and call templated LLM prompts2022 · github.com · ▲9
For the past few months I've been building a lot of things with LLMs (GPT-3, Codex, etc.) as I've been trying to push them to their limits (especially towards applying them to the tabular data domain) When working on this, I've found there are some common patterns for solving problems (templating, chaining, functional-programming style operations, etc.) As I've iterated, I've come to believe that a functional style interface is likely going to power a new wave of systems I'm calling "prompt-machines"(systems where the core new unit of work is a "named" LLM prompt, extending the "function"…
- ELExperiment ▴ LLM UI for developers with tool use visualization2025 · github.com · ▲8
Hey HN! I built Experiment to solve a common frustration in LLM development: the lack of proper tools for prompt engineering experimentation. Here's what makes it different: Key Features: - Load and edit chat completion logs from CSV files - Fork and modify specific conversation entries - Run inference via Anthropic, Mistral, and OpenAI - Define custom tools using JSONSchema format - Visual tool usage analysis with collapsible, sorted key-value pairs - Full mobile support and available as installable PWA Technical Highlights: - Built with React using custom isomorphic architecture -…
- PAPromptL, a templating language designed for LLM prompting2024 · promptl.ai · ▲7
Hey HN! We just launched PromptL: a templating language built to simplify writing complex prompts for LLMs like GPT-4 and Claude. Why PromptL? Creating dynamic prompts for LLMs can get tricky, even with standardized APIs that use lists of messages and settings. While these formats are consistent, building complex interactions with custom logic or branching paths can quickly become repetitive and hard to manage as prompts grow. PromptL steps in to make this simple. It allows you to define and manage LLM conversations in a readable, single-file format, with support for control flow and…
- ODObject database for LLMs that persists across chats (MCP server)2025 · dry.ai · ▲9
I’d like to use LLMs for remembering all kinds of things: fitness, to-do lists, contacts, bug reports, research links, whatever. But there is no way to do that now. For example, if I find a great coding tutorial in chat, or tell it how much I ran yesterday, it forgets that when I close the chat. Even if I keep the chat history, I still need to scour through lots of messages to find the data I want. Ideally, Claude would remember all this, and I’d be able to find it later with ease. This is what my team built. It is a collaborative database you add to any LLM that supports MCP. (Claude Code,…
- HPHorizon – Programmatic Prompt Generation and LLM Configurations2023 · gethorizon.ai · ▲7
Hi HN. I heard you like dev tools and AI, so we wanted to share our project that we’ve been working on. We’re working on Horizon [1] - a higher level abstraction for LLMs so that developers can spend less time trying to grapple with LLMs to make them work and more time with users. This is the starting feature set which takes an auto-ML approach to identify the optimal LLM model, hyperparameters, and prompt - instead of just giving you the tooling to figure it out yourself. You can read more about it in our documentations. Our view is that as LLMs become increasingly commoditized and prompts…
- XRXybrid – run LLM and speech locally in your app (no back end, Rust)Mar 2026 · github.com · ▲6
Hi HN, We built Xybrid, a Rust library for running LLM + speech pipelines directly inside your app, no server, no daemon, just one binary. We started building it while working on a privacy-focused LLM app with Tauri and realized there wasn’t a straightforward way to embed models directly into shipped applications without relying on a separate server process. Xybrid links into your process like any other library. It supports GGUF / ONNX / CoreML and integrates with Flutter, Swift, Kotlin, Unity, and Tauri, letting you run pipelines like speech → LLM → speech in a single call. On…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 16d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, October 2024
the whole month →
Softr for Notion▲925Turn Notion databases into portals & apps with no code
Dev tools · 2024 · softr.io
One inbox for all your work discussions
Work · 2024 · generalcollaboration.com