Alternatives
Products that do what LLMUI on-premise AI plateform does
on-premise platform for managing and orchestrating LLM model
- 1
- 2
- 3
- 4
- 5
- 6
- 7LL
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai
- 8RL
Hello Hacker News! We're Yangqing, Xiang and JJ from lepton.ai. We are building a platform to run any AI models as easy as writing local code, and to get your favorite models in minutes. It's like container for AI, but without the hassle of actually building a docker image. We built and contributed to some of the world's most popular AI software - PyTorch 1.0, ONNX, Caffe, etcd, Kubernetes, etc. We also managed hundreds of thousands of computers in our previous jobs. And we found that the AI software stack is usually unnecessarily complex - and we want to change that. Imagine if you are a…
2023 · lepton.ai
- 9LS
Hi, I was a corporate lawyer for many years working with a lot of financial services and insurance companies. In practicing law, I noticed there was a lot of repetition in the tasks I was working on even as a highly paid attorney that could be automated. I wanted to solve the problem of dealing with a lot information and data in a practical way, using AI. This motivated me to start AI Bloks/LLMWare with my husband, who had a deep background in software and is a very early adopter of AI. We have been on this journey with our open source project LLMWare for the past 4 months, producing a…
2024 · github.com
- 10
- 11AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 12
- 13PE
Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…
Jan 2026 · github.com
- 14
- 15
- 16
- 17NL
Built this because I was tired of every AI tool shipping my data to someone else server n0x runs the full stack LLM inference via WebGPU, autonomous ReAct agents, RAG over your own docs, sandboxed Python execution via Pyodide all inside a single browser tab. No account No keys No backend Models download once, cache in IndexedDB permanently. Biggest challenge was context window budgeting for the agent loop and making the WASM vector search non-blocking. Happy to talk architecture. GitHub: https://github.com/ixchio/n0x | Live demo: https://n0x-three.vercel.app
Mar 2026 · n0xth.vercel.app
- 18
- 19
- 20JL
Hi HN! Here's a local-only stack I built over the weekend - hope it can be useful for you! I have been building a lot of AI apps - https://github.com/a16z-infra/ai-town https://github.com/a16z-infra/companion-app ... And there were definitely times I spent way too much $$ before deploying the app to production. So I was looking for a "local only" stack and found a few tools that worked well together. I used the following set of tools but may add more options later: - Inference: Ollama - VectorDB: Supabase pg-vector - LLM orchestration: langchain -…
2023 · github.com
- 21RA
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
- 22OS
We’re building an open-source tool that makes it easy to expose secure, LLM-optimized APIs on top of your structured data—without manually designing endpoints or worrying about compliance. AI agents and LLM-powered applications need structured access to data, but traditional APIs and databases weren’t built with AI workloads in mind. Our tool automatically generates APIs that: - Filter out PII & sensitive data to comply with GDPR, CPRA, SOC 2, and other regulations. - Provide traceability & auditing, so AI apps aren’t black boxes, and security teams stay in control. - Optimize for AI…
2025 · github.com
- 23
Enterprise AI with complete data sovereignty & control
Nov 2025
- 24
Local-first notebooks for executable LLM workflows.
Jul 2026 · icc-go.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →