KubeAI: Private Open AI on K8s

What it does

✅️ Drop-in replacement for OpenAI with API compatibility 🚀 Serve OSS LLMs on CPUs or GPUs ⚖️ Autoscaling with scale from 0 🛠️ Zero dependencies (no Istio, Knative, etc.) 🤖 Operates OSS model servers (vLLM and Ollama) 🔋 Chat UI included

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text. - kubeai-project/kubeai

• 
  I built an OSS alternative to Azure OpenAI services2023 · github.com · ▲200

  Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services. Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure. "How can I track LLM spend per API key?" "Can I create a development OpenAI API key with limited access for Bob?" "Can I see my LLM spend breakdown by models and endpoints?" "Can I create 100 OpenAI API keys that my students could use in a classroom…

• 
  Open source alternative to ChatGPT and ChatPDF-like AI tools2023 · github.com · ▲234

  Hey everyone, We have been building SecureAI Tools -- an open-source application layer for ChatGPT and ChatPDF-like AI tools. It works with locally running LLMs as well as with OpenAI-compatible APIs. For local LLMs, it supports Ollama which supports all the gguf/ggml models. Currently, it has two features: Chat-with-LLM, and Chat-with-PDFs. It is optimized for self-hosting use cases and comes with basic user management features. Here are some quick demos: * Chat with documents using OpenAI's GPT3.5 model: https://www.youtube.com/watch?v=Br2D3G9O47s * Chat with documents…

• 
  LLM Gateway2025 · llmgateway.io · ▲296

  I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…

• 
  Lemonfox.ai2023 · lemonfox.ai · ▲157

  The fast, easy and cheap OpenAI alternative

• 
  Open-Source Alternative to OpenAI Platform, for Local Models2025 · github.com · ▲93

  Hi everyone, we’re a small team, supported by Mozilla, who are working on re-imagining a UI for training, tuning and testing local LLMs. Everything is open source. If you’ve been training your own LLMs or have always wanted to, we’d love for you to play with the tool and give feedback on what the future development experience for LLM engineering could look like.

• 
  SwiftAI – open-source library to easily build LLM features on iOS/macOS2025 · github.com · ▲74

  We built SwiftAI, an open-source Swift library that lets you use Apple’s on-device LLMs when available (Apple opened access in June), and fall back to a cloud model when they aren’t available — all without duplicating code. SwiftAI gives you: - A single, model-agnostic API - An agent/tool loop - Strongly-typed structured outputs - Optional chat state Backstory: We started experimenting with Apple’s local models because they’re free (no API calls), private, and work offline. The problem: not all devices support them (older iPhones, Apple Intelligence disabled, low battery, etc.). That…

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 18d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 18d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 20d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 21d ago · stillwet.art

• 
  Beam.win▲378

  Run AI agents on one Mac and use its windows on another

  AI · 5d ago · beam.win

• 
  FastRouter.ai▲364

  Route requests to the right LLM for cost, latency & quality

  AI · 23d ago · fastrouter.ai

• 
  Wordware▲9,848

  Your tool for building AI agents with natural language

  AI · 2024 · wordware.ai

• 
  If YouTube had actual channels▲2,741

  If Youtube had actual channels.

  Life & fun · 2024 · ytch.xyz

• 
  Me.bot▲1,627

  The inspiring companion for your life

  AI · 2024 · me.bot

• 
  Upmetrics AI▲1,197

  Business plan & financial forecast using AI

  Work · 2024 · upmetrics.co

• 
  Jupitrr AI▲1,195

  Jupitrr AI turns raw talking videos into scroll-stopping content for coaches and consultants. Research, script, record, edit, and publish in one app.

  AI · 2024 · jupitrr.com

• 
  InstantDB – A Modern Firebase▲1,145

  Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…

  Dev tools · 2024 · github.com