KubeAI: Private Open AI on K8s

What it does

✅️ Drop-in replacement for OpenAI with API compatibility 🚀 Serve OSS LLMs on CPUs or GPUs ⚖️ Autoscaling with scale from 0 🛠️ Zero dependencies (no Istio, Knative, etc.) 🤖 Operates OSS model servers (vLLM and Ollama) 🔋 Chat UI included

• IB
  I built an OSS alternative to Azure OpenAI services2023 · github.com · ▲200

  Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services. Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure. "How can I track LLM spend per API key?" "Can I create a development OpenAI API key with limited access for Bob?" "Can I see my LLM spend breakdown by models and endpoints?" "Can I create 100 OpenAI API keys that my students could use in a classroom…

• 
  AskCodiNov 2025 · askcodi.com · ▲230

  Custom LLMs, without training. Use via openai compatible api

• OS
  Open source alternative to ChatGPT and ChatPDF-like AI tools2023 · github.com · ▲234

  Hey everyone, We have been building SecureAI Tools -- an open-source application layer for ChatGPT and ChatPDF-like AI tools. It works with locally running LLMs as well as with OpenAI-compatible APIs. For local LLMs, it supports Ollama which supports all the gguf/ggml models. Currently, it has two features: Chat-with-LLM, and Chat-with-PDFs. It is optimized for self-hosting use cases and comes with basic user management features. Here are some quick demos: * Chat with documents using OpenAI's GPT3.5 model: https://www.youtube.com/watch?v=Br2D3G9O47s * Chat with documents…

• 
  Lemonfox.ai2023 · ▲157

  The fast, easy and cheap OpenAI alternative

• OS
  Open-Source Alternative to OpenAI Platform, for Local Models2025 · github.com · ▲93

  Hi everyone, we’re a small team, supported by Mozilla, who are working on re-imagining a UI for training, tuning and testing local LLMs. Everything is open source. If you’ve been training your own LLMs or have always wanted to, we’d love for you to play with the tool and give feedback on what the future development experience for LLM engineering could look like.

• SO
  SwiftAI – open-source library to easily build LLM features on iOS/macOS2025 · github.com · ▲74

  We built SwiftAI, an open-source Swift library that lets you use Apple’s on-device LLMs when available (Apple opened access in June), and fall back to a cloud model when they aren’t available — all without duplicating code. SwiftAI gives you: - A single, model-agnostic API - An agent/tool loop - Strongly-typed structured outputs - Optional chat state Backstory: We started experimenting with Apple’s local models because they’re free (no API calls), private, and work offline. The problem: not all devices support them (older iPhones, Apple Intelligence disabled, low battery, etc.). That…

• 
  I trained a 125M model to autocomplete piano on-device▲598

  I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.

  AI · 18d ago · simedw.com

• 
  Astute▲585

  Automate your B2B brand going viral, with new media creators

  AI · 20d ago · company-app.joinastute.com

• 
  Grok Bot▲547

  AI teammates that you can give real work to

  AI · 27d ago · x.ai

• 
  Needle2: 14MB agentic LLM for phones, wearables, smart home and robots▲537

  Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…

  AI · 28d ago · cactuscompute.com

• 
  Monid▲478

  One wallet, every paid tool your agent needs

  AI · 8d ago · monid.ai

• 
  Clara AI SDR▲460

  Turn website visitors into qualified pipeline

  AI · 21d ago · clarasdr.ai

• 
  Wordware▲9,866

  Your tool for building AI agents with natural language

  AI · 2024 · wordware.ai

• IY
  If YouTube had actual channels▲2,741

  Life & fun · 2024 · ytch.xyz

• 
  Me.bot▲1,628

  The inspiring companion for your life

  AI · 2024 · me.bot

• 
  Upmetrics AI▲1,197

  Business plan & financial forecast using AI

  Work · 2024 · upmetrics.co

• 
  Jupitrr AI▲1,196

  Auto-generate B-roll visuals for content marketing videos

  AI · 2024 · jupitrr.com

• IA
  InstantDB – A Modern Firebase▲1,145

  Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…

  Dev tools · 2024 · github.com