Alternatives
Products that do what OpenDistil — Distillation at Scale does
Distillation-as-a-Service infrastructure
- 1
We recently used DeepSeek V4 Flash as a teacher for finance tasks with GPT-OSS-120B. Distillation works well on this problem. At a constrained 8k token budget, our self-distilled 120B scores 83.61% on FinanceReasoning, above Kimi K3 (81.93%) and Inkling (65.13%). We released the 20B open weights. With V4 as the teacher though, we realized it would be timely to measure if the censorship characteristic of it transferred to the distilled version of the base model. tl;dr it didn't, the teacher answered politically sensitive questions 7 SDs differently than expected, but the distilled model's…
Jul 2026 · ctgt.ai
- 2
- 3
- 4
- 5
- 6
Hi HN, we built an open source model gateway. It's a single place to manage our own self hosted, frontier, and open source models in one place. It’s is rust native, built for concurrency, and implements all the config quirks across models and providers (streaming formats, tool calls, model parameters, rate limits, and different error behavior). The gateway adds under 1 ms for BYOK requests and under 2 ms when Experiential supplies the provider key. It has every major inference provider, and 1000+ models refreshed daily via a codex agent that opens a PR. Compared to other similar projects…
10d ago · github.com
- 7
- 8DA
Over the past few months, I have built a distillation toolkit that supports cross-tokenizer distillation (e.g., distilling from LLaMA to Qwen vocab, or others). This approach has worked well on reasoning datasets like AIME, and we’ve validated on models like Phi and Qwen. We’ve also integrated Modal for quick deployment (with $30/month credits to try it out). Would love any feedback! GitHub: https://github.com/agokrani/distillKitPlus Docs: https://distillkitplus.mintlify.app/
2025 · github.com
- 9
- 10
- 11
- 12
- 13OS
Hey HN, I am the founder of Tensorlake. Prototyping LLM applications have become a lot easier, building decision making LLM applications that work on constantly updating data is still very challenging in production settings. The systems engineering problems that we have seen people face are - 1. Reliably process ingested content in real time if the application is sensitive to freshness of information. 2. Being able to bring in any kind of model, and run different parts of the pipeline on GPUs and CPUs. 3. Fault Tolerance to ingestion spike, compute infrastructure failure. 4. Scaling compute,…
2024 · getindexify.ai
- 14
- 15AU
2023 · github.com
- 16RL
Hello Hacker News! We're Yangqing, Xiang and JJ from lepton.ai. We are building a platform to run any AI models as easy as writing local code, and to get your favorite models in minutes. It's like container for AI, but without the hassle of actually building a docker image. We built and contributed to some of the world's most popular AI software - PyTorch 1.0, ONNX, Caffe, etcd, Kubernetes, etc. We also managed hundreds of thousands of computers in our previous jobs. And we found that the AI software stack is usually unnecessarily complex - and we want to change that. Imagine if you are a…
2023 · lepton.ai
- 17
- 18MI
I've been working on a platform that uses LLMs to build maintain and manage k8s clusters on any cloud. The system writes infra as code to your Github repos and automatically containerizes and scales any services (public or private). The goal is to give your average engineer a vercel-like deployment experience for any service in any language at minimal cost. We have humans involved at the moment auditing LLM outputs and keeping an eye on clusters. We are looking for folks who may be thinking about their first infra/devops hire. Just connect your github and your cloud provider. The system…
2024 · milkinfrastructure.com
- 19OS
2024 · github.com
- 20OS
Hi HN! I’m Farah, co-founder of Dioptra.ai. We just open sourced katiML (https://github.com/dioptra-ai/katiml) this week and wanted to get your take. katiML is a vector+data lake to debug, curate and version AI data. With katiML, teams avoid the “garbage in, garbage out” effect by taking control over the quality of their data. They quickly and effectively curate high quality data for training, fine-tuning, and fixing hallucinations and edge cases. Features include: - Data Curation: interactive embedding visualization and similarity search, mislabeling and hallucination…
2023 · loom.com
- 21PO
2023 · github.com
- 22
- 23
- 24WM
We've recently made our product, ozma.io, open-source. It's a CRM/ERP platform for building enterprise systems. We believe that AI will soon handle implementing most of the boilerplate and UIs in the specialized business software. Just look at what lovable.dev does today! Soon products which make creating business software easier for developers will become obsolete, or transform into "libraries" to be used by AIs. We are losing this race, so we go the second route — publish everything and go on building other products on top of it. GitHub repo URL:…
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →