TRI·TFM v3.0 Framework
Deterministic, LLM-as-a-Judge evaluation framework.
TRI·TFM v3.0 Framework launched on March 13, 2026.
What it does
An open-source, mathematically proven evaluation pipeline for LLMs and RAG systems. We eliminate "metric hallucination" by locking T=0.0 and applying a dynamic weight matrix (Bal = 0.75F - 0.25B) to score Facts, Bias, and Narrative deterministically.
Contribute to aisarus/tri-tfm-framework development by creating an account on GitHub.
TRI·TFM v3.0 is an open-source, mathematically proven evaluation framework for Large Language Models (LLMs) running on an LLM-as-a-Judge architecture. Unlike traditional evaluation tools (Ragas, DeepEval, TruLens) that suffer from "metric hallucination" and arbitrary weighting, TRI·TFM v3.0 introduces a Deterministic Zero-Variance Evaluator ( $T=0.0$ ) and an empirically defended dynamic balance formula ( $Bal = 0.75F - 0.25B$ ). git clone https://github.com/yourusername/tri-tfm-framework.git cd tri-tfm-framework pip install -r requirements.txt Set your API keys in .env : python app.py --dataset domain_prompts_p1.csv 🧬 The 4 Axes of TRI·TFM v3.0 Facts (F): Density and accuracy of…from github.com
Does a similar job
all alternatives →
Open-source model and scorecard for measuring hallucinations in LLMs2023 · vectara.com · ▲65Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
Demystifying Advanced RAG Pipelines2023 · github.com · ▲131I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
Fine-grained stylistic control of LLMs using model arithmetic2023 · github.com · ▲85We developed a new framework that enables flexible control of generated text in language models. By combining several models and/or system prompts in one mathematical formula, it lets you tweak your style and combine model outputs with ease. A handy tool for those working with LLMs, looking for more fine-grained control of stylistic output. More details in our paper: https://arxiv.org/abs/2311.14479. Feedback and potential applications are welcome.
Opik, an open source LLM evaluation framework2024 · github.com · ▲86Hey HN! I'm Caleb, one of the contributors to Opik, a new open source framework for LLM evaluations. Over the last few months, my colleagues and I have been working on a project to solve what we see as the most painful parts of writing evals for an LLM application. For this initial release, we've focused on a few core features that we think are the most essential: - Simplifying the implementation of more complex LLM-based evaluation metrics, like Hallucination and Moderation. - Enabling step-by-step tracking, such that you can test and debug each individual component of your LLM application,…
I implemented evals metrics for LLMs that runs locally on your machine2023 · github.com · ▲22The LLM Evaluation Framework. Contribute to confident-ai/deepeval development by creating an account on GitHub.
A new benchmark for testing LLMs for deterministic outputsApr 2026 · interfaze.ai · ▲60When building workflows that rely on LLMs, we commonly use structured output for programmatic use cases like converting an invoice into rows or meeting transcripts into tickets or even complex PDFs into database entries. The model may return the schema you want, but with hallucinated values like `invoice_date` being off by 2 months or the transcript array ordered wrongly. The JSON is valid, but the values are not. Structured output today is a big part of using LLMs, especially when building deterministic workflows. Current structured output benchmarks (e.g., JSONSchemaBench) only validate…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Product Hunt launch ↗
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
Littlebird▲709Littlebird remembers everything across your meetings, messages, docs, and apps, then takes action so you can recall any detail and never lose context.
AI · Mar 2026 · littlebird.ai
