Phoenix OSS – Applying LLM Spans, Traces, and Evals for AI Insights
We are the developers of Phoenix, which we released in April of this year with a goal of bringing LLM observability to the notebook. In the time since, the growth of LLM frameworks and complex agent workflows led us to add support for LLM spans and traces and introduce a simple Eval harness for testing the data from those spans. The latest Traces & Spans release of Phoenix offers: -Out of the box tracing for LlamaIndex and LangChain -Fully local execution, no data sent anywhere, outside of your own LLM calls -Ability to get a common dataframe format across frameworks back to a notebook for…
In plain words
Phoenix OSS is an observability tool for large language model development that runs locally in notebooks. It provides out-of-the-box tracing for LlamaIndex and LangChain frameworks, capturing spans and traces without sending data outside the user's environment except for LLM API calls. The tool converts trace data into a common dataframe format for analysis and includes a lightweight evaluation harness for testing LLM outputs. It also offers benchmarking scripts for optimizing retrieval configurations like chunk size and ranking parameters. Phoenix is designed for developers building LLM applications who need visibility into their AI workflows.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
We are the developers of Phoenix, which we released in April of this year with a goal of bringing LLM observability to the notebook. In the time since, the growth of LLM frameworks and complex agent workflows led us to add support for LLM spans and traces and introduce a simple Eval harness for testing the data from those spans. The latest Traces & Spans release of Phoenix offers: -Out of the box tracing for LlamaIndex and LangChain -Fully local execution, no data sent anywhere, outside of your own LLM calls -Ability to get a common dataframe format across frameworks back to a notebook for Evals -Code based LLM Eval harness: light, simple, and fast -Benchmarking scripts for retrieval setup - chunk size, K, and retrieval approach https://github.com/Arize-ai/phoenix We love to hear more from the community about what kind of LLM applications you are building, whether you are using a framework or building from scratch, and how are you running/measuring LLM Evals today. Thinking a lot about what “non-framework” integrations look like. Also genuinely interested in people's opinions of LLM spans and traces versus OTEL. Is the divergence because of something intrinsic to the application or as a community are we reinventing the wheel? Colab if you want to test it out: https://colab.research.google.com/github/Arize-ai/phoenix/bl...
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, October 2023
the whole month →
Nudge 2.0▲1,051In-app experiences to activate, retain, & understand users
Growth · 2023 · nudgenow.com
Unlock AI magic for elevated customer engagement, fast
AI · 2023 · tiledesk.com
- OD
Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website.
Dev tools · 2023 · github.com