Okapi – a metrics engine based on open data formats
Hi All I wanted to share an early preview of Okapi an in-memory metrics engine that also integrates with existing datalakes. Modern software systems produce a mammoth amount of telemetry. While we can discuss whether or not this is necessary, we can all agree that it happens. Most metrics engines today use proprietary formats to store data and don’t use disaggregated storage and compute. Okapi changes that by leveraging open data formats and integrating with existing data lakes. This makes it possible to use standard OLAP tools like Snowflake, Databricks, DuckDB or even Jupyter / Polars…
In plain words
Okapi is an in-memory metrics engine designed to process telemetry data from modern software systems. It works with existing data lakes using open data formats rather than proprietary ones, allowing users to analyze data with standard OLAP tools like Snowflake, Databricks, and DuckDB. The system separates storage and compute, reducing vendor lock-in and operational overhead while enabling flexible scaling and custom analysis workflows.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
Hi All I wanted to share an early preview of Okapi an in-memory metrics engine that also integrates with existing datalakes. Modern software systems produce a mammoth amount of telemetry. While we can discuss whether or not this is necessary, we can all agree that it happens. Most metrics engines today use proprietary formats to store data and don’t use disaggregated storage and compute. Okapi changes that by leveraging open data formats and integrating with existing data lakes. This makes it possible to use standard OLAP tools like Snowflake, Databricks, DuckDB or even Jupyter / Polars to run analysis workflows (such as anomaly detection) while avoiding vendor lock-in in two ways - you can bring your own workflows and have a swappable compute engine. Disaggregation also reduces Ops burden of maintaining your own storage and the compute engine can be scaled up and down on demand. Not all data can reside in a data-lake/object store though - this doesn’t work for recent data. To ease realtime queries Okapi first writes all metrics data to an in memory store and reads on recent data are served from this store. Metrics are rolled up as they arrive which helps ease memory pressure. Metrics are held in-memory for a configurable retention period after which it gets shipped out to object storage/datalake (currently only Parquet export is supported). This allows fast reads on recent data while offloading query-processing for older data. On benchmarks queries on in-memory data finish in under a millisecond while having write throughput of ~280k samples per second. On a real deployment, there’d be network delays so YMMV. Okapi it is still early — feedback, critiques, and contributions welcome. Cheers !
Does the same job
all alternatives →
- PBPenca – branchable, versioned OLTP+OLAP on one open copy of your dataJul 2026 · github.com · ▲14
Hi HN! This is an early proof of concept of a branchable, versioned OLTP + OLAP database that runs on a single, open copy of your data in object storage. If you are familiar with Databricks' LTAP (https://www.databricks.com/company/newsroom/press-releases/d...) announcement from June, you can think of this as aspiring to be a fully open source, Apache 2.0 LTAP alternative with additional data versioning/auditability guarantees that enable audit, as_of queries, and (eventually) revert straight out of the box. How it works: 1. Writes land in vanilla postgres…
We Beat MLPerf: Modern Storage for KV Offload and LLM Training18h ago · theopenlake.com · ▲35High performance storage engine for efficient LLM inference and GPU Training.
More commerce this month
the category →
Compare your startup equity grant for free.
Commerce · 26d ago · equitybee.com
Launched alongside, August 2025
the whole month →
- IS
I built the world's most impractical 1000-pixel display and anyone in the world can draw on it. It draws a single pixel at a time and takes 30-60 minutes to complete a single image. Anyone can participate in the project by voting for the next image to be drawn, and submitting images. https://kilopx.com/
Work · 2025 · benholmen.com
- KT
Kitten TTS is an open-source series of tiny and expressive text-to-speech models for on-device applications. We are excited to launch a preview of our smallest model, which is less than 25 MB. This model has 15M parameters. This release supports English text-to-speech applications in eight voices: four male and four female. The model is quantized to int8 + fp16, and it uses onnx for runtime. The model is designed to run literally anywhere eg. raspberry pi, low-end smartphones, wearables, browsers etc. No GPU required! We're releasing this to give early users a sense of the latency and voices…
Dev tools · 2025 · github.com
- IW
I was wondering how I can arrange objects along a spherical helix path, and read some articles on it. I ended up learning about parametric equations again, and make this visualization to document what I learned: https://visualrambling.space/moving-objects-in-3d/ feel free to visit and let me know what you think!
Life & fun · 2025 · visualrambling.space
- TC
For HTML Day 2025 [1], I made a web service that displays the current sky at your approximate location as a CSS gradient. Colours are simulated on-demand using atmospheric absorption and scattering coefficients. Updates every minute, without the use of client-side JavaScript. Source code and additional information is available on GitHub: https://github.com/dnlzro/horizon [1] https://html.energy/html-day/2025/index.html
Dev tools · 2025 · sky.dlazaro.ca