Alternatives
Products that do what A benchmark + latency sim for LLM db queries: ClickHouse / Postgres does
- 1FT
May 2026 · github.com
- 2CL
Me with my friend Vitaly Ludvichenko made an experiment to combine ClickHouse server and client to make a self-contained program running a database engine and processing data without a server: https://github.com/ClickHouse/ClickHouse/pull/150 Development continued in the past 6 years, and now clickhouse-local becomes a swiss-army knife for data processing. Say "ffmpeg" for datasets and more. It can resemble textql, octosql, dsq, duckdb, trdsql, q, datafusion-cli, spyql, but has better capabilities and performance. Here is a tutorial:…
2023 · clickhouse.com
- 3AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com
- 4LL
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai
- 5
- 6FL
Hi HN community, I have been working on benchmarking publicly available LLMs these past couple of weeks. More precisely, I am interested on the finetuning piece since a lot of businesses are starting to entertain the idea of self-hosting LLMs trained on their proprietary data rather than relying on third party APIs. To this point, I am tracking the following 4 pillars of evaluation that businesses are typically look into: - Performance - Time to train an LLM - Cost to train an LLM - Inference (throughput / latency / cost per token) For each LLM, my aim is to benchmark them for…
2023 · github.com
- 7PT
PgQueuer is a minimalist, high-performance job queue library for Python, leveraging the robustness of PostgreSQL. Designed for simplicity and efficiency, PgQueuer uses PostgreSQL's LISTEN/NOTIFY to manage job queues effortlessly.
2024 · github.com
- 8PP
Hey! I'm Andrei. I got frustrated by how people tend to build overcomplicated backend systems, being "motivated" by big tech case studies and popular books. So, I started exploring lean architecture, and building my digital garden of ideas, approaches and data that align with this direction. Here I want to present one of the tools – Sizing tool for PostgreSQL. I've benchmarked PostgreSQL on different EC2 instances and disks, with different initial data sets to see performance that these instances can give you. And I've built a tool to visualize this data, which I welcome you to explore. So,…
Jul 2026 · postgres.saneengineer.com
- 9UD
Hey HN! I’m the founder of Unify, and we’ve just released our Model Hub, which provides a collection of LLM endpoints with live runtime benchmarks all plotted across time: https://unify.ai/hub A key finding is that static tabular runtime benchmarks for LLMs simply do not work. It’s necessary to take a time-series perspective, and plot the variations through time. We currently have 21 models provided by: Anyscale, Perplexity AI, Replicate, Together AI, OctoAI, Mistral AI and OpenAI, with more on the roadmap. We test across different regions (Asia, US, Europe), with varied…
2024
- 10DG
2023 · github.com
- 11
- 12AB
I created a web page to compare different analytical databases (both self-managed and services, open-source and proprietary) on a realistic dataset. It contains 20+ databases, each with installation and data loading scripts. And they can be compared to each other on a set of 43 queries, by data load time or by storage size. There are switches to select different types of databases for comparison - for example, only MySQL compatible or PostgreSQL compatible. If you play with the switches, many interesting details will be uncovered. Full description:…
2022 · benchmark.clickhouse.com
- 13DT
2021 · github.com
- 14DL
2014 · databaselabs.io
- 15MP
Hello HN, this is Sai and Kaushik from ClickHouse. Today we are launching a Postgres managed service that is natively integrated with ClickHouse. It is built together with Ubicloud (YC W24). TL;DR: NVMe-backed Postgres + built-in CDC into ClickHouse + pg_clickhouse so you can keep your app Postgres-first while running analytics in ClickHouse. Try it (private preview): https://clickhouse.com/cloud/postgres Blog w/ live demo: https://clickhouse.com/blog/postgres-managed-by-clickhouse Problem Across many fast-growing companies using Postgres,…
Jan 2026
- 16LS
Hi HN, I built llm.sql, an LLM inference framework that reimagines the LLM execution pipeline as a series of structured SQL queries atop SQLite. The motivation: Edge LLMs are getting better, but hardware remains a bottleneck, especially RAM (size and bandwidth). When available memory is less than the model size and KV cache, the OS incurs page faults and swaps pages using LRU-like strategies, resulting in throughput degradation that's hard to notice and even harder to debug. In fact, the memory access pattern during LLM inference is deterministic - we know exactly which weights are needed…
Apr 2026
- 17LF
100% bootstrapped new startup. It lets you fine tune Mistral-7B and SDXL. In particular, for the LLM fine tuning we implemented a dataprep pipeline that turns websites/pdfs/doc files into question-answer pairs for training the small LLM using an big LLM. It includes a GPU scheduler that can do finegrained GPU memory scheduling (Kubernetes can only do whole-GPU, we do it per-GB of GPU memory to pack both inference and fine tuning jobs into the same fleet) to fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization It's a pretty…
2023 · docs.helix.ml
- 18LA
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com
- 19TS
2016 · torodb.com
- 20UO
2018 · arangodb.com
- 21IT
2023 · github.com
- 22BA
2015 · github.com
- 23TT
I've built a type-safe semantic layer in code, for ClickHouse. If you're building analytics off ClickHouse in TypeScript, I would love your feedback. With hypequery there is no platform to adopt, no YAML sprawl. It runs where your app runs. Key features: - Define metrics once, reuse them everywhere: Declare dimensions and measures in one place and then pull from the same source of truth. - Compiles to ClickHouse SQL: No service, no proxy, no extra runtime to deploy. It's a library that generates SQL and runs where your app runs. - Multi-tenancy & Authentication ready: Cross-tenant queries…
Jun 2026 · github.com
- 24UN
We've been working hard for the last year on building HeavyIQ, an LLM-powered plugin to the GPU-accelerated HeavyDB database, that allows users to ask natural language questions and get SQL, visualizations, and natural language answers back. The LLM itself is fine-tuned on tens of thousands of question and answer pairs, with a major focus around building the model’s proficiency at performing spatial and temporal joins. You can try it for yourself on this live demo of 400M tweets and a handful of other datasets of interest:…
2024 · demo-heavyiq.heavy.ai
Ranked by how close each launch is in meaning, then by votes. Refine with a description →