Alternatives
Products that do what Optimize Databricks SQL does
I launched my startup's Snowflake optimization product here about two years ago, and we've just released our second product: a Databricks SQL optimizer. The optimizations are basically kubernetes for data warehousing: we take over autoscaling and cluster selection to increase utilization without impacting latency. The scheduler is backed by ML models that predict runtime and capacity, which means we can run machines hotter than the providers can and thereby cut costs. More info here: https://espresso.ai/post/launching-our-databricks-sql-optimi...
- 1SO
Hello! For the past year I’ve been working on a fully-managed data warehouse built on Clickhouse. I built this because I was frustrated with how much work was required to run an OLAP database in prod: re-writing my app to do batch inserts, managing clusters and needing to look up special CREATE TABLE syntax every time I made a change. I found pricing for other warehouses confusing (what is a “credit” exactly?) and worried about getting capacity-planning wrong. I was previously building accounting software for firms with millions of transactions. I desperately needed to move from Postgres to…
2023 · github.com
- 2TA
I built a SQLite VFS in Rust that serves cold queries directly from S3 with sub-second performance, and often much faster. It’s called turbolite. It is experimental, buggy, and may corrupt data. I would not trust it with anything important yet. I wanted to explore whether object storage has gotten fast enough to support embedded databases over cloud storage. Filesystems reward tiny random reads and in-place mutation. S3 rewards fewer requests, bigger transfers, immutable objects, and aggressively parallel operations where bandwidth is often the real constraint. This was explicitly inspired…
Mar 2026 · github.com
- 3S5
I'm a former YC founder, and I've been working with some friends from Google to build Espresso AI, an ML-powered Snowflake optimizer. We use LLMs to analyze and predict your SQL workload and run your warehouses more efficiently. Our first few customers are seeing Snowflake savings from 30% to 70%. We're launching out of beta, and if your team uses Snowflake we'd like to help you cut down your bill. You can set up Espresso in under 15 minutes with the instructions here: https://espresso.ai/onboarding Before turning anything on we'll send you a savings estimate based on your…
2024
- 4AA
2021 · arc.tripl.ai
- 5AB
I created a web page to compare different analytical databases (both self-managed and services, open-source and proprietary) on a realistic dataset. It contains 20+ databases, each with installation and data loading scripts. And they can be compared to each other on a set of 43 queries, by data load time or by storage size. There are switches to select different types of databases for comparison - for example, only MySQL compatible or PostgreSQL compatible. If you play with the switches, many interesting details will be uncovered. Full description:…
2022 · benchmark.clickhouse.com
- 6
- 7
- 8SA
Hi HN, We're Luke and Phillip, and we're building Spice.ai OSS - a lightweight, portable runtime, built in Rust and powered by Apache DataFusion to locally materialize, accelerate, and query data tables sourced from any database, data warehouse or data lake. Phillip and I first introduced Spice on Show HN in September 2021. Since then, we’ve been schooled and humbled in every way building 100TB+ data and ML systems for the https://spice.ai cloud platform. Along with our customers, we struggled with getting fast, low-latency, high-concurrency SQL query within a budget, accessing and…
2024 · github.com
- 9PP
Hey! I'm Andrei. I got frustrated by how people tend to build overcomplicated backend systems, being "motivated" by big tech case studies and popular books. So, I started exploring lean architecture, and building my digital garden of ideas, approaches and data that align with this direction. Here I want to present one of the tools – Sizing tool for PostgreSQL. I've benchmarked PostgreSQL on different EC2 instances and disks, with different initial data sets to see performance that these instances can give you. And I've built a tool to visualize this data, which I welcome you to explore. So,…
Jul 2026 · postgres.saneengineer.com
- 10
- 11
- 12
- 13
Connect DecisionBox to your Databricks to validate findings
May 2026 · decisionbox.io
- 14DE
tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized Hey HN, We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost. Now a lot of workloads DuckDB is used for were normally…
2024 · github.com
- 15
- 16AT
A bunch of developers and myself have created RepliByte - an open-source tool to seed a development database from a production database. Features: - Support data backup and restore for PostgreSQL, MySQL and MongoDB - Replace sensitive data with fake data - Works on large database (> 10GB) (read Design) - Database Subsetting: Scale down a production database to a more reasonable size - Start a local database with the prod data in a single command - On-the-fly data (de)compression (Zlib) - On-the-fly data de/encryption (AES-256) - Fully stateless (no server, no daemon) and lightweight…
2022
- 17
- 18
- 19
- 20OS
Hey HN, I am the founder of Tensorlake. Prototyping LLM applications have become a lot easier, building decision making LLM applications that work on constantly updating data is still very challenging in production settings. The systems engineering problems that we have seen people face are - 1. Reliably process ingested content in real time if the application is sensitive to freshness of information. 2. Being able to bring in any kind of model, and run different parts of the pipeline on GPUs and CPUs. 3. Fault Tolerance to ingestion spike, compute infrastructure failure. 4. Scaling compute,…
2024 · getindexify.ai
- 21II
Two weeks ago I was on my babymoon in Corfu, Greece. While in transit, I was overseeing a GSoC intern submit an important feature to my array database library, Xarray-SQL. He added `to_dataset()`, which completed the roundtrip between thinking of array data in a tabular model simultaneously as gridded rasters (the premise of the project is that every Nd array can be mapped to 2d, where orthogonal dims of the Nd array are just primary keys of a tabular representation). We discussed in chat, now that this feature existed, what demos could we make that would prove this data model works? With…
Jul 2026 · github.com
- 22
- 23MZ
The overall idea is to chart out the thousands of Mini PCs by benchmark and reveal the Pareto Front so you can get the most Compute per Dollar. Definitely a labor of love as I have a number of Mini PCs for my "homelab" (TrueNAS, piHole, Plex, basic stuff). It uses Gemini to extract specs from listings (since they're not often strongly categorized). Quick blog post here: https://luke.zip/posts/pareto-pcs/
Jun 2026 · minipcs.zip
- 24LF
100% bootstrapped new startup. It lets you fine tune Mistral-7B and SDXL. In particular, for the LLM fine tuning we implemented a dataprep pipeline that turns websites/pdfs/doc files into question-answer pairs for training the small LLM using an big LLM. It includes a GPU scheduler that can do finegrained GPU memory scheduling (Kubernetes can only do whole-GPU, we do it per-GB of GPU memory to pack both inference and fine tuning jobs into the same fleet) to fit model instances into GPU memory to optimally trade off user facing latency with GPU memory utilization It's a pretty…
2023 · docs.helix.ml
Ranked by how close each launch is in meaning, then by votes. Refine with a description →