SwiftLake – Lightweight SQL Engine for Iceberg Tables
At Arcesium, we needed a way to run SQL-driven CRUD operations, manage slowly-changing dimensions, and handle schema evolution on our cloud data lakes—without the overhead of spinning up distributed clusters or relying on vendor-managed query services. That’s why we built SwiftLake, a lightweight open-source library that brings Apache Iceberg and DuckDB together: - Lightweight & cluster-free: Runs as a single Java service with embedded DuckDB—no distributed infra required. - Full CRUD and SCD1/SCD2 support: Reads, inserts/appends, updates, deletes, and merges with ACID guarantees.…
What it does
In the maker’s words, at launch
At Arcesium, we needed a way to run SQL-driven CRUD operations, manage slowly-changing dimensions, and handle schema evolution on our cloud data lakes—without the overhead of spinning up distributed clusters or relying on vendor-managed query services. That’s why we built SwiftLake, a lightweight open-source library that brings Apache Iceberg and DuckDB together: - Lightweight & cluster-free: Runs as a single Java service with embedded DuckDB—no distributed infra required. - Full CRUD and SCD1/SCD2 support: Reads, inserts/appends, updates, deletes, and merges with ACID guarantees. - Iceberg advantages: Schema evolution, partition evolution, and time travel. - Developer-friendly APIs: Simple interfaces to embed directly into apps or data workflows. - Cloud storage support: Works with S3 today, with GCS and ADLS coming soon. SwiftLake is a good fit for ETL/ELT pipelines, data quality checks, and analytical workloads where spinning up heavy distributed systems would be overkill. SwiftLake is open source under Apache 2.0: - GitHub: https://github.com/arcesium/swiftlake - Blog: https://medium.com/arcesium-engineering-blog/your-data-just-got-swifter-meet-swiftlake-7568cc85ee57 We’re just getting started—would love to hear your feedback and ideas from the HN community.
Does the same job
all alternatives →- TDTextdb.dev – simple data sharing for fun projects2020 · textdb.dev · ▲200
- FLFieldbook: Lightweight Database with a Spreadsheet UI and a REST API2015 · fieldbook.com · ▲335
- DRDuckDB + React = SQLRooms Framework for Analytics Apps Without Backend2025 · sqlrooms.org · ▲9
We’ve built SQLRooms, an open-source framework for creating single-node data analytics apps powered by DuckDB. It lets you build fully client-side, data-centric apps using React and DuckDB running in the browser (via WebAssembly) or in Electron. No server or backend is required — apps can work offline, preserve data privacy, and run queries on large datasets with sub-second performance. Features: - Query large datasets in browser with DuckDB (WASM) - Modular design for building composable data UIs (query editors, dashboards, notebooks, etc.) - Data privacy-preserving AI assistant that can…
- NONhost – Open source Firebase alternative with GraphQL2020 · nhost.io · ▲210
- AAAvionDB: A Decentralised Database with MongoDB-like Developer Interface2020 · github.com · ▲104
- 8R8base – Ready-to-Rock GraphQL API and Serverless Back End2019 · 8base.com · ▲84
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Launched alongside, September 2025
the whole month →
- AS
Commerce · Sep 2025 · anycrap.shop
- TE
I made a built-from scratch Wayland Compositor to display any GUI app* in the terminal! I think there is a lot of unexplored potential in custom Wayland compositors, a lot of really cool things you can embed existing applications into! So, I started with embedding apps into the terminal because that is the easiest input/output (output is just utf-8 and I use the great `chafa` library for that, and I just read from stdin for the input). If you have any other ideas for cool Wayland compositors, let me know. I purposedly wrote 80% the app in Typescript to appeal to the most developers and…
Dev tools · Sep 2025 · github.com
- IR
Years ago I stumbled across a basic version of this concept and it stuck with me. I knew if I was ever going to take on such a project, it would need to be flawless, but without coding experience it was just another idea that would never happen. By the end of 2024, as AI coding tools exploded everywhere, I finally had a way to make it real. I started from zero knowledge and spent months collaborating with AI agents as a learning experience. Every pixel and every function went through me. The AI translated what I asked for into code, but every decision was human. I didn't use existing OS…
AI · Sep 2025 · mitchivin.com
