Dev tools · alternatives · 2026
24 alternatives to GlassFlow – OSS streaming dedup and joins from Kafka to ClickHouse
Hi HN! We are Ashish and Armend, founders of GlassFlow. We just launched our open-source streaming ETL that deduplicates and joins Kafka streams before ingesting them to ClickHouse https://github.com/glassflow/clickhouse-etl Why we built this: Dedup with batch data is straightforward. You load the data into a temporary table. Then, find only the latest versions of the record through hashes or keys and keep them. After that, move the clean data into your main table. But have you tried this with streaming data? Users of our prev product were running real-time analytics…
GlassFlow is an open-source streaming ETL tool that deduplicates and joins Kafka streams before ingesting them into ClickHouse. It addresses the challenge of handling duplicate records in real-time data pipelines… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
We just released a major upgrade to SigNoz with support for Logs management based on ClickHouse. Would love to get any feedback from the community here on what you think and any questions you may have for us Many big companies like Uber and Cloudflare have been shifting to ClickHouse as their main workhorse for Logs management seeing much better performance. for e.g Cloudflare recently shifted from Elastic to ClickHouse and are seeing 8x improvement in memory/cpu resource requirement in ingestion. This is our first release with Logs support and we have added support for: - Filtering…
2022 · github.com · its alternatives →
- 2
Me with my friend Vitaly Ludvichenko made an experiment to combine ClickHouse server and client to make a self-contained program running a database engine and processing data without a server: https://github.com/ClickHouse/ClickHouse/pull/150 Development continued in the past 6 years, and now clickhouse-local becomes a swiss-army knife for data processing. Say "ffmpeg" for datasets and more. It can resemble textql, octosql, dsq, duckdb, trdsql, q, datafusion-cli, spyql, but has better capabilities and performance. Here is a tutorial:…
2023 · clickhouse.com · its alternatives →
- 3
Hello! For the past year I’ve been working on a fully-managed data warehouse built on Clickhouse. I built this because I was frustrated with how much work was required to run an OLAP database in prod: re-writing my app to do batch inserts, managing clusters and needing to look up special CREATE TABLE syntax every time I made a change. I found pricing for other warehouses confusing (what is a “credit” exactly?) and worried about getting capacity-planning wrong. I was previously building accounting software for firms with millions of transactions. I desperately needed to move from Postgres to…
2023 · github.com · its alternatives →
- 4
Hello Everyone! We built SQLFlow as a lightweight stream processing engine. We leverage DuckDB as the stream processing engine, which gives SQLFlow the ability to process 10's of thousands of messages a second using ~250MiB of memory! DuckDB also supports a rich ecosystem of sinks and connectors! https://sql-flow.com/docs/category/tutorials/ https://github.com/turbolytics/sql-flow We were tired of running JVM's for simple stream processing, and also of bespoke one off stream processors I would love your feedback, criticisms and/or…
Dec 2025 · sql-flow.com · its alternatives →
- 5
- 6
TableFlow▲306Open-source CSV importer, alternative to Flatfile
2023 · tableflow.com · its alternatives →
- 7
Estuary Flow▲225Connect your data, where you want it, in milliseconds
2023 · estuary.dev · its alternatives →
- 8PS
Hello HN, I am Sai Srirampur, one of the Co-founders of PeerDB. (https://github.com/PeerDB-io/peerdb). We spent the past 7 months building a solid experience to replicate data from Postgres to data warehouses. Now we're expanding to queues. PeerDB Streams provides a simple and native way to replicate changes as they happen in Postgres to Queues (Kafka, Redpanda, Google PubSub, etc). We use Postgres logical decoding to enable Change Data Capture (CDC). Blog post here: https://blog.peerdb.io/peerdb-streams-simple-native-postgres.... 10-min quickstart here:…
2024 · its alternatives →
- 9
Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
2024 · github.com · its alternatives →
- 10
- 11LV
2016 · alooma.com · its alternatives →
- 12
Hi there, Burak here. I built an open-source data copy tool called ingestr (https://github.com/bruin-data/ingestr) I did build quite a few data warehouses both for the companies I worked at, as well as for consultancy projects. One of the more common pain points I observed was that everyone had to rebuild the same data ingestion bit over and over again, and each in different ways: - some wrote code for the ingestion from scratch to various degrees - some used off-the-shelf data ingestion tools like Fivetran / Airbyte I have always disliked both of these approaches,…
2024 · github.com · its alternatives →
- 13
Get and manipulate data with a visual programming interface.
2017 · pipes.digital · its alternatives →
- 14
Optimal Quantile Approximation in Streams. Contribute to axiomhq/quantiles development by creating an account on GitHub.
2018 · github.com · its alternatives →
- 15
While building an IoT telemetry solution, we ran into hurdles with Clickhouse. For one, you can't insert quickly AND durably into Clickhouse without setting up something like Kafka, which gets complicated for quick projects wanting to make use of Clickhouse's powerful features. Then, trying to actually query Clickhouse and show data in a UI required a whole backend API to handle auth and permissions. We figured that all these parts together – fast, durable ingest, row-level and column-level security and roles, and realtime streaming – were a lot of scaffolding to have to rebuild for every…
Aug 2026 · wavehouse.dev · its alternatives →
- 16
Advanced Bloom Filter Based Algorithms for Efficient Approximate Data De-Duplication in Streams - jparkie/PDD
2016 · github.com · its alternatives →
- 17PC
Hi HN! Porter Cloud (https://porter.run/porter-cloud) is a Platform as a Service (PaaS) like Heroku, but we make it easy for you to migrate to AWS, Azure, or GCP when you're ready. Like Heroku, Porter takes care of a lot of generic DevOps work for you (like setting up CI/CD, containerizing your applications, autoscaling, SSL certificates, setting up a reverse proxy) and lets you deploy your apps with a few clicks — saving you a lot of time while developing. However, as you probably know, there’s a downside: platforms like this become constraining if and when your app…
2024 · its alternatives →
- 18
A highly configurable Google Cloud Dataflow pipeline that writes data into Google Big Query table from Pub/Sub - bomboradata/pubsub-to-bigquery
2017 · github.com · its alternatives →
- 19
Hey HN, I built StreamHouse, an open-source streaming platform that replaces Kafka's broker-managed storage with direct S3 writes. The goal: same semantics, fraction of the cost. How it works: Producers batch and compress records, a stateless server manages partition routing and metadata (SQLite for dev, PostgreSQL for prod), and segments land directly in S3. Consumers read from S3 with a local segment cache. No broker disks to manage, no replication factor to tune — S3 gives you 11 nines of durability out of the box. What's there today: - Producer API with batching, LZ4 compression, and…
Feb 2026 · github.com · its alternatives →
- 20
Stream Postgres to Apache Iceberg on S3 via logical replication, queryable over the Postgres wire protocol. - GitHub - viggy28/streambed: Stream Postgres to Apache Iceberg on S3 via logical replication, queryable over the Postgres wire protocol.
May 2026 · github.com · its alternatives →
- 21
Hey HN, I'm really proud to share with you my new open source project: Kestra https://github.com/kestra-io/kestra I created a few years ago a successful open source AKHQ project: https://github.com/tchiotludo/akhq (renamed from KafkaHQ) which has been adopted by big companies like Best Buy, Pipedrive, BMW, Decathlon and many more. 2300 stars, 120 contributors, 10M docker downloads, much more than I expected. Now let's talk about Kestra, an infinitely scalable orchestration and scheduling platform for creating, running, scheduling and monitoring…
2022 · github.com · its alternatives →
- 22
- 23
- 24
Open-Source Web UI for Apache Kafka Management. Contribute to provectus/kafka-ui development by creating an account on GitHub.
2021 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →