Alternatives
Products that do what Streaming DataFrames–a Pandas-like syntax for real-time data does
Hey all! We’ve built a Pandas-like interface to make it easy to work with streaming data using what we call ‘Streaming DataFrames’. For example, suppose that you want to convert speed measurement units from meters per second to kilometers per hour With static data in Pandas, you’d do this: df["speed_km_h"] = df["speed_m_s"] * 3.6 With Streaming DataFrames, it’s pretty much the same thing… sdf["speed_km_h"] = sdf["speed_m_s"] * 3.6 …except it’s being done continuously and the updated records can be sent to an output topic in Kafka with almost no delay after they’ve been processed. You can…
- 1TY
Hey, guys. I've just made a plugin which turns your pandas dataframe into a tableau-style component. It allows you to explore the dataframe with easy drag-and-drop UI. You can use PyGWalker in Jupyter, Google Colab, or even Kaggle Notebook to easily explore your data and generate interactive visualizations. PyGWalker (pronounced like "Pig Walker", just for fun) is named as an abbreviation of "Python binding of Graphic Walker". Here are some links to check it out: The Github Repo: https://github.com/Kanaries/pygwalker Use PyGWalker in Kaggle:…
2023 · github.com
- 2
- 3DF
Hello Everyone! We built SQLFlow as a lightweight stream processing engine. We leverage DuckDB as the stream processing engine, which gives SQLFlow the ability to process 10's of thousands of messages a second using ~250MiB of memory! DuckDB also supports a rich ecosystem of sinks and connectors! https://sql-flow.com/docs/category/tutorials/ https://github.com/turbolytics/sql-flow We were tired of running JVM's for simple stream processing, and also of bespoke one off stream processors I would love your feedback, criticisms and/or…
Dec 2025 · sql-flow.com
- 4PS
Hello HN, I am Sai Srirampur, one of the Co-founders of PeerDB. (https://github.com/PeerDB-io/peerdb). We spent the past 7 months building a solid experience to replicate data from Postgres to data warehouses. Now we're expanding to queues. PeerDB Streams provides a simple and native way to replicate changes as they happen in Postgres to Queues (Kafka, Redpanda, Google PubSub, etc). We use Postgres logical decoding to enable Change Data Capture (CDC). Blog post here: https://blog.peerdb.io/peerdb-streams-simple-native-postgres.... 10-min quickstart here:…
2024
- 5RT
We are working on a product which makes stream processing easy for data analysts and data scientists. It’s a higher level alternative to Flink, Kafka Streams and Spark Streams. As well as making stream processing simpler, we also felt that lots of people were building these AWS stream processing platforms based on Kafka and something like Flink so wanted to offer this as a service. We would welcome any input on the space or approach: Home page: http://Timeflow.systems Demo: https://youtu.be/fGfgBaHXOkE
2020
- 6
- 7
- 8GO
Hi HN! We are Ashish and Armend, founders of GlassFlow. We just launched our open-source streaming ETL that deduplicates and joins Kafka streams before ingesting them to ClickHouse https://github.com/glassflow/clickhouse-etl Why we built this: Dedup with batch data is straightforward. You load the data into a temporary table. Then, find only the latest versions of the record through hashes or keys and keep them. After that, move the clean data into your main table. But have you tried this with streaming data? Users of our prev product were running real-time analytics…
2025 · github.com
- 9SA
Hey there! This is Dan and Ustin (@uzarubin), and we want to share something cool we've been working on for the past year - an open-source `tail -f` for your data, with a UI. We call it "Streamdal" which is a word salad for streaming systems (because we love them) and DAL or data access layer (because we’re nerds). Here's the repo: https://github.com/streamdal/streamdal Here's the site: https://streamdal.com And here's a live demo: https://demo.streamdal.com (github repo has an explanation of the demo) — — — THE PROBLEM We built this because the…
2023 · github.com
- 10BA
2019 · bamboolib.com
- 11SS
May 2026 · github.com
- 12H1
hi hn, hydra ceo here hydra is an open source, column-oriented postgres. you can set up remarkably fast aggregates on your project in minutes to query billions of rows instantly. postgres is great, but aggregates can take minutes to hours to return results on large data sets. long-running analytical queries hog database resources and degrade performance. use hydra to run much faster analytics on postgres without making code changes. data is automatically loaded into columnar format and compressed. connect to hydra with your preferred postgres client (psql, dbeaver, etc). following 4 months…
2023 · hydra-so.notion.site
- 13DE
tl;dr we built an embeddable stream processing engine in Rust using apache DataFusion, check us out at https://github.com/probably-nothing-labs/denormalized Hey HN, We’d like to showcase a very early version of our embeddable stream processing engine called Denormalized. The rise of DuckDB has abundantly made it clear that even for many workloads of Terabyte scale, a single node system outshines the distributed query engines of previous generation such as Spark, Snowflake etc in terms of both performance and cost. Now a lot of workloads DuckDB is used for were normally…
2024 · github.com
- 14RT
We have been involved in a number of real time streaming projects using tools such as Flink, Spark Streams and Kafka Streams backed by "real time" databases such as Druid. We always found these projects quite complex to develop and run, with stream processing in particular being a bit of a dark art. A stream to stream join in Flink can get quite mind bending for instance. In 2020 we had the idea of building a low code SaaS product for real time streaming analytics. The first attempt failed due to being a little over-engineered and with too many changes of direction, but over the last few…
2021
- 15SP
Hi HN, Over the past 6 months I've been working on a technical book focused on helping aspiring data scientists to get hands-on experience with cloud computing environments using the Python ecosystem. The book is targeted at readers already familiar with libraries such as Pandas and scikit-learn that are looking to build out a portfolio of applied projects. To author the book, I used the Leanpub platform to provide drafts of the text as I completed each chapter. To typeset the book, I used the R bookdown package by Yihui Xie to translate my markdown into a PDF format. I also used Google docs…
2020
- 16PIPandas in JS▲14
2017 · stratodem.github.io
- 17SR
2024 · github.com
- 18PS
tl;dr we’re releasing python bindings for writing stream processing jobs using the Denormalized Rust library. Hey HN, A few months ago we released Denormalized, our embeddable stream processing engine written in rust [0]. Today we’re excited to showcase python bindings for the library. Python is still very much the language of data and we wanted to make it easy for more engineers to write fast streaming jobs without having to learn Rust. Currently, you can use the library to write streaming aggregations, stream joins, and user defined aggregate functions (thought the API for this is quite…
2024 · github.com
- 19XR
Hi HN, I’ve been exploring whether pandas can be used as a computation description, rather than a runtime. The idea is to write data logic in pandas / NumPy, then freeze that logic into a static compute graph and execute it in pure C++, without embedding Python. This is not about reimplementing pandas or speeding up Python. It’s about situations where pandas-style logic is useful, but Python itself becomes a liability (latency, embedding, deployment). The project is still small and experimental, but it already works for a restricted subset of pandas-like operations and runs…
Mar 2026
- 20
- 21
- 22AA
2025 · github.com
- 23PB
Hi HN data folks, I am excited to share Pathway, a Python data processing framework we built for ETL and RAG pipelines. https://github.com/pathwaycom/pathway We started Pathway to solve event processing for IoT and geospatial indexing. Think freight train operations in unmapped depots bringing key merchandise from China to Europe. This was not something we could use Flink or Elastic for. Then we added more connectors for streaming ETL (Kafka, Postgres CDC…), data indexing (yay vectors!), and LLM wrappers for RAG. Today Pathway provides a data indexing layer for live data…
2024 · github.com
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →