AI · alternatives · 2026
24 alternatives to Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used)
Hi HN data folks, I am excited to share Pathway, a Python data processing framework we built for ETL and RAG pipelines. https://github.com/pathwaycom/pathway We started Pathway to solve event processing for IoT and geospatial indexing. Think freight train operations in unmapped depots bringing key merchandise from China to Europe. This was not something we could use Flink or Elastic for. Then we added more connectors for streaming ETL (Kafka, Postgres CDC…), data indexing (yay vectors!), and LLM wrappers for RAG. Today Pathway provides a data indexing layer for live data…
Pathway is a Python data processing framework for building ETL and RAG pipelines. It handles streaming data processing from sources like Kafka and Postgres, performs stateless and stateful transformations, and enables… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used) launched in 2024; newer entries below may have overtaken it.
- 1
Hello everyone! I am Jan, CTO and one of the creators of Pathway, the real-time data processing framework. I’m excited to share Pathway’s ready-to-use AI Pipelines, configurable with just YAML! These frameworks offer out-of-the-box solutions for AI search, RAG, and more—optimized for real-time indexing and in-memory processing. What makes it simple? YAML templates! The pipeline templates are fully customizable using YAMLs to fit your needs, from changing the data sources to the choice of the LLM model, all without touching Pathway’s Python code. Thanks to the Pathway data processing engine,…
2024 · pathway.com · its alternatives →
- 2
Every data pipeline job I had to tackle required quite a few components to set up: - One tool to ingest data - Another one to transform it - If you wanted to run Python, set up an orchestrator - If you need to check the data, a data quality tool Let alone this being hard to set up and taking time, it is also pretty high-maintenance. I had to do a lot of infra work, and while this being billable hours for me I didn’t enjoy the work at all. For some parts of it, there were nice solutions like dbt, but in the end for an end-to-end workflow, it didn’t work. That’s why I decided to build an…
2024 · github.com · its alternatives →
- 3
- 4
Hey there HN! We’re Antonio, Luca, and Yuhang, and we’re excited to introduce Fast GraphRAG, an open-source RAG approach that leverages knowledge graphs and the 25 years old PageRank for better information retrieval and reasoning. Building a good RAG pipeline these days takes a lot of manual optimizations. Most engineers intuitively start from naive RAG: throw everything in a vector database and hope that semantic search is powerful enough. This can work for use cases where accuracy isn’t too important and hallucinations are tolerable, but it doesn’t work for more difficult queries that…
2024 · github.com · its alternatives →
- 5
Hi HN community, I’m Nick, co-founder/CEO of Transform.co. I’m thrilled to share MetricFlow, an open-source metric creation framework: https://github.com/transform-data/metricflow MetricFlow strives to make what has historically been an extremely repetitive process, writing SQL queries on core normalized data models, much more DRY. MetricFlow consolidates the definitions for joins, aggregations, filters, etc., and programmatically generates SQL to construct data marts. You can think of it like LookML, but more powerful and ergonomic (and open source!). The project…
2022 · github.com · its alternatives →
- 6
Compass▲295Ask Slack about your data and get answers instantly in chat
Dec 2025 · compass.dagster.io · its alternatives →
- 7
I built this tool because I wanted a way to just take a bunch of URLs or domains, and query their content in RAG applications. It takes away the pain of crawling, extracting content, chunking, vectorizing, and updating periodically. I'm curious to see if it can be useful to others. I meant to launch this six months ago but life got in the way...
2024 · embedding.io · its alternatives →
- 8
- 9
Hey HN, we want to share HelixDB (https://github.com/HelixDB/helix-db/), a project a college friend and I are working on. It’s a new database that natively intertwines graph and vector types, without sacrificing performance. It’s written in Rust and our initial focus is on supporting RAG. Here’s a video runthrough: https://screen.studio/share/szgQu3yq. Why a hybrid? Vector databases are useful for similarity queries, while graph databases are useful for relationship queries. Each stores data in a way that’s best for its main type of query (e.g.…
2025 · github.com · its alternatives →
- 10
Hello HN, I'm Owen from SciPhi (https://www.sciphi.ai/), a startup working on simplifying˛Retrieval-Augmented Generation (RAG). Today we’re excited to share R2R (https://github.com/SciPhi-AI/R2R), an open-source framework that makes it simpler to develop and deploy production-grade RAG systems. Just a quick reminder: RAG helps Large Language Models (LLMs) use current information and specific knowledge. For example, it allows a programming assistant to use your latest documents to answer questions. The idea is to gather all the relevant information…
2024 · github.com · its alternatives →
- 11SA
Hi HN, I’m Tullie, founder of Shaped. Previously, I was a researcher at Meta AI, worked on ranking for Instagram Reels, and was a contributor to PyTorch Lightning. We built ShapedQL because we noticed that while retrieval (finding 1,000 items) has been commoditized by vector DBs, ranking (finding the best 10 items) is still an infrastructure problem. To build a decent for you feed or a RAG system with long-term memory, you usually have to put together a vector DB (Pinecone/Milvus), a feature store (Redis), an inference service, and thousands of lines of Python to handle business logic…
Jan 2026 · playground.shaped.ai · its alternatives →
- 12
Hi HN, We're Luke and Phillip, and we're building Spice.ai OSS - a lightweight, portable runtime, built in Rust and powered by Apache DataFusion to locally materialize, accelerate, and query data tables sourced from any database, data warehouse or data lake. Phillip and I first introduced Spice on Show HN in September 2021. Since then, we’ve been schooled and humbled in every way building 100TB+ data and ML systems for the https://spice.ai cloud platform. Along with our customers, we struggled with getting fast, low-latency, high-concurrency SQL query within a budget, accessing and…
2024 · github.com · its alternatives →
- 13
Hey HN! I want to share a new open source project I've been working on called Trench (https://trench.dev). It's open source analytics infrastructure for tracking events, page views, and identifying users, and it's built on top of ClickHouse and Kafka. https://github.com/frigadehq/trench I built Trench because the Postgres table we used for tracking events at our startup (http://frigade.com/) was getting expensive and becoming a performance bottleneck as we scaled to millions of end users. Many companies run into the same problem as us (e.g.…
2024 · github.com · its alternatives →
- 14
EnterpriseDB (“EDB”), the leader in accelerating Postgres® in the enterprise, today announced its acquisition of Splitgraph, an early-stage disruptor that provides a Postgres compatible serverless SQL API for building data driven applications from hundreds of datasources. This move is seen as a mean
2020 · splitgraph.com · its alternatives →
- 15
Hi HN! We are Ashish and Armend, founders of GlassFlow. We just launched our open-source streaming ETL that deduplicates and joins Kafka streams before ingesting them to ClickHouse https://github.com/glassflow/clickhouse-etl Why we built this: Dedup with batch data is straightforward. You load the data into a temporary table. Then, find only the latest versions of the record through hashes or keys and keep them. After that, move the clean data into your main table. But have you tried this with streaming data? Users of our prev product were running real-time analytics…
2025 · github.com · its alternatives →
- 16
A container for dataclasses with multi-indexing and bulk operations. - joshlk/dataclassframe
2020 · github.com · its alternatives →
- 17
Hey hacker news, we launched a few weeks ago as a GPT-powered chatbot for developer docs, and quickly realized that the value of what we’re doing isn’t the chatbot itself. Rather, it’s the time we save developers by automating the extraction of data from their SaaS tools (Github, Zendesk, Salesforce, etc) and helping transform it to contextually relevant chunks that fit into GPT’s context window. A lot of companies are building prototypes with GPT right now and they’re all using some combination of Langchain/Llama Index + Weaviate/Pinecone + GPT3.5/GPT4 as their stack for…
2023 · github.com · its alternatives →
- 18
Hi HN, I’ve been working on CocoIndex, an open-source Data ETL framework to transform data for AI, optimized for data freshness. You can start a CocoIndex project with `pip install cocoindex` and declare a data flow that can build ETL like LEGO - build a RAG pipeline for vector embeddings, knowledge graphs, or extract, transform data with LLMs. It is a data processing framework beyond text. When you run the data flow either with live mode or batch mode, it will process the data incrementally with minimal recomputation and make it super fast to update the target stores on source changes. Get…
2025 · github.com · its alternatives →
- 19
Hi HN, I am Jan, CTO and co-founder of Pathway.com. We’ve built a LLM microservice that answers questions about a corpus of documents, while automatically reacting to additions of new docs. The single, self-contained service fully replaces a complex multi-system pipeline that scans in real-time for new documents, indexes them into a specialized database and queries it to generate answers. Everyone can have their own real-time vector now. Github: https://github.com/pathwaycom/llm-app Demo video: https://youtu.be/kcrJSk00duw I am eager to hear your thoughts…
2023 · github.com · its alternatives →
- 20
Hi HN, I’m Ignacio, founder at Basekick Labs. Over the past months I’ve been building Arc, a time-series data platform designed to combine very fast ingestion with strong analytical queries. What Arc does? Ingest via a binary MessagePack API (fast path), Compatible with Line Protocol for existing tools (Like InfluxDB, I'm ex Influxer), Store data as Parquet with hourly partitions, Query via DuckDB engine using SQL Why I built it: Many systems force you to trade retention, throughput, or complexity. I wanted something where ingestion performance doesn’t kill your analytics. Performance &…
2025 · github.com · its alternatives →
- 21TF
Hi All! We've spent a few months on getting an MVP together, and would love to get some feedback on whether this tool meets you needs. Here is a link to a demo video: https://www.youtube.com/watch?v=FBLi3vdKB-4&feature=emb_rel_pause Here's a link to our website: https://www.structure.rest And here's a blog article, I published today in the space: https://www.structure.rest/blog/using-a-data-analytics-stack-to-gain-business-insights
2020 · its alternatives →
- 22AI
Hi HN, I’m Sean, the founder of Ascend.io (https://www.ascend.io). I’m really excited to post here and announce the launch of Ascend.io, a radical new way of designing, scaling, and automating data pipelines. Ascend is the result of nearly 4 years of development effort for a team that is now 30-strong, and I would love for you to give it a test drive and let me what you think. I’ve felt this pain since I wrote my first MapReduce in 2004 (using Sawzall @ Google), and in the 15 years since, things have not improved at the pace of other parts of the technology ecosystem. When I went…
2019 · its alternatives →
- 23
- Discovering the most effective RAG pipeline for your specific data and use case can be daunting. It requires experimenting with various RAG modules and configurations, which are both time-consuming and complex. - AutoRAG addresses this challenge by automatically evaluating different combinations of RAG modules and their parameters. You don't need to write implementation code yourself; everything is set up through a single YAML file. - Our aim is to save you the hassle of continuously adapting to new RAG modules and configurations. Instead, you can focus on developing robust data for your…
2024 · github.com · its alternatives →
- 24
RAGLite is a Python package for building Retrieval-Augmented Generation (RAG) applications. RAG applications can be magical when they work well, but anyone who has built one knows how much the output quality depends on the quality of retrieval and augmentation. With RAGLite, we set out to unhobble RAG by mapping out all of its subproblems and implementing the best solutions to those subproblems. For example, RAGLite solves the chunking problem by partitioning documents in provably optimal level 4 semantic chunks. Another unique contribution is its optimal closed-form linear query adapter…
2024 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →