Alternatives

Products that do what SQLFrame – I ran PySpark without Spark on a SQL database does

Recently I open-sourced SQLFrame, a DataFrame library that implements the PySpark DataFrame API but removes Spark as a dependency. It does this by generating the corresponding SQL for the DataFrame operations using SQLGlot. Since the output is SQL this also means that the PySpark DataFrame API can now be used directly against other databases without the Spark middleman. I built this because of two common problems I have faced in my career: 1. I prefer to write complex pipelines in PySpark but they can be hard to read for SQL-proficient co-workers. Therefore I find myself in a tradeoff…

1. Not indexed yet — check back in a few minutes.

Ranked by how close each launch is in meaning, then by votes. Refine with a description →