Alternatives

Products that do what Setup a gpu optimized spark/hadoop cluster with docker compose does

Hey folks, I figured this might be useful for a few people who want to play with hadoop and spark locally. Feedback hugely appreciated. I built this from the pain of wanting to setup a cluster for demos. I have gpu bits in there due to the use case but figured maybe some could draw inspiration from this. Happy to answer questions if there's any interest. Link here: https://github.com/deeplearning4j/docker

1. 1RP
   Raspberry Pi3 cluster with Docker/Kubernetes▲86

   2016 · sweclockers.com

2. 2DG
   Dockerized GPU Deep Learning Solution (Code and Blog and TensorFlow Demo)▲80

   2016 · github.com

3. 3MD
   my database engine for GPU▲364

   2012 · sourceforge.net

4. 4
   Spark▲85

   Build internet-connected hardware.

   2014

5. 5
   Spark 2.0▲91

   A streaming engine for huge 3D Gaussian worlds

   Apr 2026

6. 6MI
   Marina.io, an open source Docker Hub with ARM support▲59

   2017 · marina.io

7. 7
   Dokku▲90

   Docker powered mini-Heroku in around 100 lines of Bash

   2015

8. 8PD
   PNGR – Docker-compose for a Postgres-Nginx-Golang-React stack▲138

   2019 · github.com

9. 9PA
   Pfs – A Git-like distributed file system for the Docker ecosystem▲74

   2014 · pachyderm-io.github.io

10. 10KD
    Kontena – Docker Platform Written in Ruby▲63

    2015 · kontena.io

11. 11AH
    A Hands-On Guide on PySpark Coding and Best Practices▲52

    2019 · github.com

12. 12RS
    Run `SSH bitbop.io`, get a personal GPU dev machine▲7

    Hi HN! I've been hacking on this side project for the last month or two with the goal of making it dead simple to use cloud GPUs. I ran into this problem personally during the phd, and built my own tooling around it. I always thought it'd be fun to try to turn that tooling into a more general product... and bitbop.io is the result! All you have to do is run `ssh bitbop.io`, and you get your own personal dev GPU workstation in the cloud. Looking forward to hearing your thoughts!

    2024 · twitter.com

13. 13GI
    Get into physical computing with Docker and Raspberry Pi (workshop)▲49

    2017 · github.com

14. 14SI
    Stampede.io – Hybrid IaaS/Docker Orchestation Running on CoreOS▲40

    2014 · ibuildthecloud.com

15. 15FN
    Fast native RPM building from any distro, for any distro, via Docker▲32

    2015 · github.com

16. 16SA
    Sparky as a simple orchestrator for self hosted clusters▲9

    Sep 2025 · github.com

17. 17DD
    Deep Dive into Spark and Join Optimizations▲5

    2020 · github.com

18. 18RA
    Run a kafka cluster using docker▲9

    2016 · github.com

19. 19DI
    Deepcluster.io, A cheap and easy to use GPU cluster for deep learning▲5

    Hi HN community, Shen and I created a service for anyone to easily train deep learning model on GPU power harnessed from the crowd. We have completed the first version DeepCluster.io (http://deepcluster.io) and welcome ML researchers to try it out for free! We are enthusiastic of deep learning, but often found training models with GPU instances on AWS very expensive. Meanwhile, some of our friends have idle GPUs that are used to mine cryptos. So we decided to borrow their GPUs for training deep learning model ourselves, and believe this could be a service that benefits other ML…

    2019

20. 20AQ
    A Quick Demo of Apache Beam with Docker▲5

    2016 · medium.com

21. 21MT
    Minimal Tensorflow and Syntaxnet Docker image (17x smaller than official)▲8

    2018 · github.com

22. 22RF
    Redshift - Flux Alternative, Easy to Install on Ubuntu▲13

    2013 · jonls.dk

23. 23LV
    Liqo v0.3 – Build a K8s super-cluster▲14

    2021 · github.com

24. 24MS
    Minimalist self-hosted CI server written on Raku▲6

    Sparky is a flexible and minimalist continuous integration server and distribute tasks runner written in Raku. https://github.com/melezhik/sparky

    2023

Ranked by how close each launch is in meaning, then by votes. Refine with a description →