nowfound

Dev tools · alternatives · 2026

SA

24 alternatives to Setup a gpu optimized spark/hadoop cluster with docker compose

Hey folks, I figured this might be useful for a few people who want to play with hadoop and spark locally. Feedback hugely appreciated. I built this from the pain of wanting to setup a cluster for demos. I have gpu bits in there due to the use case but figured maybe some could draw inspiration from this. Happy to answer questions if there's any interest. Link here: https://github.com/deeplearning4j/docker

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Setup a gpu optimized spark/hadoop cluster with docker compose launched in 2016; newer entries below may have overtaken it.

  1. 1TC

    Hello HN! I’m Jonathan from TensorDock. After 7 months in beta, we’re finally launching Core Cloud, our platform to deploy GPU virtual machines in as little as 45 seconds! https://www.tensordock.com/product-core Why? Training machine learning workloads at large clouds can be extremely expensive. This left us wondering, “how did cloud ever become more expensive than on-prem?” I’ve seen too many ML startups buy their own hardware. Cheaper dedicated servers with NVIDIA GPUs are not too hard to find, but they lack the functionality and scalability of the big clouds. We thought to…

    2022 · tensordock.com · its alternatives →

  2. 2

    ohai! I've released Lightwhale 3, which is possibly the easiest way to self-host Docker containers. It's a free, immutable Linux system purpose-built to live-boot straight into a working Docker Engine, thereby shortcutting the need for installation, configuration, and maintenance. Its simple design makes it easy to learn, and its low memory footprint should make it especially attractive during these times of RAMageddon. If this has piqued your interest, do check it out, along with its easy-to-follow Getting Started guide. In any event, have a nice day! =)

    Apr 2026 · lightwhale.asklandd.dk · its alternatives →

  3. 3

    An on-premises, bare-metal solution for deploying GPU-powered applications in containers - emergingstack/es-dev-stack

    2016 · github.com · its alternatives →

  4. 4

    I'm the co-creator and maintainer of https://aframe.io/ and long time Web 3D graphics dev. Super excited about new techniques to author / render / represent 3D. Spark is a an open source library to easily integrate Gaussian splats in your THREE.js scene I worked with some friends and I hope you find useful. Looking forward to hearing what features / rendering techniques you would love to see next.

    2025 · sparkjs.dev · its alternatives →

  5. 5

    Runs anything you can put into a Docker container. Improve your workflow with polyglot notebooks, automatic versioning and real-time collaboration. Save time and money with on-demand provisioning, including GPU support. Use it for ML, data science, runnable tutorials, for your classroom or scientifi

    2019 · nextjournal.com · its alternatives →

  6. 6
    Spark▲85

    Build internet-connected hardware.

    2014 · particle.io · its alternatives →

  7. 7RP
  8. 8

    A low power 1U Raspberry Pi cluster server for inexpensive colocation. - pawl/raspberry-pi-1u-server

    2021 · github.com · its alternatives →

  9. 9
    Dokku▲90

    Docker powered mini-Heroku in around 100 lines of Bash

    2015 · github.com · its alternatives →

  10. 10

    Create and manage full-stack micro-apps with AI prompts

    2024 · githubnext.com · its alternatives →

  11. 11

    This project addresses the inefficiencies of traditional Dockerfile-based container builds where each customization layer creates storage bloat through duplicate dependencies from repeated apt-get install commands, network inefficiency from redundant package downloads across different images, and slow iteration cycles requiring full rebuilds of all previous steps. Our solution enables building minimal base images from scratch using debootstrap that precisely include only required components in the initial build, while allowing creation of specialized variants (Java, Kafka, etc.) from these…

    2025 · github.com · its alternatives →

  12. 12TG

    Hey folks! I’m Jonathan from TensorDock, and we’re building a cloud GPU marketplace. We want to make GPUs truly affordable and accessible. I once started a web hosting service on self-hosted servers in middle school. But building servers isn’t the same as selling cloud. There’s a lot of open source software to manage your homelab for side projects, but there isn’t anything to commercialize that. Large cloud providers charge obscene prices — so much so that they can often pay back their hardware in under 6 months with 24x7 utilization. We are building the software that allows anyone to become…

    2024 · dashboard.tensordock.com · its alternatives →

  13. 13

    Hi HN! I've been hacking on this side project for the last month or two with the goal of making it dead simple to use cloud GPUs. I ran into this problem personally during the phd, and built my own tooling around it. I always thought it'd be fun to try to turn that tooling into a more general product... and bitbop.io is the result! All you have to do is run `ssh bitbop.io`, and you get your own personal dev GPU workstation in the cloud. Looking forward to hearing your thoughts!

    2024 · twitter.com · its alternatives →

  14. 14

    Git-integrated GPU workloads, push to deploy & pull results

    2024 · aisilicon.com · its alternatives →

  15. 15

    Get into physical computing with Docker and Raspberry Pi - alexellis/docker-blinkt-workshop

    2017 · github.com · its alternatives →

  16. 16

    Introducing Sphere - A lightweight, sandboxed compute runtime written in Rust. - introducing-sphere.md

    2025 · gist.github.com · its alternatives →

  17. 17

    Sparky is a flexible and minimalist continuous integration server and distribute tasks runner written in Raku. - sparky/posts/SparkyAsSimpleOrchestratorforSelfHostedClusters.md at master · melezhik/sparky

    2025 · github.com · its alternatives →

  18. 18

    Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory. A single 256K token conversation on Gemma 4 31B produces approximately 43GB of KV state, more than half the memory of an 80GB H100. The problem becomes even harder across a cluster: a prefix cached on one GPU host is unavailable when the next request lands on a different GPU, forcing the new GPU to repeat work the fleet has already completed. Once the KV cache is offloaded,…

    Jul 2026 · github.com · its alternatives →

  19. 19DI

    Hi HN community, Shen and I created a service for anyone to easily train deep learning model on GPU power harnessed from the crowd. We have completed the first version DeepCluster.io (http://deepcluster.io) and welcome ML researchers to try it out for free! We are enthusiastic of deep learning, but often found training models with GPU instances on AWS very expensive. Meanwhile, some of our friends have idle GPUs that are used to mine cryptos. So we decided to borrow their GPUs for training deep learning model ourselves, and believe this could be a service that benefits other ML…

    2019 · its alternatives →

  20. 20AQ
  21. 21

    Cloud GPUs from $0.29/hour. Cost-effective. API. Scalable.

    2022 · its alternatives →

  22. 22

    Hi everybody, I created my personal weekend project Gnoseed to learn more about Kubernetes with spaced repetition. Very soon it evolved into a small library of various topics about modern tech stack - AI/LLMs, DevOps, Security, and anything I found interesting. Currently it has 47 decks and ~4000 questions. A bit more about the process: I create decks from the open documentations or free papers I found here or there. The decks are built with a custom made pipeline of skills to extract the data, select questions, creating distractors and verification. I'm curating the entire process. To…

    Jul 2026 · gnoseed.com · its alternatives →

  23. 23MS

    Sparky is a flexible and minimalist continuous integration server and distribute tasks runner written in Raku. https://github.com/melezhik/sparky

    2023 · its alternatives →

  24. 24

    One API for AI image and vision tasks. Runfra classifies each request and routes it to the right model and the cheapest capable backend — running on distributed GPUs with built-in quality filtering and automatic retry, so you keep only the best results. The routing engine behind Runfra Studio.

    Apr 2026 · runfra.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →