Setup a gpu optimized spark/hadoop cluster with docker compose
Hey folks, I figured this might be useful for a few people who want to play with hadoop and spark locally. Feedback hugely appreciated. I built this from the pain of wanting to setup a cluster for demos. I have gpu bits in there due to the use case but figured maybe some could draw inspiration from this. Happy to answer questions if there's any interest. Link here: https://github.com/deeplearning4j/docker
Setup a gpu optimized spark/hadoop cluster with docker compose launched on October 27, 2016 with 7 votes and more than 50% of that year's launches.
Does a similar job
all alternatives →- TCTensorDock Core GPU Cloud – GPU servers from $0.29/hr2022 · tensordock.com · ▲147
Hello HN! I’m Jonathan from TensorDock. After 7 months in beta, we’re finally launching Core Cloud, our platform to deploy GPU virtual machines in as little as 45 seconds! https://www.tensordock.com/product-core Why? Training machine learning workloads at large clouds can be extremely expensive. This left us wondering, “how did cloud ever become more expensive than on-prem?” I’ve seen too many ML startups buy their own hardware. Cheaper dedicated servers with NVIDIA GPUs are not too hard to find, but they lack the functionality and scalability of the big clouds. We thought to…
I've built a nice home server OSApr 2026 · lightwhale.asklandd.dk · ▲194ohai! I've released Lightwhale 3, which is possibly the easiest way to self-host Docker containers. It's a free, immutable Linux system purpose-built to live-boot straight into a working Docker Engine, thereby shortcutting the need for installation, configuration, and maintenance. Its simple design makes it easy to learn, and its low memory footprint should make it especially attractive during these times of RAMageddon. If this has piqued your interest, do check it out, along with its easy-to-follow Getting Started guide. In any event, have a nice day! =)
Dockerized GPU Deep Learning Solution (Code and Blog and TensorFlow Demo)2016 · github.com · ▲80An on-premises, bare-metal solution for deploying GPU-powered applications in containers - emergingstack/es-dev-stack
Spark, An advanced 3D Gaussian Splatting renderer for Three.js2025 · sparkjs.dev · ▲377I'm the co-creator and maintainer of https://aframe.io/ and long time Web 3D graphics dev. Super excited about new techniques to author / render / represent 3D. Spark is a an open source library to easily integrate Gaussian splats in your THREE.js scene I worked with some friends and I hope you find useful. Looking forward to hearing what features / rendering techniques you would love to see next.
Nextjournal – seamless data science for teams2019 · nextjournal.com · ▲170Runs anything you can put into a Docker container. Improve your workflow with polyglot notebooks, automatic versioning and real-time collaboration. Save time and money with on-demand provisioning, including GPU support. Use it for ML, data science, runnable tutorials, for your classroom or scientifi
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 28d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 26d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 23d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 21d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 28d ago · corespeed.io
Source: Show HN post ↗
Launched alongside, October 2016
the whole month →
Without your consent most major web platforms leak whether you are logged in. This allows any website to detect on which platforms you
Growth · 2016 · robinlinus.github.io
Surface Studio▲832A brilliant screen for your ideas, from Microsoft
Life & fun · 2016 · microsoft.com
