nowfound

Dev tools · June 17, 2021

H5records – simple large dataset for pytorch training

Large dataset storage format for Pytorch. Contribute to theblackcat102/H5Record development by creating an account on GitHub.

H5records – simple large dataset for pytorch training launched on June 17, 2021 with 5 votes, #1,710 of 3,650 launches that month and more than 50% of that year's launches. The site was still online in September 2026, 5 years on.

Visit github.com ↗Alternatives

Does a similar job

all alternatives →
  • WS
    We scaled Git to support 1 TB repos2022 · xethub.com · ▲279

    I’ve been in the MLOps space for ~10 years, and data is still the hardest unsolved open problem. Code is versioned using Git, data is stored somewhere else, and context often lives in a 3rd location like Slack or GDocs. This is why we built XetHub, a platform that enables teams to treat data like code, using Git. Unlike Git LFS, we don’t just store the files. We use content-defined chunking and Merkle Trees to dedupe against everything in history. This allows small changes in large files to be stored compactly. Read more here:…

  • Weights & Biases2019 · wandb.ai · ▲110

    Developer tools for deep learning & machine learning

  • DataTorch2021 · datatorch.io · ▲69

    Build and share datasets to experiment with ML

  • DC
    Datasets.co – Share and discover new machine learning datasets2016 · datasets.co · ▲81
  • Encrypted Deep Learning in PyTorch2018 · github.com · ▲37

    Perform data science on data that remains in someone else's server - OpenMined/PySyft

  • PK
    PyTorch K-Means GPU-friendly, single-file, hierarchical and resampling2025 · ▲10

    I built a small, self-contained K-Means implementation in pure PyTorch: https://gitlab.com/hassonofer/pt_kmeans I was working on dataset sampling and approximate nearest neighbor search, and tried several existing libraries for large-scale K-Means. I couldn't find something that was fast, simple, and would run comfortably on my own workstation without hitting memory limits. Maybe I missed an existing solution, but I ended up writing one that fit my needs. The core insight: Keep your data on CPU (where you have more RAM) and intelligently move only the necessary chunks to…

More dev tools this month

the category →
  • /dev/null▲494

    a social network for devs

    Dev tools · 4d ago · dev-null.sh

  • Run full iOS 16.5 firmware on Android Devices▲472

    Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.

    Dev tools · 23h ago · github.com

  • Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

    open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

    Dev tools · 29d ago · github.com

  • Reladraw – A diagram language where you decide where to place things▲411

    Contribute to reladraw/reladraw development by creating an account on GitHub.

    Dev tools · 27d ago · github.com

  • OpenBot▲362

    Grok Bot alternative: free, local, open-source, multiplayer

    Dev tools · 24d ago · openbot.run

  • Talanoa▲346

    Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

    Dev tools · 22d ago · talanoa.email

Source: Show HN post ↗

Launched alongside, June 2021

the whole month →