Disk.frame, a larger-than-RAM data manipulation package
A disk-based data manipulation tool for working with large-than-RAM datasets. Aims to lower the barrier-to-entry for manipulating large datasets by adhering closely to popular and familiar data manipulation paradigms like dplyr verbs and data.table syntax.
Disk.frame, a larger-than-RAM data manipulation package launched on October 10, 2019 with 6 votes, #1,208 of 3,363 launches that month and more than 50% of that year's launches. The site was still online in September 2026, 7 years on.
What it does
{disk.frame} has been soft-deprecated in favor of {arrow} . With the {arrow} 6.0.0 release, it’s now capable of doing larger-than-RAM data analysis quite well see release note . Hence, there is no strong reason to prefer {disk.frame} unless you have very specific feature needs. For the above reason, I’ve decided to soft-deprecate {disk.frame} which means I will no longer actively develop new features for it but it will remain on CRAN in maintenance mode. To help with the transition I’ve created a function, disk.frame::disk.frame_to_parquet(df, outdir) to help you convert existing {disk.frame}s to the parquet format so you can use {arrow} with it. I am working on an reincarnation of…from diskframe.com
Does a similar job
all alternatives →
Vortex – a high-performance columnar file format2024 · github.com · ▲249An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation. - vortex-data/vortex
Redframes, a Python data manipulation library like dplyr2022 · github.com · ▲8General Purpose Data Manipulation Library. Contribute to maxhumber/redframes development by creating an account on GitHub.
- DDData Diff – compare tables of any size across databases2022 · ▲127
Gleb, Alex, Erez and Simon here – we are building an open-source tool for comparing data within and across databases at any scale. The repo is at https://github.com/datafold/data-diff, and our home page is https://datafold.com/. As a company, Datafold builds tools for data engineers to automate the most tedious and error-prone tasks falling through the cracks of the modern data stack, such as data testing and lineage. We launched two years ago with a tool for regression-testing changes to ETL code…
A Python container for dataclasses with multi-indexing and vector opps2020 · github.com · ▲122A container for dataclasses with multi-indexing and bulk operations. - joshlk/dataclassframe
More life & fun this month
the category →
A free city-building game in your browser. You are the mayor. You know what the city needs. Now get it past the zoning board, the neighbors, the appeals court and the next election.
Life & fun · 1d ago · permit.petercai.com
- IV
Life & fun · 13d ago · tanzaniaeducationproject.org
Turn a phone, tablet or TV into a sign. Type a message and it fills the screen: welcome signs, timers, Wi-Fi passwords and more. No app, no account, free.
Life & fun · 14d ago · bigwords.page
Play a free virtual guitar online with your keyboard or by touch. Slow songs down and see every chord, string and strum. Acoustic, classic, electric and bass.
Life & fun · 8d ago · guitoro.com
Carrier settings from iPhone, Pixel and Galaxy firmware, decoded and compared: APNs, VoLTE, 5G and Wi-Fi Calling per carrier, and what each build changed.
Life & fun · 12d ago · carrierexplode.com
Life & fun · 22d ago · space.bl2.net
Source: Show HN post ↗
Launched alongside, October 2019
the whole month →
Bel▲1,288Life & fun · 2019 · paulgraham.com
Create free personalized illustrations from 3000+ elements
Growth · 2019 · icons8.com
The next generation of temperature controlled mugs
Life & fun · 2019 · ember.com
