How to analyse 100 GB of data on your laptop with Python

In plain words

This is a guide for developers on analyzing large datasets using Python on standard laptops. Published in 2019, it addresses the practical challenge of working with 100 GB files without requiring specialized infrastructure or cloud services. The resource targets Python developers who need efficient data processing techniques for their local machines. It focuses on methods and tools that make large-scale analysis feasible on resource-constrained hardware.

written from the facts on this page · September 2026

• 
  Python for Data Analysis2016 · amazon.com · ▲72

  Pandas is the most common python library for data analytics

• 
  All-pair similarity search on millions of sets in Python and on laptop2018 · github.com · ▲123

  All-pair set similarity search on millions of sets in Python and on a laptop - GitHub - ekzhu/SetSimilaritySearch: All-pair set similarity search on millions of sets in Python and on a laptop

• 1R
  1M rows/s from Postgres to Python2016 · magic.io · ▲296
• 
  Taipy 3.02024 · github.com · ▲299

  Build powerful data and AI apps in pure Python

• BA
  Bamboolib – A GUI for Pandas (Python Data Science)2019 · bamboolib.com · ▲119
• PN
  Pandas, Numpy, and SciPy users can now access our over 4M datasets2013 · quandl.com · ▲93

  The source for financial, economic, and alternative datasets, serving investment professionals.

• 
  Spira Maxima▲578

  Video model that turns script into viral social media videos

  AI · 18d ago · spira.ai

• 
  Rill Browser▲540

  The browser where Claude Code and Codex work beside you

  AI · 18d ago · rill.love

• 
  Germany's new sovereign AI model Kolibri▲424

  Kolibri is Aleph Alpha

  AI · 20d ago · tej.as

• 
  Giving Opus 5.5 a simulated paint canvas▲392

  Paintings by AI models: programs run through a simulation of oil paint.

  AI · 21d ago · stillwet.art

• 
  Beam.win▲378

  Run AI agents on one Mac and use its windows on another

  AI · 5d ago · beam.win

• 
  FastRouter.ai▲364

  Route requests to the right LLM for cost, latency & quality

  AI · 23d ago · fastrouter.ai

• 
  Exploding Topics▲949

  Find proven topics before they take off.

  Growth · 2019 · explodingtopics.com

• AA
  Angeldust – a fast and efficient video game▲716

  After years of solo development I'm presenting: Angeldust, an omni-compatible action RPG and building game! Website: http://angeldu.st My single server hosts 250+K active players in one giant, dynamic fantasy world. Angeldust's game server is pervasively multithreaded, implementing John Carmack's dream of processing an immutable world state each tick. Both world handling and network traffic routing uses any number of threads. For networking, a typical game session only uses ~3Kbps of bandwidth, enabling decent play even over 2G/EDGE networks. The client is programmed in C++…

  Growth · 2019

• AS
  A searchable list of self-hosted software with screenshots▲656

  Dev tools · 2019 · selfhostedsource.tech

• 
  A tiny isometric city builder in JavaScript▲593

  A isometric city builder in JavaScript. Contribute to victorqribeiro/isocity development by creating an account on GitHub.

  Dev tools · 2019 · github.com

• 
  AI Dungeon 2 – AI-generated text adventure built with 1.5B param GPT-2▲584

  Play and create AI-generated adventures with infinite possibilities.

  AI · 2019 · aidungeon.io

• 
  Mailmeteor▲567

  Send hundreds of personalized emails with Gmail.

  Growth · 2019 · mailmeteor.com