Web Scraping for Beginners

What it does

A course to teach you a proven system of web scraping hacks you can use to increase your chances of scraping public data from pretty much any website, even the ones with bot detection. Techniques I used to scrape over 70k job postings for TechStackLeads.com

The pain of data projectsThe worst feeling is when there is no API to get the data for your new project. Sometimes you get a glimmer of hope. You find a company with the data you need but then you find out the data costs a lot of money and can get outdated very quickly. I know these feelings because that’s what I faced.I once started a project to build a search engine to find more software enginee

• 
  Scraping for Growth2014 · ellsworthy.com · ▲108

  Web scraping course for sales & growth hackers.

• 
  ScrapeAll2022 · ▲87

  Automated web data scraper

• 
  ScrapeIN'2022 · soax.com · ▲128

  Effortless data extraction from any website

• 
  ScrapeNinja2023 · scrapeninja.net · ▲249

  Swiss army knife for high-performance web scraping

• ST
  Scrape the web by injecting JavaScript into web pages2014 · espion.io · ▲109
• 
  I made a Chrome extension that can scrape any website with one click2024 · easyscraper.com · ▲31

  Hi HN! Ah yes, another web scraper. There are so many out there, so why make another one? Because I wanted one that's stupid simple and stupid fast. Unlike most web scrapers, Easy Scraper doesn’t make you build a new scraper for each site or individually select fields. You just open it and bam, your data's ready to go. My favorite use case for instantly extracting data anywhere is being able to easily talk to ChatGPT about custom datasets. For example, here's a 1-minute demo of me scraping my Twitter followers to find the ones that live in San Francisco:…

• 
  /dev/null▲483

  a social network for devs

  Dev tools · 3d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 25d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 23d ago · openbot.run

• 
  Talanoa▲345

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 20d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 28d ago · corespeed.io

• 
  AI Content Genie▲1,071

  AI autopilot for content creation & marketing

  AI · 2023 · typeface.ai

• 
  Nudge 2.0▲1,051

  In-app experiences to activate, retain, & understand users

  Growth · 2023 · nudgenow.com

• 
  Cal.ai Email Assistant▲918

  World's first open source AI scheduling assistant

  AI · 2023 · cal.com

• 
  Design Studio by Tiledesk▲867

  Unlock AI magic for elevated customer engagement, fast

  AI · 2023 · tiledesk.com

• 
  TalkNotes▲854

  Turn your thoughts into actionable notes, fast

  AI · 2023 · talknotes.io

• 
  OpenAPI DevTools – Chrome extension that generates an API spec▲811

  Effortlessly discover API behaviour with a Chrome extension that automatically generates OpenAPI specifications in real time for any app or website.

  Dev tools · 2023 · github.com