Simple Blog For Crawler And BigData

What it does

Demanejar is a blog about bigdata and things surrounding bigdata. Big data is one of the important branches of Computer Science and the processing of this data to produce new data is increasing exponentially.

(c) Copyright Demanejar. A small website about Bigdata writen by team Demanejar.

Some initial research on k8s. Relationship between containerd and docker Low level runtime currently mainly uses runc High level runtime currently mainly uses containerd graph TD; ... Scrapy provides the scrapy-selenium library that allows using Selenium to fetch website data before returning it to the spider for processing. However, in some cases, for example, I don’t want to u... Javascript Variables May Contain Necessary Data When Crawling Javascript-Rendered Websites When crawling websites rendered with Javascript like in previous articles, we have to use a front-end com... Spline is an OpenSource tool that enables automatic tracking of Data Lineage and Data Pipeline Structure. Its most…from demanejar.github.io

• 
  Crawly2016 · crawly.diffbot.com · ▲606

  Never write another web scraper

• 
  Crawlee2022 · crawlee.dev · ▲229

  Build reliable web scrapers and robots, fast!

• 
  A Python Spider System with Web UI2014 · github.com · ▲159

  A Powerful Spider(Web Crawler) System in Python. Contribute to binux/pyspider development by creating an account on GitHub.

• 
  LogPacker2017 · khamama.com · ▲101

  Collect, aggregate and analyze log data

• 
  DataDiver2024 · ▲100

  Your own data scraping agent

• 
  Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language2019 · github.com · ▲116

  Distributed web crawler admin platform for spiders management regardless of languages and frameworks. - tikazyq/crawlab

• 
  /dev/null▲482

  a social network for devs

  Dev tools · 3d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 27d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 26d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 23d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 20d ago · talanoa.email

• 
  CoreSpeed▲332

  One MCP for everything your agents need: apps, memory, tools

  Dev tools · 28d ago · corespeed.io

• 
  Amie▲1,334

  Turn meeting notes into automated workflows. Handle meetings, summaries, todos and emails (soon) with your AI personal assistant.

  Work · 2024 · amie.so

• 
  Findr▲1,296

  Search all your apps, at once

  AI · 2024 · usefindr.com

• 
  Notion Calendar▲1,199

  Beautifully designed for your work and life

  Work · 2024 · notion.so

• 
  Huudle AI Project Assistant▲1,070

  Connect meetings, track to-do, get aligned

  AI · 2024 · huudle.io

• IM
  I made a website to find best bus seat to avoid the sun while traveling▲963

  Sit in Shade - Discover the optimal seat to stay in the shade during your journey. Sit In Shade provides real-time sun exposure data and route visualization.

  Life & fun · 2024 · sitinshade.com

• 
  Letterly▲937

  Turn your speech into well-written text

  AI · 2024 · letterly.app