Alternatives

Products that do what A distributed and persistent web crawler written in Go does

1. 1CO
   Crawlab: Open-Source Web Crawler Admin Platform That Runs Any Language▲116

   2019 · github.com

2. 2AM
   A Multi-billion page web crawler in the cloud▲14

   2017 · mixnode.com

3. 3WA
   Webhook – A lightweight configurable tool written in Go▲144

   2017 · github.com

4. 4AG
   A Go web server with logging, tracing, and demo apps in a single file▲116

   2019 · github.com

5. 5GI
   GoCode.io – A Golang Game in the Browser▲129

   2019 · gocode.io

6. 6AS
   A simple Go web server with logging, tracing, health check▲318

   2018 · gist.github.com

7. 7AM
   A modular, durable web-crawler for Clojure▲29

   2016 · github.com

8. 8GG
   Goyave – Golang Web Framework▲75

   2020 · github.com

9. 9AH
   Apifier – hosted web crawler for developers▲101

   2015 · apifier.com

10. 10AE
    An experimental, people-powered search engine▲215

    2021 · ninfex.com

11. 11AA
    ABCWeb – A Go Web Framework▲50

    2017 · github.com

12. 12AE
    An experimental distributed SQL database from scratch in Go▲104

    2019 · github.com

13. 13TD
    The dht(distributed hash table) network crawler▲5

    2015 · github.com

14. 14SM
    (Sukhoi) Minimalist and Powerful Web Crawler in Python▲6

    2017 · github.com

15. 15SA
    Sukhoi – A flexible and extensible Webcrawler in Python▲131

    2017 · github.com

16. 16AS
    Apify SDK – A scalable web crawling and scraping library for JavaScript▲78

    2018 · github.com

17. 17BA
    Badger – an embeddable, persistent key-value store written in Go▲88

    2017 · github.com

18. 18IH
    I have written a cloud native dark web crawler in Go▲7

    2021 · github.com

19. 19AO
    An open source web crawler for the Mwmbl non-profit search engine▲6

    2022 · github.com

20. 20CF
    Checkbot for Chrome – web crawler that tests for web best practices▲83

    2018 · checkbot.io

21. 21VW
    Voyager – write a web crawler/scraper as a state machine in Rust▲110

    2020 · github.com

22. 22GA
    GoWork – A Go Distributed Work Library▲68

    2015 · github.com

23. 23AB
    Alloy – Boilerplate for building web applications with Go▲7

    2017 · growthmetrics.io

24. 24HC
    Highly cost-efficient search engine in Rust▲61

    2021 · github.com

Ranked by how close each launch is in meaning, then by votes. Refine with a description →