Crawler Detect bots/crawlers/spiders using the user agent string Detect bots/crawlers/spiders using the user agent string 03 October 2022
Crawler Teach daily is web crawl from dev.to, freecodecamp, medium, hashnode Teach daily is web crawl from dev.to, freecodecamp, medium, hashnode 22 July 2022
Crawler A sample & efficient web crawler for golang A sample & efficient web crawler for golang 15 May 2022
Crawler Crawls the website and finds broken social media links that can be hijacked Crawls the website and finds broken social media links that can be hijacked 10 April 2022
Crawler A crawler that tests HTML forms for reflection A crawler that tests HTML forms for reflection 18 March 2022
Reflection A crawler that tests HTML forms for reflection A crawler that tests HTML forms for reflection 15 March 2022
Crawler Go Site Crawler - a simple application written in go that can fetch content from a url endpoint Go Site Crawler - a simple application written in go that can fetch content from a url endpoint 05 February 2022
Domain Remake of the original sqlifinder but in GOlang, and allows for listed targets, domain crawling, and tor connections Remake of the original sqlifinder but in GOlang, and allows for listed targets, domain crawling, and tor connections 09 January 2022
Distributed Systems Pholcus: a distributed high-concurrency crawler software written in pure golang Pholcus: a distributed high-concurrency crawler software written in pure golang 03 January 2022
Crawler Crawler for StandNews, a news-network shut down by the Chinese Goverment Crawler for StandNews, a news-network shut down by the Chinese Goverment 31 December 2021
Query Pagser: a simple and deserialize html page to struct based on goquery and struct tags for golang crawler Pagser: a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler 30 December 2021
Scraper Elegant Scraper and Crawler Framework for Golang Elegant Scraper and Crawler Framework for Golang 26 December 2021
Crawler Geziyor: a fast web crawling & scraping framework for Go. Supports JS rendering Geziyor: a fast web crawling & scraping framework for Go. Supports JS rendering 21 December 2021
Distributed Systems A lightweight distributed friendly Golang crawler framework A lightweight distributed friendly Golang crawler framework 20 December 2021
Web Scraping Antch, a fast, powerful and extensible web crawling & scraping framework for Go Antch, a fast, powerful and extensible web crawling & scraping framework for Go 20 December 2021
Crawler Go spider: A crawler of vertical communities achieved by GOLANG Go spider: A crawler of vertical communities achieved by GOLANG 10 December 2021
Crawler Creeper: The Next Generation Crawler Framework (Go) Creeper: The Next Generation Crawler Framework (Go) 09 December 2021
Distributed Systems Distributed web crawler admin platform for spiders management regardless of languages and frameworks Distributed web crawler admin platform for spiders management regardless of languages and frameworks 09 December 2021
Crawler Verify IP addresses of respectful crawlers like Googlebot by reverse dns and forward dns lookups Verify IP addresses of respectful crawlers like Googlebot by reverse dns and forward dns lookups 06 December 2021
Crawler Curry is a crawler written in Golang with the purpose of verifying the exchange value of USD(Dollar) x BRL(Real) in stores in Paraguay Curry is a crawler written in Golang with the purpose of verifying the exchange value of USD(Dollar) x BRL(Real) in stores in Paraguay 28 November 2021
Domain Take a list of domains and scan for endpoints, secrets, api keys, file extensions, tokens and more Take a list of domains and scan for endpoints, secrets, api keys, file extensions, tokens and more 23 November 2021
Crawler A polite, slim and concurrent web crawler written in Go A polite, slim and concurrent web crawler written in Go 15 November 2021
Crawler A libp2p DHT crawler that also monitors the liveness and availability of peers A libp2p DHT crawler that also monitors the liveness and availability of peers 14 November 2021
Script A small script written in Go to crawl all the scores of the National High School Exam-2021 A small script written in Go to crawl all the scores of the National High School Exam-2021 12 November 2021
Search A Unix-style personal search engine and web crawler for your digital footprint A Unix-style personal search engine and web crawler for your digital footprint 12 November 2021
Telegram A simple crawler sending Telegram notification when Refurbished Macbook Air / Pro in stock A simple crawler sending Telegram notification when Refurbished Macbook Air / Pro in stock 11 November 2021
Wrapper An API wrapper for a webcrawler that finds free items off of freecycle.com and returns the in JSON format An API wrapper for a webcrawler that finds free items off of freecycle.com and returns the in JSON format 11 November 2021
Crawler A simple crawling study that obtained the new houses for sale in Wuhan, China A simple crawling study that obtained the new houses for sale in Wuhan, China 05 November 2021
Crawler Simple content crawler for joyreactor.cc Simple content crawler for joyreactor.cc 18 October 2021
Crawler New World Auction House Crawler In Golang New World Auction House Crawler In Golang 10 October 2021
Crawler A Unix-style personal search engine and web crawler for your digital footprint Interactive Terraform visualization, State and configuration explorer 08 October 2021
Web Scraping A very useful and productive web crawlign framework A very useful and productive web crawlign framework 06 October 2021
Web Scraping Gets joke from icanhazdadjoke.com, prints it Gets joke from icanhazdadjoke.com, prints it 29 September 2021
Tools A companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable A companion tool to your existing web scraper that instantly makes it scalable, maintainable, and more unblockable 28 September 2021
Web Scraping A Fast, highly configurable, cloud native dark web crawler Written in go Fast, highly configurable, cloud native dark web crawler. 22 September 2021
Crawler Web crawling by using golang Construct a product price comparison service that enables customers to search and compare prices from two online shopping platforms (Amazon and Walmart) simultaneously. 10 September 2021
Web Scraping A crawler/scraper based on golang + colly, configurable via JSON This a very thin layer on top of Colly which allows configuration from a JSON file. The output is JSONL which is ready to be imported into Typesense. 04 September 2021
Crawler A IMDb Crawler Made using Go + Colly Want to know which celebrities have a common birthday with yours? Get the full data about them. Made using Go + Colly 29 July 2021
Spider GoSpider - Fast web spider written in Go GoSpider - Fast web spider written in Go 23 June 2021