Wild Spider
Item media 1 screenshot

Overview

web pages are crawled by being loaded into browser using multiple tabs parallelly

WATCH OUT: more tabs you use, more computer resources (CPU, memory) will be used, and each page costs a bit disk to save the content (in IndexedDb, accessible from extensions -> Inspect views: background page). The "spider" works in this way: 1) The current url is used as the starting point, and it's loaded again in a new tab. 2) After this page is loaded, fetch all the links on the page. 3) Get all the links on the page, including relative urls. 4) Open the extracted link parallelly in all the tabs used (by default 3, set in eventPage). 5) repeat 2-4 All source code at: https://github.com/nobodxbodon/ChromeCrawlerWildSpider

1 out of 51 rating

Google doesn't verify reviews. Learn more about results and reviews.

Review's profile picture

Matt CMar 19, 2018

doesnt work

Details

  • Version
    0.0.3
  • Updated
    March 8, 2019
  • Offered by
    Xuan Wu
  • Size
    121KiB
  • Languages
    English (United States)
  • Non-trader
    This developer has not identified itself as a trader. For consumers in the European Union, please note that consumer rights do not apply to contracts between you and this developer.

Privacy

The developer has not provided any information about the collection or usage of your data.

Support

Related

get-set, Fetch! web scraper

2.5(2)

Open source data scraper with csv, zip export capabilities. Provides scraping scenarios for fast, minimal configuration scraping.

beABot

3.7(3)

Become a javascript crawler bot (beta)

Scraper Crawler V3

2.2(29)

Crawl to any search results and scrape texts, links, images, keywords, emails & videos

Crawler

3.2(12)

This extension could be used to crawl all images of a website

Fun Crawler

2.3(3)

Fun Crawler aimlessly crawls your website by clicking on clinks

JIRA Crawler

5.0(1)

Bug tracking

Page Crawler

4.5(6)

Scrape links from any site

Creeper

3.2(4)

Scrape data from list-style web pages. There are samples for demo. Very easy to use. One minute configuration by just clicking.

Semantic Spider

4.0(3)

Chrome becomes more intelligent and personalized with W3C Semantic Web

Link Collector

5.0(1)

Quickly view and access all the links on any page.

Recursive

3.4(12)

Explore the endless web!

LinkAnalysis

4.6(18)

Analyses the links in a page

get-set, Fetch! web scraper

2.5(2)

Open source data scraper with csv, zip export capabilities. Provides scraping scenarios for fast, minimal configuration scraping.

beABot

3.7(3)

Become a javascript crawler bot (beta)

Scraper Crawler V3

2.2(29)

Crawl to any search results and scrape texts, links, images, keywords, emails & videos

Crawler

3.2(12)

This extension could be used to crawl all images of a website

Fun Crawler

2.3(3)

Fun Crawler aimlessly crawls your website by clicking on clinks

JIRA Crawler

5.0(1)

Bug tracking

Page Crawler

4.5(6)

Scrape links from any site

Creeper

3.2(4)

Scrape data from list-style web pages. There are samples for demo. Very easy to use. One minute configuration by just clicking.

Google apps