Robots.txt Disallowed
ExtensionDeveloper Tools3 users
Overview
This extension will report if a page is disallowed in robots.txt
See if a URL is allowed or disallowed in robots.txt for a specific user-agent token. If the URL is disallowed, it will not be crawled by the crawler with the matching user-agent token.
0 out of 5No ratings
Google doesn't verify reviews. Learn more about results and reviews.
Details
- Version1.1
- UpdatedJune 5, 2020
- Offered byXYZing
- Size43.69KiB
- LanguagesEnglish
- Developer
Email
chris@chris24.co.uk - Non-traderThis developer has not identified itself as a trader. For consumers in the European Union, please note that consumer rights do not apply to contracts between you and this developer.
Privacy
The developer has not provided any information about the collection or usage of your data.