disallow

Here are 3 public repositories matching this topic...

A python script to check if URLs are allowed or disallowed by a robots.txt file.

web robots-txt check bugbounty bypass allow disallow

Chrome extension that audits which paths a site disallows in robots.txt and how many of them Google has indexed anyways.

A simple restrictions addon for gmod

lua addon gmod pac3 restrictions qmenu toolgun prop-spawning disallow q-menu allowing gmod-restrictions

Add a description, image, and links to the disallow topic page so that developers can more easily learn about it.

To associate your repository with the disallow topic, visit your repo's landing page and select "manage topics."