Difference between revisions of "Trouble with BOTs"

From The System Administrator Zone
(Created page with "Category:System Management Tools BOTs can cause serious system load issues. === Google BOT === Google does not follow the instructions set in the robots.txt file. If yo...")
(No difference)

Revision as of 22:48, 10 August 2017


BOTs can cause serious system load issues.

Google BOT

Google does not follow the instructions set in the robots.txt file. If you are having system load problems related to Goggle BOT, you need to reduce the Google crawl rate to a minimum to lower the resource usage. You do that from your Google Search Console:

See: Change Googlebot crawl rate