Difference between revisions of "Trouble with BOTs"

From The System Administrator Zone
(Created page with "Category:System Management Tools BOTs can cause serious system load issues. === Google BOT === Google does not follow the instructions set in the robots.txt file. If yo...")
 
(Google BOT)
 
Line 8: Line 8:
 
If you are having system load problems related to Goggle BOT,  
 
If you are having system load problems related to Goggle BOT,  
 
you need to reduce the Google crawl rate to a minimum to lower the resource usage.   
 
you need to reduce the Google crawl rate to a minimum to lower the resource usage.   
You do that from your Google Search Console:
+
You do that from your [https://www.google.com/webmasters/tools/home?hl=en Google Search Console].
  
 
See: [https://support.google.com/webmasters/answer/48620?hl=en Change Googlebot crawl rate]
 
See: [https://support.google.com/webmasters/answer/48620?hl=en Change Googlebot crawl rate]

Latest revision as of 22:52, 10 August 2017


BOTs can cause serious system load issues.

Google BOT

Google does not follow the instructions set in the robots.txt file. If you are having system load problems related to Goggle BOT, you need to reduce the Google crawl rate to a minimum to lower the resource usage. You do that from your Google Search Console.

See: Change Googlebot crawl rate