If you have wondered where you could redirect the robots.txt disobedient crawlers and bad bots, you have found the right place
|
|
|
Add the following to your .htaccess, in quotes to redirect bad robots and crawlers (not the old fashioned block) "
RewriteEngine on ^(.*) = the ^(.*) signifies any user agent that contains the given name, and we have used Overkill. This will block any user agent string containing Overkill, a word, phrase, or characters you specify. ^$ = the ^$ signifies a specific, and full user agent name that will be blocked. It will block the full, complete, and exact user agent string Exact, or any other user agent string you specify. If you do not add any user agent name and leave it blank, every suspicious person not using a user agent will be blocked. ^ = the ^ by itself will block any user agent string that starts with Starter, or any other name of a user agent that you write. |
|