- Fixed: robots.txt should not exclude all crawlers from the website by def...

mcramer

2013-01-31 5aaa0eff10ef3835a8edbaa20e1272f04a88ab1f

1
2
3
[Definition]
failregex = (?: pop3-login|imap-login): .*(?:Authentication failure|Aborted login \(auth failed|Aborted login \(tried to use disabled|Disconnected \(auth failed).*rip=(?P<host>\S*),.*
ignoreregex =