MSNPTC/1.0 appears to be a poorly written spider bot. It has no concept of distributing requests. I am not the only person that thinks this:
- http://www.webmasterworld.com/forum11/2845.htm
- http://www.botsvsbrowsers.com/details/4800/index.html
- http://boardreader.com/t/Spiders_Crawlers_509179/MSNPTC_1_0_2849.html
Let me demonstrate with an example:
$ grep -c "MSNPTC/1.0" freshports.org-access.log 1975 $ grep "MSNPTC/1.0" freshports.org-access.log | head -1 219.142.53.25 - - [01/Nov/2007:16:52:00 -0400] "GET /login.php?origin=%2Fcommit.php%3Fcategory%3Dwww %26port%3Deventum%26files%3Dyes%26message_id %3D200708050837.l758baWl059884%40repoman.freebsd.org HTTP/1.1" 200 16533 "-" "MSNPTC/1.0" $ grep "MSNPTC/1.0" freshports.org-access.log | tail -1 219.142.53.27 - - [01/Nov/2007:17:05:44 -0400] "GET /devel/?page=4 HTTP/1.1" 200 3849 "-" "MSNPTC/1.0"
That’s 1975 requests in 14 minutes. Roughly roughly 141 per minute or 2.35 per second. Now, that’s not quite reflective of what happened.
The peak period of activity ended at 16:56:12 with 1971 hits. That is a period of 252 seconds. Or about 8 requests per second.
That’s not very sharing and caring. I’ll have no more of that. Take this:
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} "MSNPTC/1.0"
RewriteRule .* http://news.freshports.org/
Goodbye Mr Chips
