Nov 012007
 

MSNPTC/1.0 appears to be a poorly written spider bot. It has no concept of distributing requests. I am not the only person that thinks this:

Let me demonstrate with an example:

$ grep -c "MSNPTC/1.0" freshports.org-access.log
1975

$ grep "MSNPTC/1.0" freshports.org-access.log | head -1
219.142.53.25 - - [01/Nov/2007:16:52:00 -0400] "GET /login.php?origin=%2Fcommit.php%3Fcategory%3Dwww
%26port%3Deventum%26files%3Dyes%26message_id
%3D200708050837.l758baWl059884%40repoman.freebsd.org 
HTTP/1.1" 200 16533 "-" "MSNPTC/1.0"

$ grep "MSNPTC/1.0" freshports.org-access.log | tail -1
219.142.53.27 - - [01/Nov/2007:17:05:44 -0400] "GET /devel/?page=4 HTTP/1.1" 200 3849 "-" "MSNPTC/1.0"

That’s 1975 requests in 14 minutes. Roughly roughly 141 per minute or 2.35 per second. Now, that’s not quite reflective of what happened.

The peak period of activity ended at 16:56:12 with 1971 hits. That is a period of 252 seconds. Or about 8 requests per second.

That’s not very sharing and caring. I’ll have no more of that. Take this:

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} "MSNPTC/1.0"
    RewriteRule .*  http://news.freshports.org/

Goodbye Mr Chips

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive