Nov 012007

MSNPTC/1.0 appears to be a poorly written spider bot. It has no concept of distributing requests. I am not the only person that thinks this:

Let me demonstrate with an example:

$ grep -c "MSNPTC/1.0"

$ grep "MSNPTC/1.0" | head -1 - - [01/Nov/2007:16:52:00 -0400] "GET /login.php?origin=%2Fcommit.php%3Fcategory%3Dwww
HTTP/1.1" 200 16533 "-" "MSNPTC/1.0"

$ grep "MSNPTC/1.0" | tail -1 - - [01/Nov/2007:17:05:44 -0400] "GET /devel/?page=4 HTTP/1.1" 200 3849 "-" "MSNPTC/1.0"

That’s 1975 requests in 14 minutes. Roughly roughly 141 per minute or 2.35 per second. Now, that’s not quite reflective of what happened.

The peak period of activity ended at 16:56:12 with 1971 hits. That is a period of 252 seconds. Or about 8 requests per second.

That’s not very sharing and caring. I’ll have no more of that. Take this:

    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} "MSNPTC/1.0"
    RewriteRule .*

Goodbye Mr Chips

Website Pin Facebook Twitter Myspace Friendfeed Technorati Digg Google StumbleUpon Premium Responsive