Webmaster forum - Re: Example 404's

Author

Re: Example 404's

Roy Schestowitz

2005-07-29, 12:03 am

Roy Schestowitz wrote:

> Brian Cryer wrote:
>
>
> The last time I checked, there was apparently no referrer information, but
> perhaps I wasn't carefully enough to spot it.
>
> I have just tried to open the monthly log file in order to provide you
> with an answer. Little did I realise it was around 110 MB so the RAM and
> text editor could not handle it (last time I looked at such logs was last
> year when they were much, much smaller). Looking at my daily log file, I
> can't pull out enough information. This has become really frustrating...
> *sign* why did I not stick to lowercase from day one?

I have scooped up some 404's that came up throughout the night:

===

216.32.73.122 - - [28/Jul/2005:18:17:32 +0100] "GET
/weblog/archives/2005/03/31/uninvited-mail/ HTTP/1.1" 404 2436 "-"
"anuxfmsgmic esgU6nmcpxinU 6"
216.32.73.122 - - [28/Jul/2005:18:17:40 +0100] "GET
/weblog/archives/2005/03/31/ HTTP/1.1" 404 2425 "-"
"gkbagdrvrjaNNxkbpoucdg6jqrekekm"
216.32.73.122 - - [28/Jul/2005:18:18:14 +0100] "GET
/Weblog/archives/2005/03/ HTTP/1.1" 200 150984 "-" "oukvjvcpxfll
jvnvoqidx6bjh6fncfbo"
65.190.197.211 - - [28/Jul/2005:18:51:42 +0100] "GET
/usenet/2005/june_2005_2/msg00067.html HTTP/1.1" 404 2434 "" "Opera/7.23
(Windows 98; U) [en]"
65.190.197.211 - - [28/Jul/2005:18:51:42 +0100] "GET
/usenet/2005/june_2005_2/msg00072.html HTTP/1.1" 404 2494 "" "Mozilla/5.0
(Macintosh; U; PPC Mac OS X; en) AppleWebKit/124 (KHTML, like Gecko)
Safari/125"
65.190.197.211 - - [28/Jul/2005:18:51:43 +0100] "GET
/usenet/2005/june_2005_2/msg00068.html HTTP/1.1" 404 2487 "" "Mozilla/5.0
(Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041210 Firefox/1.0"

===

There are more oddities than answers...

-What is the gibberish at end of the entries at the top? And how did the
user reach "weblog..." and not "Weblog..." (note case)

-In the latter case, even more strangely, the user requests 3 different
pages at the same time. The IP address is identical. One is Opera on
Windows 98, one is a Macintosh and one is Firefox on Window$ NT...

How can this be? These sorts of errors never stop...

Thanks for any help,

Roy

Spamhuntress

2005-07-30, 7:38 pm

I got a visit from 65.190.197.211 and its friends:
84.9.48.202
84.9.49.254

Found this post through searching, because of the oddities.

Note the fake referrer. "" is not normally used. Note the revolving
user agent. Note that these IP numbers don't load images etc. In other
words, it's a bot.

I found a document in Google's cache with a lot of inetd errors, where
the connection was closed before the server could respond.

On my site, there were usually two requests back to back for the same
file. And the 301 came first in the log, because it was completed
first. They're logged as happening the same second.

The second requests (the ones coming back with 301) were for lower case
URL's, even though the first request was the correct one with a mix of
lower and upper case (wiki, so that makes sense). On URL's with no
upper case characters, only one reqest.

The IP number 65.190.197.211 is on Road Runner from Raleigh, North
Carolina. The less used 84.9.48.202 comes back as
host-84-9-48-202.bulldogdsl.com, which is (according to RIPE) a dynamic
IP pool from a UK DSL provider! Same with 84.9.49.254.

Spamhuntress

2005-07-31, 7:17 am

In my haste, I made a few wrong conclusions here. First of all, the 301
status code, which we don't see too often, is "moved permanently", not
304 "not modified". So it's my server that sends the second request,
not the other way around. The bot only asks for the files by all small
caps, and the software (the wiki) ensures the request goes to the right
file.

Sorry for the confusion...