next »

[ctmiller]

Hi I have been running calendarscript fine for quite a while.  In the last couple of days It has started to overload the server I am not sure why.  
This is all I get from the host saying what is happening.
any ideas.

ctmille2 7784 0.0 0.0 9748 7936 ? R 02:03 0:00 /usr/bin/perl calendar.pl
ctmille2 15422 0.0 0.0 8956 6348 ? R 02:16 0:00 /usr/bin/perl calendar.pl
ctmille2 1713 0.0 0.0 8560 6984 ? R 02:25 0:00 /usr/bin/perl calendar.pl
ctmille2 14249 0.0 0.0 8296 6668 ? R 02:33 0:00 /usr/bin/perl calendar.pl
ctmille2 29866 0.0 0.0 8032 6488 ? R 02:40 0:00 /usr/bin/perl calendar.pl
ctmille2 9625 0.0 0.0 7768 5088 ? R 02:48 0:00 /usr/bin/perl calendar.pl
ctmille2 22468 0.0 0.0 7636 6156 ? R 02:55 0:00 /usr/bin/perl calendar.pl
ctmille2 5956 0.0 0.0 7504 4976 ? R 03:03 0:00 /usr/bin/perl calendar.pl
ctmille2 12274 0.0 0.0 7372 5956 ? R 03:19 0:00 /usr/bin/perl calendar.pl
ctmille2 32239 0.0 0.0 7240 5876 ? R 03:27 0:00 /usr/bin/perl calendar.pl
ctmille2 14012 0.0 0.0 7240 5796 ? R 03:35 0:00 /usr/bin/perl calendar.pl
ctmille2 27972 0.0 0.0 7108 5796 ? R 03:43 0:00 /usr/bin/perl calendar.pl
ctmille2 9607 0.0 0.0 7108 4152 ? R 03:50 0:00 /usr/bin/perl calendar.pl
ctmille2 20149 0.0 0.0 6976 5168 ? R 04:08 0:00 /usr/bin/perl calendar.pl
ctmille2 15623 0.0 0.0 6844 4744 ? R 04:15 0:00 /usr/bin/perl calendar.pl
ctmille2 32022 0.0 0.0 6844 5500 ? R 04:22 0:00 /usr/bin/perl calendar.pl
ctmille2 20478 0.0 0.0 6844 5436 ? R 04:29 0:00 /usr/bin/perl calendar.pl
ctmille2 13539 0.0 0.0 6712 5188 ? R 04:37 0:00 /usr/bin/perl calendar.pl
ctmille2 25894 0.0 0.0 6712 5340 ? R 04:44 0:00 /usr/bin/perl calendar.pl

[DanO]

** any ideas. **

Maybe your calendar is being spidered multiple times by search engines. What is the IP address of all those hits?

Dan O.

------------------

[musicvid]

ctmiller,
Try putting these lines in your root .htaccess file.

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^OmniExplorer_Bot [NC]
RewriteRule .* - [F]

OmniExplorer is a "bad" bot that doesn't honor robots.txt and follows every possible link in calendarscript to the end of the chain. One of these bots took 500MB of bandwidth from one of my sites in just under two days!

------------------

[ctmiller]

I will do that for this bot but looking at my logs it looks like the googlebot is the one getting stuck how would i do a rewrite to keep the google spider from getting stuck in the calendar it seems it has eaten close in 800 meg of bandwidth in one visit  

------------------
Charles T Miller

[musicvid]

I just keep them all out by doing this in robots.txt

User-agent: *
Disallow: /cgi-bin/calendar/

Rather than do a global you could just put Googlebot instead

User-agent: Googlebot
Disallow: /cgi-bin/calendar/

Most "good" bots honor robots.txt
Unfortunately Calendarscript isn't crawler-aware. Maybe someone could write a mod to do this.

------------------

[This message has been edited by musicvid (edited May 26, 2006).]