[horde] runaway httpd processes?

[Dominaux, Craig C]

I sort of found out why our httpd processes are hanging up. Students are
creating large text messages about 1.2 MB in size. I copied one of those
messages to my account and tried to open it, the message will not open
and the one of the httpd process starts to hang. (CPU Usage sticks on
99% utilization)

Now I'm not sure if there is a fix for this yet, or I have to reduce the
POST message size to something smaller than 1MB.
But at least I understand why it happens; I just need to figure out how
to prevent it from happening again.

Thanks,

Craig 


-----Original Message-----
From: horde-bounces at lists.horde.org
[mailto:horde-bounces at lists.horde.org] On Behalf Of Michael M Slusarz
Sent: Monday, January 29, 2007 3:14 PM
To: horde at lists.horde.org
Subject: Re: [horde] runaway httpd processes?

Quoting "Dominaux, Craig C" <CCDominaux at cbe.ab.ca>:

> I don't think the issue for us anyway is with the SQL queries because
we
> truncate the horde_datatree and horde_datatree_attributes every hour
to
> keep these tables low. And the reason we can get away with this is we
> don't allow any sharing or set permissions on our address books,
> calendars, notes, or tasks.
>
> Our horde_datatree_attributes don't usually get any higher than 150000
> rows or the horde_datatree with 35000 rows.
>
> Since I put in this script login time have been very fast but didn't
> help with runaway httpd processes.

I don't want to sound demanding (well, maybe I do...) but it would  
*tremendously* help if you could figure out the URI's these runaway  
httpd processes are attempting to serve.  Horde developers can give  
much better advice and, if appropriate, attempt to remedy the problem  
when we know what the problem is.

For example, we recently ran into an issue where a Horde installation  
was seeing runaway processes.  Trying to debug, that information  
didn't help much.  However, analysis of the runaway httpd processes  
indicated there was a specific URI call to a page in kronolith with  
one single specific parameter passed in that allowed me to track this  
down in an hour to make this tiny change:
http://lists.horde.org/archives/cvs/Week-of-Mon-20070115/064855.html

Here was the server analysis that sparked the discovery (thanks nuno):

20:19:59 up 29 days,  3:14,  1 user,  load average: 55.16, 38.82, 23.92
  412 processes: 403 sleeping, 8 running, 1 zombie, 0 stopped
  CPU states:  82.5% user,  17.5% system,   0.0% nice,   0.0% idle
  Mem:   3883356K total,  3865044K used,    18312K free,   207128K
buffers
  Swap:   979924K total,    61724K used,   918200K free,  2434148K
cached

    PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
   4858 webmail   25   0 29196  28M 24876 R    71.7  0.7  17:39 httpd
  27948 webmail   25   0 31888  31M 27104 R    13.5  0.8   6:20 httpd

  PID   REQUEST
  4858  GET /kronolith/year.php?year=1104543267 HTTP/1.1
  27948 GET /kronolith/year.php?year=1199151268 HTTP/1.1

The biggest issue was the fact that I didn't see this on my local  
machine when debugging because the runaway process turned out to be a  
PHP bug.  It wasn't until I was able to track the potential issues  
down to a few lines of code that I was able to cross-reference with  
the PHP bugs database to determine the problem was this installation  
was using an older version of PHP that had the bug (in the mktime()  
function) while my local installation wasn't seeing any problems  
because I was using a newer version of PHP.

As far as how to do this server analysis - that's not something that I  
am personally going to be able to help with.  Depending on your setup,  
this will vary.  But if you can at least track the runaway processes  
down to a specific URI (and most likely a specific e-mail message), we  
will be more than happy to try to track it down.

michael

___________________________________
Michael Slusarz [slusarz at horde.org]

-- 
Horde mailing list - Join the hunt: http://horde.org/bounties/#horde
Frequently Asked Questions: http://horde.org/faq/
To unsubscribe, mail: horde-unsubscribe at lists.horde.org