[horde] Apache running Horde segfaults

Tue Apr 4 10:27:17 PDT 2006

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Daniel A. Ramaley wrote:
> Thanks for the response. That you are seeing similar problems with
> slightly different software (MySQL instead of PostgreSQL, slightly
> older Horde, RHEL WS instead of AS) indicates to me that it is a
> general problem, and not something peculiar to the exact configuration
> i'm working with.
>
> So far it looks like restarting Apache every hour or so (!) would fix
> the problem for me. However, that seems a bit excessive to me. During
> peak periods the problem doesn't occur until about an hour and a half.
> This morning the server hasn't been getting hit as hard and it took 3
> 1/2 hours.
>
> As i've worked with more 64-bit systems in the last few months i've
> gotten the impression that 64-bit is just not ready for prime time. So
> it doesn't surprise me that you suspect that to be the problem.

Unfortunately, as I said before, that is what is sold currently. The
only option would appear to be running a 32-bit OS on top of them, or at
least some 32-bit software.

Where did you get all of your Apache stuff? RHEL RPM's?

> As to what you've tried, i had noticed in mailing list archives that
> mod_perl is blamed for many segmentation fault problems. I checked my
> installation and we don't have it installed (i was kind of hoping we
> did have it; we aren't using it and if removing it would have fixed the
> problem, well that would have been nice and simple). We aren't running
> turck_mmcache, but are running eAccelerator, which does the same thing.
> I can try turning off the PHP accelerator. We might need a faster
> machine to run Horde if i do that though. And a dual processor Opteron
> is already a fairly beefy system.

Are you certain? It was installed by default on my system. Unless you
are running from non-RPM's, there is probably a perl.conf in your
/etc/httpd/conf.d directory that's being sourced (unless you've already
removed it, but from the sound of it, you said you've never had it).

> Could you please check what your MaxRequestsPerChild is set to? For
> comparison, the httpd.conf section that deals with child processes at
> my installation looks like this:
>     StartServers       8
>     MinSpareServers    5
>     MaxSpareServers   20
>     ServerLimit      256
>     MaxClients       128
>     MaxRequestsPerChild  1024
> I might try setting MaxRequestsPerChild to something extremely low and
> see if that fixes the problem. If it does and i don't have to disable
> the PHP accelerator, that would be great. I tend to think of turning
> down MaxRequestsPerChild to be a kludge similar to restarting Apache on
> a regular basis, though.

I just lowered mine to 1024 after talking to you. It has segfaulted once
an hour later. Typically I get a LOT of faults, though, so that might be
an improvement. I'll have to watch awhile longer to really be able to tell.

Incidentally, watch out for restarts -- the cache directory of those
accelerators will really screw things up if there are orphaned files in
there.

> On Tuesday 04 April 2006 11:15, Ryan Novosielski wrote:
>> Hey Daniel,
>>
>> To start out, I've got roughly the same configuration. I run RHEL4.3
>> WS on a V20z machine. The differences are that I run Horde 3.0.10 and
>> MySQL, but my problem is the same and everything else is the same,
>> and I'm currently solving it by restarting Apache every once in
>> awhile during peak periods. This is a solution I abhor, but I have to
>> if I want to keep the machine running.
>>
>> The problem appears to be at least somewhat related the the machine
>> being 64-bit. I've heard suggestions by the author of Bacula that
>> there might be a gcc bug on 64-bit platforms with regard to memory
>> addressing. I've also seen some evidence to indicate that 64-bit
>> Linux platforms are not stable for PHP. Unfortunately, that one
>> doesn't fly for me because that is the equipment I've got. I've
>> considered putting 32-bit Apache on the machine, but that's a gamble
>> and I do not have spare equipment to try this on.
>>
>> It might be worth noting what I've tried so far, and the results.
>> First off, I've removed mod_perl; reports say that these two share
>> memory improperly. I don't use mod_perl, so there's no reason for me
>> to have had it installed anyway. This improved things a certain
>> amount, but did not kill off the problem. Removing turck_mmcache DID
>> seem to quell the problem altogether, but my machine cannot handle
>> the load without the piece of software. I've seen suggestions that
>> PHP accelerators are particularly susceptible to this kind of thing
>> on 64-bit platforms.
>>
>> Where do we go from here? No idea really. One idea that recently came
>> to me is to lower "MaxRequestsPerChild" to ensure more subserver
>> turnover. Not sure what mine is set to right now. If it's unlimited,
>> changing this value might stop the problem.
>>
>> It is definitely worth knowing that this IS a problem though, it's on
>> a major architecture (AMD64, or at least Sun AMD64 and RHEL4). I hope
>> that something can be accomplished here.
>>
>> Daniel A. Ramaley wrote:
>>> Hello. I have been testing a Horde Imp webmail server for some time.
>>> Yesterday it finally went live, and the amount of traffic the server
>>> had to deal with increased by a couple orders of magnitude. Since
>>> then there has been a major problem. After running fine for awhile
>>> (where "awhile" is on the order of an hour and a half), the server
>>> will start returning empty pages. When this happens, Apache's
>>> error_log gets filled with lines like this:
>>>
>>> [Tue Apr 04 08:07:06 2006] [notice] child pid 9954 exit signal
>>> Segmentation fault (11)
>>>
>>> We are using a PostgreSQL database. It continually writes messages
>>> to its log (whether there is a problem or not) that look like:
>>>
>>> Apr  4 08:23:17 sun12 postgres[10281]: [1-1] LOG:  unexpected EOF on
>>> client connection
>>>
>>> At one time i was having a problem with PostgreSQL not accepting
>>> enough client connections; right now it is set to "max_connections =
>>> 256". I believe the relevant line in Apache's configuration is
>>> "MaxClients 128" which is clearly less than PostgreSQL's limit so it
>>> shouldn't be a problem anymore.
>>>
>>> Has anyone seen a problem like this? How did you fix it?
>>>
>>> The server is a Sun v40z, which is a dual Opteron box with 2 GB RAM.
>>> It is running Red Hat Enterprise AS 4, with these versions of the
>>> Horde software and other components:
>>>   Horde 3.1.1
>>>   Imp 4.1
>>>   Ingo 1.1
>>>   Kronolith 2.1
>>>   Passwd 3.0
>>>   Turba 2.1
>>> The Apache version is 2.0.52, with PHP 4.3.9 and PostgreSQL 7.4.8.
>>> All the server runs is the web server and database; the Imap server
>>> it connects to is on a different machine. We are also using ldap
>>> address books, but they again are on a different machine. The Horde
>>> server has plenty of CPU (about 70% idle when it is getting hit
>>> hardest) and RAM (over half of the RAM is just being used by Linux
>>> for caching).
>
> ------------------------------------------------------------------------
> Dan Ramaley                            Dial Center 118, Drake University
> Network Programmer/Analyst             2407 Carpenter Ave
> +1 515 271-4540                        Des Moines IA 50311 USA
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (MingW32)

iD8DBQFEMqx1mb+gadEcsb4RAhzOAKCLbcU+l8ktxRIopeGW0JnEQYG+XgCg4AIr
U0aA3HDV04aMGGXmelzjRl8=
=qs77
-----END PGP SIGNATURE-----