[dev] datatree performance (again)

[Didi Rieder]

Quoting "Kevin M. Myer" <kevin_myer at iu13.org>:

> Quoting Eric Jon Rostetter <eric.rostetter at physics.utexas.edu>:
>
>> Quoting Didi Rieder <adrieder at sbox.tugraz.at>:
>>
>>>> With less users on the system (night time) the queries take less than
>>>> a second. But with many (> 500) concurrent users they can take 8 sec.
>>>> and more.
>>
>> One idea I had was that it might be a DB issue (mysql vs pgsql vs
>> oracle, etc).
>> The other idea people have expressed is that it is a concurrent user issue.
>>
>> I don't see any problems on my site, but I max out at 64 concurrent users.
>> So maybe I don't have enough people hitting my site to trigger a melt down.
>>
>>> so it is about 100 queries / sec. on average in the last 21 days,
>>> which include nighttime and holidays.
>>
>> Since it is no doubt much higher at peak, it could be just too many queries
>> for the DB to handle...
>
> Just to add some more data to the mix:
>
> MySQL 4.1.X, InnoDB and MyISAM tables; about 30 databases total
>
> Supports multiple Horde installs (12), but each install has a separate
> database; all Horde databases are MyISAM
>
> The two largest concurrent users peak at 100 users and 60 users,
> respectively.  Our own install is the latter, so I'm pretty sure I'd
> pick up degradation since I use it on a daily basis.
>
> I see no performance degradation, although these are fairly beefy
> database servers.

Well when I was talking about conurrent users I was not counting  
sessions, but cached imap connections on our imapproxy (which reflects  
the number of active users in a 5 minute window). This means that  
there will be actually much more concurrent sessions (users which were  
not reading/writing emails within the 5 minute window)

I also do not have performance problems with 250 active users. But as  
the number is reaching 500 and more I do. I think/thought that the  
database machine is good enough to handle a reasonable number of  
requests (4way Sun system with 8Gb of memory). But now I see that the  
number of requests seem just to be to much.

I would like to turn of the datatree (set the driver to the null  
driver) but than kronolith and nag are not working anymore (users  
can't add new entries to their own calendar or tasklist).

In the mean time I just delete the horde.shares.[kronolith|nag]  
entries form the datatree tables with a cronjob every 10 minutes. This  
at least keeps the load down.

Didi

-- 
-------------------------
Didi Rieder
adrieder at sbox.tugraz.at
PGPKey ID: 3431D0B0
-------------------------