[imp] Horde/IMP performance problems

[lst_hoe02 at kwsoft.de]

Zitat von Jochen Roderburg <Roderburg at uni-koeln.de>:

>
> Thank you, Andreas, I see some interesting points there.
>
> The following remarks are not only for Andreas, but contain some  
> more questions for other readers, too.  ;-)
>
>> There are some "bells-and-whistels" in the new version which can  
>> kill  your performance..
>> Some suspect settings
>
>> - message preview
>
> Yes, I have read about this and have seen it myself in my own test.  
> I had even a particular single message which alone caused a huge  
> delay when displaying the mailbox list. But this is more a thing to  
> watch out in the future. As this is a new user option which has to  
> be explicitly activated it cannot be involved in my initial problems.

It depends. It is also possible to activate it by default in the  
prefs.php files.

>> - message sort
>
> I remember a discussion here that there can be a problem with  
> certain (also new) sort options. Anybody can help me what these were  
> exactly?

It boils down to sorting options which the IMAP server does not handle  
well because using a not indexed value or for some other reasons.

>> - spell check
>
> I think, this is not much used here.
>
> But annother candidate in this category could also be the new  
> autocomplete in the address input fields. I do not see an option to  
> deactivate it, and it certainly creates more database traffic.

There was a hack to limit the autocomplete to only trigger after at  
least typed in n-characters which should save you from big result sets  
floating around.

> And database traffic in general: During my unintended "public  
> performance test" I had a situation where database connection were  
> not all accepted by our MySQL server and I saw many warnings/errors  
> in the imp log with the names of the new groupware components which  
> we had not in use before. I got the impression that these program  
> parts create a lot of background activity even when they are not  
> really used.

You have not really mentioned in your first post but from this i guess  
the biggest problem is the DB load? If yes check for the following:

- It was some discussion on the list about MySQL don't handle a  
frequent used query used for "shares" very well. You can try disable  
sharing at all.

- There were some heavy queries used by Kronolith and the reminder  
feature. This has also been discussed on the list.

- You need to allow as many connections to the database as you have  
webserver processes and maybe persistant connections could help.

As i'm not affected by these (using PostgreSQL) i can only point you  
to the list archives.

>> - not using the cache settings
>
> Hmm, I actually have this new option deactivated currently.
> During my tests I had it active first (with local file storage) but  
> after a while the system came in a state where it did no longer  
> displayed things correctly. This issue disappeared again with  
> deactivated horde cache. So for the moment I have not yet much  
> confidence in this caching. I mean there was also recently a  
> discussion here that there are known errors in this caching code  
> which will only be corrected in the next imp generation. Anybody can  
> shed more light on this?

Not seen here. But our load is much lower.

> If it works, I think using the recommended memcache system would help here.
> In my situation I have enough RAM available for storing everything  
> possible into RAM (all our web servers run on VMware systems). More  
> CPUs would cause additional costs (for a Redhat Server license) but  
> would also be possible if I find out that they help.

Be aware that virtualisation could cut I/O by a big margin and heavy  
I/O lead to high CPU loads on the host.

>> - using maillog with high mail volume
>
> Yes, but that's a feature that we want to use, it will help us  
> detecting cases of abuse (automated sending of spams).
>
>> What is needed at PHP level
>> - a opcode cache like APC
>
> This seems to be the single most given recommendation for PHP  
> applications, I will certainly try it out. It should mainly reduce  
> CPU usage?

Yes, in our case it halfed the CPU load...

>> - mcrypt support
>
> On board.
>
>> Also check that all the index settings in the database are correct.
>
> What do you mean with correct, what could I do/check here?
>
> As the old database was no longer usable with the new version (e.g.  
> different field names, different character codes) I had first  
> installed a completely new database with the scripts from the  
> current versions. Later on I have transferred the contents of our  
> old DB to the new one.
> This was a complicated process of its own, which I found nowhere  
> really explained, and it needed several iterations with a lot of  
> trial-and-error until I had this approximately correct.
> Must I do something with the Indices after such a transfer?

Not in normal cases when starting from scratch. But it is always  
advisable if your database is maxing out to have a look at the queries  
and the indexes used.

>> Another point was a problem with UW-IMAP with default settings some  
>>  time ago.
>
> Not here, our IMAP server is Cyrus.

Should be fine.

Regards

Andreas


-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6046 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.horde.org/archives/imp/attachments/20100910/d055263a/attachment.bin>