[horde] horde/imp migration problems, cpu bound on solaris

Thu Apr 10 12:58:40 PDT 2003

Hi folks,

I've installed setup and verified phpa from IonCube.  It's working, and
caching things, and while I think my login page is faster my inbox display
is still taking a while.

I broke down and decided to find out what in particular was being so slow.
I just used fputs and such to write to a file on the server.  it's using
the following function to track time usage:

function getmicrotime()
{
  list($usec, $sec) = explode(" ",microtime());
  return ((float)$usec + (float)$sec);
}

which I stole from phpa's website.

A single load of:

/horde/imp/mailbox.php

takes about 4.5-6 seconds, and that's after it's been loaded a couple of
times and should be cached via phpa.

here's an example run (excuse the unclear notes, but I do include line
numbers which should be only slightly off thanks to the code I inserted):

Top of mailbox.php: 0.000151038169861, line 16
  end of function definitions: 0.000332951545715, line 85
  end require_once: 0.568125009537, line 108
  end checkAuthentication(): 1.07391905785,line 117
  initialize identities: 1.07466304302, line 159
  about to run through the action handlers: 1.07507896423, line 166
  done running through the action handlers: 1.07539904118, line 384
  begin IMP::flistSelect: 1.07555103302, line 390
  end IMP::flistSelect: 3.6760610342, line 394
  begin some inbox sorting: 3.67635095119, line 399
  end sorting: 3.67826700211, line 447
  end page logic: 3.67850601673, line 563
  end some other stuff: 3.69362401962, line 600
  done with requires of .inc: 5.43777596951, line 624
  about to build array of message info and sort order: 5.43799602985, line 628
  about to display message information: 5.45003604889, line 681
  now processing footer: 5.47204101086, line 910
  storing prefs: 5.49746406078, line 934
End of mailbox.php 5.50000596046, line 943

The 2 big, bad guys are IMP::flistSelect done here (with my code stuck
in, ~2.6 seconds):

if ($conf['user']['allow_folders']) {

    $timestamp = getmicrotime() - $script_start;
    fputs($profile,"  begin IMP::flistSelect: $timestamp, line 390\n");
    fflush($profile);

    $options = IMP::flistSelect(_("Messages to"), true, array($imp['mailbox']), null, true);

    $timestamp = getmicrotime() - $script_start;
    fputs($profile,"  end IMP::flistSelect: $timestamp, line 394\n");
    fflush($profile);
}

and the "requires of .inc" which are from where I marked "other stuff" to
where I marked "done with requires of .inc" (~1.75 seconds):

$timestamp = getmicrotime() - $script_start;
fputs($profile,"  end some other stuff: $timestamp, line 600\n");
fflush($profile);

require IMP_TEMPLATES . '/common-header.inc';
if ($browser->hasFeature('javascript')) {
    include IMP_TEMPLATES . '/mailbox/javascript.inc';
}
require IMP_BASE . '/menu.php';
require IMP_BASE . '/status.php';

if (!empty($conf['hooks']['quota']) &&
function_exists($conf['hooks']['quota'])) {
    echo call_user_func($conf['hooks']['quota'], $imp);
}

require IMP_TEMPLATES . '/mailbox/header.inc';
$navform = 1;
require IMP_TEMPLATES . '/mailbox/navbar.inc';
require IMP_TEMPLATES . '/mailbox/actions.inc';
if ($imp['mailbox'] != '**search') {
    include IMP_TEMPLATES . '/mailbox/message_headers.inc';
}

$timestamp = getmicrotime() - $script_start;
fputs($profile,"  done with requires of .inc: $timestamp, line 624\n");
fflush($profile);

So I'm going to dig around and see what's holding those up.  If you folks,
who are more familiar with the code, gather any bright ideas from this...
please let me know.

-V

On Thu, 10 Apr 2003, Vasilios Hoffman wrote:

> > Try a php accelerator.  Also mcrypt in php.  Both of these were already
> > mentioned.  If you have lots of mail boxes (you don't say how many mailboxes
> > you serve) you might be able to do some kernel tuning to cache more inodes
> > and files in memory...
>
> I'm going to try the php accelerator & mcrypt actually, and will let folks
> know.
>
> > Are you sure the "seconds to execute" isn't really in the imap connection
> > and authentication, rather than in the php/horde software?
>
> Yes, I've timed that, both by hand (telnet localhost 143 and "speaking"
> imap -- horde makes a localhost connection.  I've also tried it with the
> imapproxy) and by digging into the code and logging what was going on with
> imap.  Basically, enough to know fairly certainly that imap is okay.  The
> current imp is also using the same imap server and not having the problem.
>
> >
> > No.
>
> Thank god :)
>
> > > Is this the performance everybody else is getting?
> >
> > No.
>
> Alleluia! or however you spell that.
>
> > It isn't Horde, unless you do things on the summary page and so on to slow
> > it down.
>
> nope
>
> > It could be php, it could be apache, it could be your mail server, it
> > could be a firewall, etc.  You've really not told us enough to know for
> > sure.
>
> my apologies, I wanted the email to be short enough to be paid attention
> to.  I have found that if I exauhstively (sp?) list everything I have
> done,  it drags on and nobody understands my point nor replies to it.
> Perhaps I'm wrong, but that's just my experience.  I found it was
> processor bound, and having trouble with the php code -- not imap, not
> memory, not I/O.  I tried to communicate that, and hoped folks would take
> it at face value.  If I'm wrong, I'll eat my shorts and share it with the
> list :)
>
> I also didn't want to spend all day profiling it and seeing
> where the slowdowns were in the code, not without checking to see if I
> missed anything foolish -- hence the short synopsis.
>
> I will try mcrypt and php accelerator, and get back to everybody!  I
> really appreciate the feedback.
>
> > You really need to isolate where the major slowness is.  How long, for
> > example, does a page that doesn't use the imap server run compared to
> > one that does use the imap server?  That can help answer if the imap
> > server is part of the problem or not...  If you connect via a local
> > web browser on the loopback device is it slower/faster than a remote
> > browser over the network?  If so, your network may be slow...
> > Simple tests like that will go a long way to finding out where the
> > problems/slowness is really at.
>
> Again, most of these variables are ruled out because the machine is
> currently in use, running imp, and running it well enough.  I know the
> network is fined, switched 100base-T and the switches are not at max.  the
> imap is used by the old imp, and I've tested it otherwise, and the only
> thing left with imap that's plausible is some software bug between this
> version of php and this version of imap, but I've tried the old imap and
> another version of php and can rule that out enough to satisfy me.
>
> Sorry, I didn't want to bore everybody listing everything I did!  We also
> use this software (apache, mod_ssl, etc) in dozens of places and don't
> have problems with it, I'm familiar enough with the setup and the
> variables to say with fair certainty that it is the php processing time
> that is dogging it.  (Oh, the database is fast enough too, I checked
> that).
>
> Thanks!  If anybody has php.ini suggestions or other tracks I'll follow
> those, otherwise I'll be back in a bit with mcrypt/php accelerator
> results.
>
> -V
>
>
>
> --
> Horde mailing list
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: horde-unsubscribe at lists.horde.org
>