[imp] Re: Requested message not found

[Gary Windham]

Quoting Joaquim Homrighausen <joho at webbplatsen.se>:

> Hi there,
>
> I too have seen this; for a long time. But I haven't been able to figure
> out a pattern. It does happen more frequently right after switching
> folders. I've had situations where I've clicked on the most recent
> message (the same one) over 50 times and getting the same error message
> every time.
<Long-winded explanation of "requested message not found" follows>

We used to suffer from this problem, but I believe I finally figured out the
root cause.  If you are using a PHP session handler other than the standard
PHP "mod_files" one, then it is extremely likely that it is not implementing
serialization of session access (none of the ones that are included in the
Horde framework do).  The "mod_files" PHP session handler uses flock() to
exclusively lock a session from the time it is opened until the time it is
closed, thereby preventing shared (even read) access to the session data.

Several Horde scripts output Javascript that causes the browser to reconnect
to the server in order to retrieve CSS files, Javascript function libraries,
etc.  Unless you are using compression, or some other output handling
mechanism that forces blocking of output until the entire PHP script
completes execution, it is very likely that one of these ancillary
connections to the web server will be accessing the same session data
concurrent with the primary script.  A typical scenario that we saw went
something like this (imagine that you are current sitting in folder 'foo' and
you switch to folder 'bar'):

/horde/imp/mailbox.php?mailbox=bar
        |
        V
   session_read()
        |
        +----(data output)----> browser reconnects to get some JS code
        |                                     |
        |  (more data...)                     V
        |                                session_read()
        V                                     |
    session_write()                           V
    session_close()                      session_write()
                                         session_close()

As you can see, the secondary connection to retrieve the CSS data reads the
stale session state before the initial PHP script has a chance to call
session_write().  And since every PHP request involves re-writing the session
data, the session state written by the "mailbox.php" invocation is trampled by
the second one.  Now, according to the session state, you are still in folder
"foo", although your browser shows you in folder "bar".  Of course, message
UID numbers are probably different in these two folders, resulting in the
aforementioned error message when you try to open one.

We run 6 Horde/IMP servers behind a load balancer and, for obvious reasons,
want to share session state between them.  We had great difficulties with
this (we were using NFS for session files) until we implemented a kludgey
workaround: we now use memcached to store session data, and our layer-7 load
balancer pins a user's session to a particular server; we then serialize
session access (in a custom Horde/SessionHandler driver) via local lock files
and flock().  If you are using a load balancing solution that doesn't pin a
user's session to a single server, then you'll need to find a session cache
mechanism that properly serializes access.

Hope this helps,
--Gary
--
Gary Windham
Systems Programmer, Principal
The University of Arizona, CCIT