[dev] Session Prefs Caching

[Michael M Slusarz]

A problem that I've been mulling over for the last few weeks.

The problem is when the prefs data starts getting larger, we are  
hauling around this data in a storage container that must be read from  
the backend/transmitted on network on every single page access.   
Horde_Pack helps with the size issue, and the data segmentation in  
Horde_Session ensures that we don't need to unserialize the data for  
the apps we don't use.  But my unscientific examination of my personal  
session data indicates 2/3 of a typical session is taken up by this  
data.

It has become an issue since I'm currently writing support for adding  
images to HTML signature data for a client.  The problem is that  
realistically the only place to save this data is embedded within the  
HTML signature data itself (data URLs).  But images are most likely  
incompressible data, and potentially large (say 30 kb).  And there is  
no reason to be carrying around this kind of data when we are only  
using it in a single location ... and its a location where it would be  
perfectly acceptable to allow access to the prefs backend to get this  
data rather than needing to cache it.

I'm trying to figure out the right balance of caching, and features  
like HTML sig data to me is tipping the scales for getting rid of  
in-session prefs caching.  Since this is a fairly big change, I'm  
looking for input.

Possible solutions:

1.) On-demand access.  I think this should be done regardless of what  
caching solution we decide on.  Currently, every time we push an  
application on the stack we retrieve the prefs regardless of whether  
we use them or not.  We could save at least some accesses by either 1)  
editing Horde_Prefs or 2) adding an abstraction layer to the Scope  
object where we don't load the underlying Scope unless we need it.   
I'd imagine there are many page accesses where we never access prefs  
at all, especially AJAX-y actions.

1a.) Extension to this idea is to only load app prefs scope at first.   
Only if pref is not found in app scope do we need to load base horde  
prefs.

2.) Move all Prefs caching to Horde_Cache.  Simplest solution.   
Doesn't deal with inefficiencies in our current Horde_Prefs  
architecture (i.e. we still need to load the large prefs values even  
if we don't use them).  But requires minimal coding.

3.) Incremental prefs loading.  Only load/cache prefs values that are  
actually used.  No idea what kind of load this would cause on backends  
though, and this is probably more inefficient during the retrieval  
stage.

4.) "Global defaults" scope.  Now, each user has a scope object that  
combines their personal preferences with the current app defaults.  We  
can remove the default prefs from this object and instead add this as  
a separate scope, which is only queried if not found in the personal  
prefs data.  Benefit: the default scope should be the same for most  
users, so this data could be stored in a globally accessible cache key.

5.) Add ability to flag a particular cache entry as 'non-cacheable' in  
config/prefs.php.  Entries such as identity data or PGP keys should  
not be stored in session cache data and should instead be loaded  
on-demand only.  Unfortunately, the way Horde_Prefs_Scope is written  
as of now, this would be difficult to do as the non-cacheable state is  
not available at the time of storage/caching without doing something  
like re-reading the prefs.php file, which is not ideal.

michael

___________________________________
Michael Slusarz [slusarz at horde.org]