[dev] Fix kronolith-agenda script

[Jan Schneider]

Zitat von Gonçalo Queirós <goncalo.queiros at portugalmail.net>:

> Hi there dev.
>
>  We were trying to use the kronolith-agenda script to send daily agendas to
> everyone on our service, but the problem is that we currently have more
> than 350.000 shares and the script just runs out of memory, even with 2Gb!
>
>  Looking at the script more closely, we think there's a way to make the
> script work, regardless of the number of shares on the system, but we would
> like your opinion on that before coming out with a patch.
>
>  Current script:
>  1 - Get every calendar share
>  2 - For every share, list its events, to check if it has any event to the
> current day

This is not correct, there is no such step 2.

>  3 - For the remaining list get all users that have access to the calendars
>  4 - For every user, check if he desires to receive the daily agenda (pref)
>  5 - For every user that wants to receive the daily agenda, get his
> calendars
>  6 - Again for every calendar get the ones that have any event to the
> current day
>  7 - Send the email if there's any calendar left
>
>  For our installation the current script stops on the first step, because
> it runs out of memory.
>  We thought on creating sub-sets for the shares, but the problem is that we
> only know the full agenda of a user after we analyze all shares he has
> access to.
>
>  What we propose:
>  1 - Get every users that desire to receive the daily agenda (pref)

That's exactly what steps 1, 3, and 4 do.

>  2 - execute the steps 5,6,7 of the original script
>
>  In the worst case scenario this script will still perform better than the
> current one, because it doesn't have the first 3 steps.
>  With this approach we can create sub-sets of users which will allow the
> script to run until the end without running out of memory (even if this is
> a long process, it will execute)
>
>  Problems:
>  We don't think there's currently a method to retrieve all prefs from the
> backed by its name. Maybe we need to create it, and state that this is for
> admin purposes only and shouldn't be called by the user-level code (just
> like the listAllShares method from Horde_Share_Sql).
>  Currently the pref_name column is not indexed, so we expect slow queries.
> The fix for that is obvious.

The problem is that not all backends support listing preference  
details for others than the current user. On the other hand, those  
backends are already missing some features anyway, and we already have  
a listScopes() method that is only implemented in a few backends too.  
In the end this might be an option.
Actually, since the same script already requires the preference  
backend to return any user's preference, this would be a safe approach.

Another approach would be do take a further look at why  
listAllShares() exceeds memory. Well, there is not much to look  
actually when creating 350.000+ share objects and then attempting to  
sort them. Alternatively we could implement an Iterator interface in  
the share drivers and only receive the shares one by one while looping  
them.

Jan.

-- 
The Horde Project
http://www.horde.org/