[dev] Fix kronolith-agenda script

[Gonçalo Queirós]

Ping? :-)


On 03/14/2012 06:31 PM, Gonçalo Queirós wrote:
> Hi there dev.
>
>  We were trying to use the kronolith-agenda script to send daily 
> agendas to
> everyone on our service, but the problem is that we currently have more
> than 350.000 shares and the script just runs out of memory, even with 
> 2Gb!
>
>  Looking at the script more closely, we think there's a way to make the
> script work, regardless of the number of shares on the system, but we 
> would
> like your opinion on that before coming out with a patch.
>
>  Current script:
>  1 - Get every calendar share
>  2 - For every share, list its events, to check if it has any event to 
> the
> current day
>  3 - For the remaining list get all users that have access to the 
> calendars
>  4 - For every user, check if he desires to receive the daily agenda 
> (pref)
>  5 - For every user that wants to receive the daily agenda, get his
> calendars
>  6 - Again for every calendar get the ones that have any event to the
> current day
>  7 - Send the email if there's any calendar left
>
>  For our installation the current script stops on the first step, because
> it runs out of memory.
>  We thought on creating sub-sets for the shares, but the problem is 
> that we
> only know the full agenda of a user after we analyze all shares he has
> access to.
>
>  What we propose:
>  1 - Get every users that desire to receive the daily agenda (pref)
>  2 - execute the steps 5,6,7 of the original script
>
>  In the worst case scenario this script will still perform better than 
> the
> current one, because it doesn't have the first 3 steps.
>  With this approach we can create sub-sets of users which will allow the
> script to run until the end without running out of memory (even if 
> this is
> a long process, it will execute)
>
>  Problems:
>  We don't think there's currently a method to retrieve all prefs from the
> backed by its name. Maybe we need to create it, and state that this is 
> for
> admin purposes only and shouldn't be called by the user-level code (just
> like the listAllShares method from Horde_Share_Sql).
>  Currently the pref_name column is not indexed, so we expect slow 
> queries.
> The fix for that is obvious.
>
>  What do you think?
>
>  Thanks,
>  Gonçalo Queirós