[dev] Fix kronolith-agenda script

Gonçalo Queirós goncalo.queiros at portugalmail.net
Wed Mar 14 18:31:26 UTC 2012


Hi there dev.

  We were trying to use the kronolith-agenda script to send daily agendas to
everyone on our service, but the problem is that we currently have more
than 350.000 shares and the script just runs out of memory, even with 2Gb!

  Looking at the script more closely, we think there's a way to make the
script work, regardless of the number of shares on the system, but we would
like your opinion on that before coming out with a patch.

  Current script:
  1 - Get every calendar share
  2 - For every share, list its events, to check if it has any event to the
current day
  3 - For the remaining list get all users that have access to the calendars
  4 - For every user, check if he desires to receive the daily agenda (pref)
  5 - For every user that wants to receive the daily agenda, get his
calendars
  6 - Again for every calendar get the ones that have any event to the
current day
  7 - Send the email if there's any calendar left

  For our installation the current script stops on the first step, because
it runs out of memory.
  We thought on creating sub-sets for the shares, but the problem is that we
only know the full agenda of a user after we analyze all shares he has
access to.

  What we propose:
  1 - Get every users that desire to receive the daily agenda (pref)
  2 - execute the steps 5,6,7 of the original script

  In the worst case scenario this script will still perform better than the
current one, because it doesn't have the first 3 steps.
  With this approach we can create sub-sets of users which will allow the
script to run until the end without running out of memory (even if this is
a long process, it will execute)

  Problems:
  We don't think there's currently a method to retrieve all prefs from the
backed by its name. Maybe we need to create it, and state that this is for
admin purposes only and shouldn't be called by the user-level code (just
like the listAllShares method from Horde_Share_Sql).
  Currently the pref_name column is not indexed, so we expect slow queries.
The fix for that is obvious.

  What do you think?

  Thanks,
  Gonçalo Queirós


More information about the dev mailing list