[imp] Odd LDAP replication behavior on filters attribute

[Scott Courtney]

I've got a customer using Imp in production with about 35000 users, and things
are generally working quite well.

We just found a bug (?) in Imp's handling of the "Blacklist" link from the
inbox display page, though. It turns out that there is no limit on how many
filter rules can be created by users. In this situation, we have users who
select hundreds of spam messages and then blacklist them all. We have a couple
of users who have over 1500 filter rules in Imp's configuration!

The good news is that Imp actually works -- albeit slowly -- in these extreme
cases.

The bad news is that our distributed LDAP directory chokes when trying to
replicate this particular attribute value (in the replog, it ends up being
about 3400 lines of base64 encoded data). The Horde LDAP schema defines the
impPrefs attribute as being 1024 characters max. OpenLDAP seems not to enforce
that limit, and the huge filter rules go into the master LDAP server just
fine. But the slurpd replication process fails to transfer this attribute on
these overflow cases. Apparently it has an upper limit somewhere around 160K
bytes (that's an estimate).

I've looked at the Imp code (running version 3.2.1), and it seems that the
best thing to do would be to add a test in mailbox.php, in the logic for the
IMP_BLACKLIST action-id case, to ensure that the total number of existing
rules isn't over some reasonable upper limit. I know that doesn't cover the
situation of them adding rules one-by-one the old fashioned way, but one
hopes that nobody would have the patience to add 1500 rules that way.

It would be trivial to look at how many rules are in a single "Blacklist"
transaction, but that doesn't solve the problem of users adding hundreds and
hundreds of rules a few at a time. With 35000+ users at an ISP, we are
already seeing cases of people doing that sort of thing. A million monkeys
at a million keyboards......you know. :-)

The questions I have, then are:

1. Has this already been addressed by someone else? Or does someone else have
   a better idea? I don't want to reinvent a wheel.

2. What should that "reasonable upper limit" be? Should it be a constant
   in the config file, perhaps? Suggestions for the default value? Clearly,
   no webmail system was ever intended for every user to have over 1000
   personalized filtering rules. Anti-spam filtering should be done elsewhere
   (and indeed, this company uses RBLs and SpamAssassin already, so users
   shouldn't need to do much, but you know how it goes).

Comments welcome.

Kind regards,

Scott

-- 
-----------------------+------------------------------------------------------
Scott Courtney         | "I don't mind Microsoft making money. I mind them
courtney at 4th.com       | having a bad operating system."    -- Linus Torvalds
http://4th.com/        | ("The Rebel Code," NY Times, 21 February 1999)
                       | PGP Public Key at http://4th.com/keys/courtney.pubkey