[dev] Horde_Imap_Client_Base::vanished issues

Sat Mar 3 07:59:08 UTC 2012

I've noticed an issue with performing a vanished search. It first  
happened after the recent API changes were pulled, but I can't  
definitively say the problem did not exist before this.

The issue is, that sometimes calling $imap->vanished($mbox,  
$lastKnownModSeq) seems to return ALL UIDs that have ever vanished  
from the mailbox. I first noticed this while working on syncing email  
via ActiveSync and the server started trying to send over 100k DELETE  
commands to the client. I've run the following test code in Horde's  
phpshell:

$imap = $registry->mail->imapOb();
$mbox = new Horde_Imap_Client_Mailbox('INBOX');
// Output so I can track the values as I go.
$status = $imap->status($mbox,
     Horde_Imap_Client::STATUS_HIGHESTMODSEQ |
     Horde_Imap_Client::STATUS_UIDVALIDITY |
     Horde_Imap_Client::STATUS_UIDNEXT);
var_dump($status);

// 445126 is last HIGHTESTMODSEQ value output above.
var_dump($imap->vanished($mbox, 445126));

At some point, the VANISHED response stops showing the UIDs since the  
MODSEQ and just returns ALL of them. I can't figure out what causes  
this to happen. Any pointers/ideas?

The IMAP log of one instance when all UIDs were returned. I believe  
there is also another fetch going on here as well:

>> Timestamp: Sat, 03 Mar 2012 02:43:40 -0500
S: * OK [CAPABILITY IMAP4rev1 LITERAL+ LOGIN-REFERRALS ID ENABLE   
XIMAPPROXY] Dovecot ready.
C: [LOGIN Command - username: mike]
S: * OK [XPROXYREUSE] IMAP connection reused by imapproxy
S: 1 OK User logged in
C: 2 CAPABILITY
S: * CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE  
SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT  
IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE  
QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS
S: 2 OK Capability completed.
C: 3 ENABLE QRESYNC
S: * ENABLED QRESYNC
S: 3 OK Enabled.
C: 4 NAMESPACE
S: * NAMESPACE (("" "/")) NIL NIL
S: 4 OK Namespace completed.
C: 4 NAMESPACE
S: * NAMESPACE (("" "/")) NIL NIL
S: 4 OK Namespace completed.
C: 5 STATUS INBOX (UIDNEXT UIDVALIDITY HIGHESTMODSEQ)
S: * STATUS "INBOX" (UIDNEXT 103674 UIDVALIDITY 1292457115  
HIGHESTMODSEQ 445390)
S: 5 OK Status completed.
C: 6 EXAMINE INBOX (QRESYNC (1292457115 445390  
88312,89464,89506,92040,92403,94207,94653,96579,97547,98895,99471,99627,99633,99635:99636,99853,100111,100156,101342,101430,101562,101568,102041,102044,102147,102153,102166,102210,102224,102905,103042,103048,103105,103135,103140:103143,103181,103185,103189,103196,103198,103206,103208:103209,103212,103214,103220:103221,103223,103227:103228,103230,103233:103234,103237,103240,103242,103246,103266,103269,103274:103275,103277:103278,103285,103331,103339,103343,103380,103382:103383,103391,103394,103398,103448,103450,103453,103456:103457,103463:103464,103468,103477,103479,103489:103490,103492:103493,103501,103509,103511,103515,103521,103523:103525,103528,103533,103535:103536,103538,103543,103546,103549,103552,103555,103585,103628,103642,103666:103667,103671))
S: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $Junk $NotJunk  
$Forwarded Test_Tso test_one JunkRecorded)
S: * OK [PERMANENTFLAGS ()] Read-only mailbox.
S: * 114 EXISTS
S: * 0 RECENT
S: * OK [UNSEEN 112] First unseen.
S: * OK [UIDVALIDITY 1292457115] UIDs valid
S: * OK [UIDNEXT 103674] Predicted next UID
S: * OK [HIGHESTMODSEQ 445390] Highest
S: 6 OK [READ-ONLY] Select completed.
C: 7 UID SEARCH RETURN (ALL COUNT) ALL
C: 5 LOGOUT
S: * ESEARCH (TAG "7") UID ALL  
88312,89464,89506,92040,92403,94207,94653,96579,97547,98895,99471,99627,99633,99635:99636,99853,100111,100156,101342,101430,101562,101568,102041,102044,102147,102153,102166,102210,102224,102905,103042,103048,103105,103135,103140:103143,103181,103185,103189,103196,103198,103206,103208:103209,103212,103214,103220:103221,103223,103227:103228,103230,103233:103234,103237,103240,103242,103246,103266,103269,103274:103275,103277:103278,103285,103331,103339,103343,103380,103382:103383,103391,103394,103398,103448,103450,103453,103456:103457,103463:103464,103468,103477,103479,103489:103490,103492:103493,103501,103509,103511,103515,103521,103523:103525,103528,103533,103535:103536,103538,103543,103546,103549,103552,103555,103585,103628,103642,103666:103667,103671 COUNT  
114
S: 7 OK Search completed (0.000 secs).
>> Saved search results to cache (mailbox: INBOX; id:  
>> e711e5dc23f3f88a379cdd1ed363c2ed)
C: 8 UID FETCH 1:* () (CHANGEDSINCE 445330)
S: * BYE LOGOUT received
S: 5 OK Completed
S: * 114 FETCH (UID 103671 MODSEQ (445335))
S: 8 OK Fetch completed.
>> CACHE: Deleted messages (mailbox: INBOX; UIDs: 103661:103665)
>> CACHE: Retrieved messages (mailbox: INBOX; UIDs:  
>> 88312,89464,89506,92040,92403,94207,94653,96579,97547,98895,99471,99627,99633,99635:99636,99853,100111,100156,101342,101430,101562,101568,102041,102044,102147,102153,102166,102210,102224,102905,103042,103048,103105,103135,103140:103143,103181,103185,103189,103196,103198,103206,103208:103209,103212,103214,103220:103221,103223,103227:103228,103230,103233:103234,103237,103240,103242,103246,103266,103269,103274:103275,103277:103278,103285,103331,103339,103343,103380,103382:103383,103391,103394,103398,103448,103450,103453,103456:103457,103463:103464,103468,103477,103479,103489:103490,103492:103493,103501,103509,103511,103515,103521,103523:103525,103528,103533,103535:103536,103538,103543,103546,103549,103552,103555,103585,103628,103642,103666:103667,103671)
C: 9 UID FETCH 103671 () (CHANGEDSINCE 445330)
S: * 114 FETCH (UID 103671 MODSEQ (445335))
S: 9 OK Fetch completed.
>> CACHE: Stored messages (mailbox: INBOX; UIDs: 103671)
C: 10 UID FETCH 1:* () (VANISHED CHANGEDSINCE 445330)
S: * VANISHED (EARLIER)  
1:88311,88313:89463,89465:89505,89507:92039,92041:92402,92404:94206,94208:94652,94654:96578,96580:97546,97548:98894,98896:99470,99472:99626,99628:99632,99634,99637:99852,99854:100110,100112:100155,100157:101341,101343:101429,101431:101561,101563:101567,101569:102040,102042:102043,102045:102146,102148:102152,102154:102165,102167:102209,102211:102223,102225:102904,102906:103041,103043:103047,103049:103104,103106:103134,103136:103139,103144:103180,103182:103184,103186:103188,103190:103195,103197,103199:103205,103207,103210:103211,103213,103215:103219,103222,103224:103226,103229,103231:103232,103235:103236,103238:103239,103241,103243:103245,103247:103265,103267:103268,103270:103273,103276,103279:103284,103286:103330,103332:103338,103340:103342,103344:103379,103381,103384:103390,103392:103393,103395:103397,103399:103447,103449,103451:103452,103454:103455,103458:103462,103465:103467,103469:103476,103478,103480:103488,103491,103494:103500,103502:103508,103510,103512:103514,103516:10
 3520,103
522,103526:103527,103529:103532,103534,103537,103539:103542,103544:103545,103547:103548,103550:103551,103553:103554,103556:103584,103586:103627,103629:103641,103643:103665,103668:103670,103672:103673
S: * 114 FETCH (UID 103671 MODSEQ (445335))
S: 10 OK Fetch completed.

-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6096 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.horde.org/archives/dev/attachments/20120303/48a27f50/attachment.bin>