[horde] Server side email searches with ActiveSync not working

Thu Apr 26 00:44:27 UTC 2018

Quoting Sebastian Arcus <s.arcus at open-t.co.uk>:

> On 18/04/18 09:08, Sebastian Arcus wrote:
>>
>> On 17/04/18 14:53, Sebastian Arcus wrote:
>>>
>>> On 17/04/18 13:16, Ad-Min wrote:
>>>> Sebastian Arcus-2 wrote
>>>>> I am struggling to get server side email searches working with  
>>>>> ActiveSync.
>>>>>
>>>>> My setup is:
>>>>>
>>>>> Lighttpd 1.4.49
>>>>> Dovecot 2.2.33
>>>>> Exim 4.90
>>>>> PHP 5.6.35
>>>>> Postgresql 9.2.24
>>>>> Horde 5.2.17
>>>>>
>>>>> I have tested imap searches through Thunderbird (there is an option "run
>>>>> search on server) and it is working fine and fast. I assume this means
>>>>> Dovecot is configured correctly and can do imap searches?
>>>>>
>>>>> I have tested the Horde email searches on an iPhone 5S with iOS 11.2.6
>>>>> and a Jolla phone with Sailfish 2.1. Both of them have email configured
>>>>> in ActiveSync mode. On both of them, if I force them to search on the
>>>>> server only, they sit there saying "searching" for a few minutes, and
>>>>> then return no results (and no error either).
>>>>>
>>>>> On the server side, I can see the command being sent by the device to
>>>>> Horde in the individual device ActiveSync log
>>
>> </snip>
>>
>>>>>
>>>>> But I just can't be sure it is actually passed to Dovecot - at least I
>>>>> can't see anything in the Dovecot logs to indicate that.
>>>>>
>>>>> Are there any other settings needed for ActiveSync server side email
>>>>> searches to work? Can I troubleshoot this further, or look in any other
>>>>> logs?
>>>>>
>>>>> Any advice much appreciated.
>>>
>>>>
>>>>
>>>>
>>>> I had a similar issue (windows, hmailserver as backend). My  
>>>> problem was not
>>>> that it doesn't work, but rather that clients timed out searching  
>>>> very large
>>>> mailboxes. I was not able to fix this, but it works fine on smaller
>>>> mailboxes. You didn't mention anything about mailbox size, so I'm just
>>>> throwing this out there.
>>>>
>>>> I looked at my imap logs on hmailserver and confirmed that the search was
>>>> actually talking place. The issue was the search took about 4 minutes to
>>>> complete and the clients (all that I tried) timed out after 60 or 90
>>>> seconds.
>>>>
>>>> I would start by enabling imap logging in your mail server and looking to
>>>> see if the search is actually talking place. Likely it is (since that is a
>>>> working function of horde activesync), so maybe you can track  
>>>> down the error
>>>> from there.
>>>
>>> Thank you for answering and the suggestions. In the meanwhile I  
>>> have spent a number of hours last night investigating. One of the  
>>> possibilities I considered is large mailboxes which would mean  
>>> long search time. I logged over telnet to Dovecot and manually ran  
>>> a SEARCH BODY imap command. This indeed took over 5 minutes to  
>>> return results.
>>>
>>> Then I configured Lucene FTS indexing plugin in Dovecot - and  
>>> after indexing the mailbox, the search started over the telnet  
>>> session went down to less then 1 second! Still, no joy over  
>>> ActiveSync.
>>>
>>> Then I went and enabled debugging in Dovecot to the highest level  
>>> of verbosity. Still, there is no sign of SEARCH commands being  
>>> sent from Horde to Dovecot. I'm not really sure what is happening,  
>>> but the ActiveSync server-side email search seems to be broken  
>>> somehow. I have posted a bug report at Horde.org - maybe someone  
>>> will get a chance to look into this.
>>
>> OK - I think I finally figured out the mystery. My IMAP server in  
>> horde/imp/config/backends.local.php was using an old syntax:
>>
>> $servers['secure-imap'] = array(
>>
>> This syntax is not even present any more in the current  
>> backends.php - so I guess it must have been defaulting to the  
>> simple IMAP server - which, again, I am guessing that it doesn't  
>> support server side email searches.
>>
>> I should have read the upgrade DOCS a lot more closely - but it  
>> doesn't help at all that there are no warning or error messages in  
>> the Horde log when an ActiveSync device initiates server-side email  
>> searches, but the back-end doesn't support it (or at least I didn't  
>> find any errors in the logs, even in debug mode).
>>
>> Now everything is working fine and server-side searches return  
>> results correctly. I thought I would post here, in case anyone else  
>> stumbles over the same problem.
>
> I am sorry to report that this problem has reared its head again.  
> The server-side email searches through ActiveSync are being so  
> unpredictable to the point that 1 in 20 might work.

This works for me without issue. Sometimes it *is* a bit slow, but I  
get consistent results. A search for "activesync" issues the following:

C: 8 UID SORT RETURN (ALL COUNT) (REVERSE ARRIVAL) UTF-8 TEXT activesync
>> Slow Command: 15.069 seconds
S: * ESEARCH (TAG "8") UID ALL  
386679,386678,386670,386660,386659,386638,386529,386528,386522,386519,385900,385856,385850,385824,385820,385812,385328,383433,383348,383312,383310,372739 COUNT  
22
S: 8 OK Sort completed (0.234 + 0.001 + 0.001 secs).
>> Command 8 took 0.275 seconds.

> My Dovecot server has Lucene full text indexing enabled, and if a  
> run a BODY search through Thunderbird, passing the search to the  
> server over IMAP, the results come back in 2 seconds, on an Inbox  
> with 20 000 emails.

The ActiveSync log you posted earlier shows that you are searching ALL  
mailboxes, not just your INBOX.

> The same search executed from a phone through ActiveSync doesn't  
> return any results.
>
> I've turned up debugging all the way up in Dovecot, and when  
> searching from the phone, all I see is just a bunch of EXAMINE imap  
> commands - no SEARCH command. I don't know if Horde is supposed to  
> use SEARCH imap command, or has a different way of doing server side  
> email searches?

Depending on your server and CAPABILITIES, you might be using the IMAP  
SORT extension (RFC-5256).

> Any thoughts on this would be gratefully appreciated.
> -- 
> Horde mailing list
> Frequently Asked Questions: http://horde.org/faq/
> To unsubscribe, mail: horde-unsubscribe at lists.horde.org

-- 
mike
The Horde Project
http://www.horde.org
https://www.facebook.com/hordeproject
https://www.twitter.com/hordeproject
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-keys
Size: 9272 bytes
Desc: PGP Public Key
URL: <https://lists.horde.org/archives/horde/attachments/20180426/390e58d6/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 821 bytes
Desc: PGP Digital Signature
URL: <https://lists.horde.org/archives/horde/attachments/20180426/390e58d6/attachment-0003.bin>