[horde] Server side email searches with ActiveSync not working

[Sebastian Arcus]

On 18/04/18 09:08, Sebastian Arcus wrote:
> 
> On 17/04/18 14:53, Sebastian Arcus wrote:
>>
>> On 17/04/18 13:16, Ad-Min wrote:
>>> Sebastian Arcus-2 wrote
>>>> I am struggling to get server side email searches working with 
>>>> ActiveSync.
>>>>
>>>> My setup is:
>>>>
>>>> Lighttpd 1.4.49
>>>> Dovecot 2.2.33
>>>> Exim 4.90
>>>> PHP 5.6.35
>>>> Postgresql 9.2.24
>>>> Horde 5.2.17
>>>>
>>>> I have tested imap searches through Thunderbird (there is an option 
>>>> "run
>>>> search on server) and it is working fine and fast. I assume this means
>>>> Dovecot is configured correctly and can do imap searches?
>>>>
>>>> I have tested the Horde email searches on an iPhone 5S with iOS 11.2.6
>>>> and a Jolla phone with Sailfish 2.1. Both of them have email configured
>>>> in ActiveSync mode. On both of them, if I force them to search on the
>>>> server only, they sit there saying "searching" for a few minutes, and
>>>> then return no results (and no error either).
>>>>
>>>> On the server side, I can see the command being sent by the device to
>>>> Horde in the individual device ActiveSync log 
> 
> </snip>
> 
>>>>
>>>> But I just can't be sure it is actually passed to Dovecot - at least I
>>>> can't see anything in the Dovecot logs to indicate that.
>>>>
>>>> Are there any other settings needed for ActiveSync server side email
>>>> searches to work? Can I troubleshoot this further, or look in any other
>>>> logs?
>>>>
>>>> Any advice much appreciated.
>>
>>>
>>>
>>>
>>> I had a similar issue (windows, hmailserver as backend). My problem 
>>> was not
>>> that it doesn't work, but rather that clients timed out searching 
>>> very large
>>> mailboxes. I was not able to fix this, but it works fine on smaller
>>> mailboxes. You didn't mention anything about mailbox size, so I'm just
>>> throwing this out there.
>>>
>>> I looked at my imap logs on hmailserver and confirmed that the search 
>>> was
>>> actually talking place. The issue was the search took about 4 minutes to
>>> complete and the clients (all that I tried) timed out after 60 or 90
>>> seconds.
>>>
>>> I would start by enabling imap logging in your mail server and 
>>> looking to
>>> see if the search is actually talking place. Likely it is (since that 
>>> is a
>>> working function of horde activesync), so maybe you can track down 
>>> the error
>>> from there.
>>
>> Thank you for answering and the suggestions. In the meanwhile I have 
>> spent a number of hours last night investigating. One of the 
>> possibilities I considered is large mailboxes which would mean long 
>> search time. I logged over telnet to Dovecot and manually ran a SEARCH 
>> BODY imap command. This indeed took over 5 minutes to return results.
>>
>> Then I configured Lucene FTS indexing plugin in Dovecot - and after 
>> indexing the mailbox, the search started over the telnet session went 
>> down to less then 1 second! Still, no joy over ActiveSync.
>>
>> Then I went and enabled debugging in Dovecot to the highest level of 
>> verbosity. Still, there is no sign of SEARCH commands being sent from 
>> Horde to Dovecot. I'm not really sure what is happening, but the 
>> ActiveSync server-side email search seems to be broken somehow. I have 
>> posted a bug report at Horde.org - maybe someone will get a chance to 
>> look into this.
> 
> OK - I think I finally figured out the mystery. My IMAP server in 
> horde/imp/config/backends.local.php was using an old syntax:
> 
> $servers['secure-imap'] = array(
> 
> This syntax is not even present any more in the current backends.php - 
> so I guess it must have been defaulting to the simple IMAP server - 
> which, again, I am guessing that it doesn't support server side email 
> searches.
> 
> I should have read the upgrade DOCS a lot more closely - but it doesn't 
> help at all that there are no warning or error messages in the Horde log 
> when an ActiveSync device initiates server-side email searches, but the 
> back-end doesn't support it (or at least I didn't find any errors in the 
> logs, even in debug mode).
> 
> Now everything is working fine and server-side searches return results 
> correctly. I thought I would post here, in case anyone else stumbles 
> over the same problem.

I am sorry to report that this problem has reared its head again. The 
server-side email searches through ActiveSync are being so unpredictable 
to the point that 1 in 20 might work.

My Dovecot server has Lucene full text indexing enabled, and if a run a 
BODY search through Thunderbird, passing the search to the server over 
IMAP, the results come back in 2 seconds, on an Inbox with 20 000 emails.

The same search executed from a phone through ActiveSync doesn't return 
any results.

I've turned up debugging all the way up in Dovecot, and when searching 
from the phone, all I see is just a bunch of EXAMINE imap commands - no 
SEARCH command. I don't know if Horde is supposed to use SEARCH imap 
command, or has a different way of doing server side email searches?

Any thoughts on this would be gratefully appreciated.