[dev] assertion failed: mailbox.c: 2180: !message_guid_isnull(&record->guid)

[Jan Schneider]

Zitat von Michael M Slusarz <slusarz at horde.org>:

> Quoting Jan Schneider <jan at horde.org>:
>
>> Zitat von Jan Schneider <jan at horde.org>:
>>
>>> Zitat von Bron Gondwana <brong at fastmail.fm>:
>>>
>>>> On Tue, Nov 20, 2012, at 07:15 PM, Aleksander Machniak wrote:
>>>>> On 11/20/2012 04:07 PM, steffo76 at gmx.de wrote:
>>>>>>> 1353423361>* BYE Fatal error: Internal error: assertion  
>>>>>>> failed: mailbox.c: 2180: !message_guid_isnull(&record->guid)^M
>>>>>
>>>>> I created a ticket for this yesterday.
>>>>> https://bugzilla.cyrusimap.org/show_bug.cgi?id=3755. It looks like a bug
>>>>> with APPEND + BINARY implementation.
>>>>
>>>> I'll have a look soon, promise!  I'd like to get a fix into 2.4.17
>>>> and release it super-soon.  There are a couple of people who have
>>>> seen it recently.  Did some client switch to using BINARY APPEND?
>>>
>>> The Horde_Imap_Client library did.
>>>
>>>> Definitely a bug!
>>>>
>>>> I'm in the process of moving country this week, so I'm only online
>>>> sporadically...
>>>>
>>>> Bron.
>>>
>>
>> Bron just fixed this in Cyrus, but since this affects major  
>> functionality (saving sent-mail and drafts) with a version of a  
>> major IMAP server that is shipped with major distributions, can we  
>> workaround this bug? It looks like it is triggered with BINARY  
>> APPEND only.
>
> F***.  This is not something that can be easily worked around  
> without totally destroying performance.  Here's why:
>
> 1. Previous scanning for null's in data could be accomplished by  
> doing a strpos().  This is fast, but requires the full message data  
> to be stored in PHP memory (which is bad as it can cause memory  
> fatal errors).
> 2. To fix memory usage, data needs to be stored in streams.
> 3. Now, to search for nulls, we use a stream filter  
> (Horde_Imap_Client_Data_Format_Filter_String).  This stream filter  
> unfortunately has fairly deplorable performance.
> Example: To import a 20MB mbox file in IMP, it would take 30-40  
> seconds to process.
> 4. Since #2 can't be reversed, which means going back to #1 is not  
> an option, a workaround (implemented in Horde_Imap_Client 2.2.0) is  
> to skip null scanning of the input data and simply output the data  
> as-is.  The reasoning:
>   - if BINARY is available, we know the server can handle null data  
> so we can just activate binary literals and output the raw data.
>   - if BINARY is not available, there's nothing we can do if the  
> data contains a null character since there is no workaround. So we  
> just output to the server and hope for the best (it is possible that  
> some servers could handle that data, or at least ignore the nulls)
>
> These optimizations reduce the time required to import a 20MB mbox  
> file by a factor of 10.
>
> I think the current method of working around this is still the  
> correct method (try with BINARY; re-do if the APPEND fails).  I see  
> that you implemented a hotfix  
> (1220a7ed6888eb61d29d8e1228d216dac3e2bb97), but this hotfix is too  
> broad.  NO is a perfectly valid (and possibly expected) response  
> code from APPEND so we can't lump that in with BAD/BYE responses.

Okay, then all we need to do is to retry if the server response  
explicitly with either BAD or BYE. So far we only did this on BAD  
responses.
-- 
Jan Schneider
The Horde Project
http://www.horde.org/