[imp] Spam not reporting

Thu Jan 5 06:31:49 UTC 2017

>>>>>> I have spam reporting enabled.  What I discovered is that  
>>>>>> messages sent through this method are not actually being  
>>>>>> submitted correctly.  I also found through manual testing, that  
>>>>>> the parameter '-C revoke' is not always successful.  So I  
>>>>>> changed the program line for spamc to use '-L spam' and '-L  
>>>>>> forget' respectively.  Here's how I tested.
>>>>>>
>>>>>> In Horde and logged in as user1, find a message in Inbox to  
>>>>>> route through the Spam reporting process.  Select it and choose  
>>>>>> Other Options->View Source.  Copy all the source into a new  
>>>>>> file named email.eml on the mail server system.
>>>>>>
>>>>>> Log into the mail server system as user1.
>>>>>> Using spamc, manually report the email as spam to the system to  
>>>>>> understand the intended behavior.
>>>>>> $ spamc -d localhost -L spam -s 1500000 -u user1 < email.eml
>>>>>>
>>>>>> Responds with 
>>>>>> Message successfully un/learned
>>>>>>
>>>>>> Run the same command a second time.
>>>>>> $ spamc -d localhost -L spam -s 1500000 -u user1 < email.eml
>>>>>>
>>>>>> Responds with
>>>>>> Message was already un/learned
>>>>>>
>>>>>> Remove the email from spamassassin with 
>>>>>> $ spamc -d localhost -L forget -s 1500000 -u user1 < email.eml
>>>>>>
>>>>>> Responds with
>>>>>> Message successfully un/learned
>>>>>>
>>>>>> Next, let's report the same email using Horde Spam Reporting  
>>>>>> then manually test that it was learned.  
>>>>>> In Horde, choose the same message from the Inbox and use the  
>>>>>> Spam reporting method.  Horde handles the spam reporting  
>>>>>> correctly and the messsage is moved to an alternative folder as  
>>>>>> intended.  This works fine.
>>>>>>
>>>>>> Let's return to the email server system and manually check that  
>>>>>> the email was submitted correctly.
>>>>>>
>>>>>> Rerun spamc with the email.eml file.  This should respond with  
>>>>>> a meesage that this was already learned.
>>>>>>
>>>>>> $ spamc -d localhost -L spam -s 1500000 -u user1 < email.eml
>>>>>>
>>>>>> Responds with 
>>>>>> Message successfully un/learned
>>>>>>
>>>>>> It does NOT!  Instead this message has now been learned  
>>>>>> manually.  Horde did not report this correctly.
>>>>>>
>>>>>> Here is my backends.local.php file for reference.
>>>>>>
>>>>>> <?php
>>>>>> $servers['imap']['disabled'] = true;
>>>>>> $servers['advanced']['disabled'] = false;
>>>>>> $servers['advanced']['name'] = 'Advanced IMAP Server';
>>>>>> $servers['advanced']['hostspec'] = 'localhost';
>>>>>> $servers['advanced']['port'] = 143;
>>>>>> $servers['advanced']['maildomain'] = 'domain.net';
>>>>>> $servers['advanced']['spam']['innocent']['display'] = true;
>>>>>> $servers['advanced']['spam']['innocent']['program'] =  
>>>>>> '/usr/bin/spamc -s 1500000 -d localhost -L forget -u %l';
>>>>>> $servers['advanced']['spam']['spam']['display'] = true;
>>>>>> $servers['advanced']['spam']['spam']['program'] =  
>>>>>> '/usr/bin/spamc -s 1500000 -d localhost -L spam -u %l';
>>>>>> $servers['advanced']['special_mboxes']['IMP_Mailbox::MBOX_DRAFTS'] =  
>>>>>> 'Drafts';
>>>>>> $servers['advanced']['special_mboxes']['IMP_Mailbox::MBOX_SENT'] =  
>>>>>> 'Sent';
>>>>>> $servers['advanced']['special_mboxes']['IMP_Mailbox::MBOX_SPAM'] =  
>>>>>> 'Spam';
>>>>>> $servers['advanced']['special_mboxes']['IMP_Mailbox::MBOX_TEMPLATES'] =  
>>>>>> 'Templates';
>>>>>> $servers['advanced']['special_mboxes']['IMP_Mailbox::MBOX_TRASH'] =  
>>>>>> 'Trash';
>>>>>> $servers['advanced']['autocreate_special'] = true;
>>>>>>
>>>>>> Looking at the apache status, it does show that spamc process  
>>>>>> was kicked off, but it does not appear that any intended email  
>>>>>> files are actually being submitted, unless this is just how the  
>>>>>> log lines appear.
>>>>>>
>>>>>> # /etc/init.d/apache2 status 
>>>>>> ● apache2.service - LSB: Apache2 web server
>>>>>>    Loaded: loaded (/etc/init.d/apache2)
>>>>>>   Drop-In: /lib/systemd/system/apache2.service.d
>>>>>>            └─forking.conf
>>>>>>    Active: active (running) since Thu 2016-12-22 01:54:09 PST;  
>>>>>> 5 days ago
>>>>>>   Process: 25714 ExecStop=/etc/init.d/apache2 stop  
>>>>>> (code=exited, status=0/SUCCESS)
>>>>>>   Process: 24796 ExecReload=/etc/init.d/apache2 reload  
>>>>>> (code=exited, status=0/SUCCESS)
>>>>>>   Process: 25739 ExecStart=/etc/init.d/apache2 start  
>>>>>> (code=exited, status=0/SUCCESS)
>>>>>>    CGroup: /system.slice/apache2.service
>>>>>>            ├─  417 /usr/sbin/apache2 -k start
>>>>>>            ├─  431 /usr/sbin/apache2 -k start
>>>>>>            ├─  432 /usr/sbin/apache2 -k start
>>>>>>            ├─  448 /usr/sbin/apache2 -k start
>>>>>>            ├─  449 /usr/sbin/apache2 -k start
>>>>>>            ├─  452 /usr/sbin/apache2 -k start
>>>>>>            ├─  454 /usr/sbin/apache2 -k start
>>>>>>            ├─25018 /usr/sbin/apache2 -k start
>>>>>>            ├─25753 /usr/sbin/apache2 -k start
>>>>>>            ├─25757 /usr/sbin/apache2 -k start
>>>>>>            ├─25758 /usr/sbin/apache2 -k start
>>>>>>            ├─25843 /usr/sbin/apache2 -k start
>>>>>>            ├─25846 /usr/sbin/apache2 -k start
>>>>>>            ├─25847 /usr/sbin/apache2 -k start
>>>>>>            ├─26512 /usr/sbin/apache2 -k start
>>>>>>            ├─27332 /usr/sbin/apache2 -k start
>>>>>>            ├─27334 /usr/sbin/apache2 -k start
>>>>>>            ├─27403 sh -c /usr/bin/spamc -s 1500000 -d localhost  
>>>>>> -L spam -u 'user1'
>>>>>>            ├─27404 /usr/bin/spamc -s 1500000 -d localhost -L  
>>>>>> spam -u user1
>>>>>>            ├─28056 /usr/sbin/apache2 -k start
>>>>>>            ├─31845 /usr/sbin/apache2 -k start
>>>>>>            ├─31987 /usr/sbin/apache2 -k start
>>>>>>            └─32190 /usr/sbin/apache2 -k start
>>>>>>  
>>>>>
>>>>> Hi dev list.  Is there any other details I can add to this  
>>>>> report that could be helpful to solve this? 
>>>>>
>>>>> Thank you,
>>>>> Art
>>>>> --
>>>>
>>>> I added the line
>>>> $servers['debug'] = '/tmp/horde-debug.log';
>>>> to imp/config/backends.local.php
>>>> in hopes to get some additional details to append here.  The log  
>>>> file is not created.
>>>
>>> You are missing the server key. I.e.,
>>>
>>> $servers['advanced']['debug'] = '/tmp/imap-debug.log';
>
>>> Though I'm not sure this will show anything about your spam problem.
>>>
>>> Are you sure that your webserver user has the correct permissions  
>>> to report spam to spamc?
>>
>> Additionally, are you sure you are running the spamd daemon and not  
>> using Amavis? The later won't work with spamc.
>
> I changed /etc/passwd and allowed www-data a login shell.  I used  
> `su - www-data` to change to this user in which the web server runs  
> as.  As the www-data user, I ran the commands
>
> `spamc -d localhost -L spam -s 1500000 -u user1 < test.eml`  
>        as well as 
> `spamc -d localhost -L forget -s 1500000 -u user1 < test.eml`  
>
> The reported behavior above was expected and this was the same  
> behavior that was seen when running as www-data.  This test was  
> successful and it validates that the www-data user can run spamc.  
>  In /etc/apache2/envvars is the env variable `export  
> APACHE_RUN_USER=www-data`
>
> netstat lists
> tcp        0      0 localhost:spamd         *:*                     LISTEN
>
> /etc/services shows the spamd port as
> spamd 783/tcp # spamassassin daemon
>
> I do have amavis installed and have set a $sa_spam_subject_tag = **JUNK**
> spamassassin local,cf config has this
> rewrite_header Subject ***** SPAM *****
>
> All auto reported Spam has the Subject that is used in the  
> spamassassin local.cf file.
>  

Enabling debug logging with $servers['advanced']['debug'] =  
'/tmp/imap-debug.log'; did not provide any usable insight.