[dev] [commits] Horde branch master updated. 3d5433806251d1e14d73fb2f5abf8347cf07389d

[Michael M Slusarz]

Quoting Chuck Hagenbuch <chuck at horde.org>:

> Quoting Jan Schneider <jan at horde.org>:
>
>> Zitat von Gunnar Wrobel <p at rdus.de>:
>>
>>> Quoting Michael M Slusarz <slusarz at horde.org>:
>>>
>>>
>>>> -----------------------------------------------------------------------
>>>>
>>>> commit 21d7218dc143d6f728c6d2f833e08c2c5e833cce
>>>> Author: Michael M Slusarz <slusarz at curecanti.org>
>>>> Date:   Fri Sep 10 22:38:37 2010 -0600
>>>>
>>>>  Use PHPUnit mock function, rather than static mock files
>>>
>>> Why do you prefer the mocks here? I do admit that the stubs I  
>>> wrote were really simple and replacing them with mocks probably  
>>> makes sense in the context of this single test.
>>>
>>> But in my experience test mocks just don't work well in the long  
>>> term. For one thing they are extremely hard to read. I believe  
>>> some people recently wrote alternative mock implementations for  
>>> PHPUnit that solve this problem at least. But I know from  
>>> experience that all these mock expectations are hard to maintain.  
>>> They end up all over the test suites and if you are changing  
>>> anything about the mocked interface it gets hard to adapt all  
>>> those calls. In order to understand the mock expectations you  
>>> often need to look into the corresponding code to see why some of  
>>> these are actually needed. So mocking will often tie your test to  
>>> the actual implementation.
>>>
>>> As the Itip test was the first application level unit test I added  
>>> for Horde the stubs were still residing within IMP. The plan would  
>>> have been to move these into the "Test" package at some point and  
>>> extending the stubs into a general replacement for "the real  
>>> thing" when writing application level unit tests. So you would  
>>> have a stub registry implementation that would allow access to and  
>>> manipulation of all the features provided by the real registry.
>>>
>>> This approach also has some drawbacks:
>>>
>>> For one thing these are not really unit tests anymore but tend to  
>>> be integration tests (without any access to external systems  
>>> though). But to me this makes sense for the application level  
>>> where it seems reasonable to define a test that reads "user clicks  
>>> a, user clicks b, event got saved to storage".
>>>
>>> Another thing is the complexity of the stubs: Once these get  
>>> closer to the real thing it is harder to actually change them as  
>>> they may easily affect hundreds of tests. But on the other hand  
>>> this can actually be quite useful: Horde_Registry is rather  
>>> central so it also cannot be easily changed. If it needs  
>>> modification though you would adapt the stub implementation, too.  
>>> The fact that this stub is used in many places where the registry  
>>> is required should help in finding spots that still need to be  
>>> adapted to the change in the real registry.
>>>
>>> Last but not least these tests usually turn out to run a lot  
>>> slower as more elements are involved in testing. Not good but hard  
>>> to change. And something I tend to accept for the benefit I see in  
>>> basing our application level testing on stubs.
>>>
>>> In summary may not see much of a problem with the one commit I  
>>> commented here. For me the main question is how others view this  
>>> topic and which style of testing they'd prefer. I have seen the  
>>> mock based approach fail horribly in the past half year and that  
>>> is why this issue is important to me.
>>
>> Even though I have barely worked with mocks yet, my experience in  
>> those case haven't been good either. Each time I ended up debugging  
>> the generated mocks, which is a major pita.
>> So, with my limited experience with mock functionality, I tend to  
>> agree with Gunnar.
>
> My limited experience lines up this way also - I know some of my  
> coworkers like mocks and have gotten to the point of "grokking"  
> them, but I find them really dense and it's hard to predict/debug  
> their behavior.

My rationale for switching when finishing implementing the IMP tests:
1. IMHO, I would rather have all test requirements contained in a  
single file.  Then, when creating/debugging tests, I don't have to  
switch between 2 or 3 windows to try to figure out where I need to fix  
something.
2. Overhead of having to create a separate file, copying/modifying  
phpdoc info, adding file to git and possibly adding entry to  
package.xml files, and then figuring out location to add requires to a  
file to load the stub.
3. The IMP tests were broken due to a change in a framework library  
that had no bearing on the functionality being tested.  This is a big  
one for me - tests can not fail simply because the stub definitions  
are not up to date.  Constant maintenance of stubs is not acceptable.
4. Small side benefit: using mocks identified a problem with the  
framework Itip library.  Namely, there were several Identity methods  
that were called that didn't exist in the base framework lib - they  
were only defined in IMP.  This was identified by using Mocks.

Just some ideas to further the discussion.

michael

-- 
___________________________________
Michael Slusarz [slusarz at horde.org]