[dev] [commits] Horde branch master updated. 3d5433806251d1e14d73fb2f5abf8347cf07389d

[Michael Rubinsky]

Quoting Chuck Hagenbuch <chuck at horde.org>:

> Quoting Jan Schneider <jan at horde.org>:
>
>> Zitat von Gunnar Wrobel <p at rdus.de>:
>>
>>> Quoting Michael M Slusarz <slusarz at horde.org>:
>>>
>>>
>>>> -----------------------------------------------------------------------
>>>>
>>>> commit 21d7218dc143d6f728c6d2f833e08c2c5e833cce
>>>> Author: Michael M Slusarz <slusarz at curecanti.org>
>>>> Date:   Fri Sep 10 22:38:37 2010 -0600
>>>>
>>>>  Use PHPUnit mock function, rather than static mock files
>>>
>>> Why do you prefer the mocks here? I do admit that the stubs I  
>>> wrote were really simple and replacing them with mocks probably  
>>> makes sense in the context of this single test.
>>>
>>> But in my experience test mocks just don't work well in the long  
>>> term. For one thing they are extremely hard to read. I believe  
>>> some people recently wrote alternative mock implementations for  
>>> PHPUnit that solve this problem at least. But I know from  
>>> experience that all these mock expectations are hard to maintain.  
>>> They end up all over the test suites and if you are changing  
>>> anything about the mocked interface it gets hard to adapt all  
>>> those calls. In order to understand the mock expectations you  
>>> often need to look into the corresponding code to see why some of  
>>> these are actually needed. So mocking will often tie your test to  
>>> the actual implementation.
>>>
>>> As the Itip test was the first application level unit test I added  
>>> for Horde the stubs were still residing within IMP. The plan would  
>>> have been to move these into the "Test" package at some point and  
>>> extending the stubs into a general replacement for "the real  
>>> thing" when writing application level unit tests. So you would  
>>> have a stub registry implementation that would allow access to and  
>>> manipulation of all the features provided by the real registry.
>>>
>>> This approach also has some drawbacks:
>>>
>>> For one thing these are not really unit tests anymore but tend to  
>>> be integration tests (without any access to external systems  
>>> though). But to me this makes sense for the application level  
>>> where it seems reasonable to define a test that reads "user clicks  
>>> a, user clicks b, event got saved to storage".
>>>
>>> Another thing is the complexity of the stubs: Once these get  
>>> closer to the real thing it is harder to actually change them as  
>>> they may easily affect hundreds of tests. But on the other hand  
>>> this can actually be quite useful: Horde_Registry is rather  
>>> central so it also cannot be easily changed. If it needs  
>>> modification though you would adapt the stub implementation, too.  
>>> The fact that this stub is used in many places where the registry  
>>> is required should help in finding spots that still need to be  
>>> adapted to the change in the real registry.
>>>
>>> Last but not least these tests usually turn out to run a lot  
>>> slower as more elements are involved in testing. Not good but hard  
>>> to change. And something I tend to accept for the benefit I see in  
>>> basing our application level testing on stubs.
>>>
>>> In summary may not see much of a problem with the one commit I  
>>> commented here. For me the main question is how others view this  
>>> topic and which style of testing they'd prefer. I have seen the  
>>> mock based approach fail horribly in the past half year and that  
>>> is why this issue is important to me.
>>
>> Even though I have barely worked with mocks yet, my experience in  
>> those case haven't been good either. Each time I ended up debugging  
>> the generated mocks, which is a major pita.
>> So, with my limited experience with mock functionality, I tend to  
>> agree with Gunnar.
>
> My limited experience lines up this way also - I know some of my  
> coworkers like mocks and have gotten to the point of "grokking"  
> them, but I find them really dense and it's hard to predict/debug  
> their behavior.


FWIW, I've had similar experiences. My experience with PHPUnit mocks  
is also limited, mostly in the ActiveSync tests, but I found them very  
hard to work with and found that the majority of my test failures were  
due to incorrect mock definitions. I was also spending more time  
debugging the mocks than I was spending on any other part of the  
testing process.  I finally got what I wanted, but I feel they are  
very fragile. Unless we really need some functionality they provide,  
such as specifying the number of times a method should be called  
etc..., IMHO, we should stick to using stubs, except maybe in the most  
simplistic cases.

-- 
Mike

--
The Horde Project (www.horde.org)
mrubinsk at horde.org

Mike

?The only reason for time is so that everything doesn't happen at  
once." - A. Einstein