[dev] improved unit testing (was: [commits] Horde branch master updated. f174f08002872dab2019bcca58175749f598bb6a)

[Thomas Jarosch]

On Wednesday, 5. March 2014 14:25:30 Michael M Slusarz wrote:
> > if it helps, we are currently developing a test suite
> > based upon autotest[1].
> > ...
> > 
>
> > > Sounds really cool, although it appears to be more particular to
> testing the actual UI.
> 
> For framework libraries, PHPUnit is fine.  It's just structuring the
> code to allow for reasonable testing.

I just thought we could use autotest to set up virtual machines f.e. that 
run cyrus imapd / dovecot / whatever to actually test the IMAP library
with a real server. The tests would still be executed via PHPUnit.


The way we currently use autotest is to install a distribution in a virtal 
machine, install the test suite on the guest and take a LVM snapshot.

Then we boot the VM, run a test, collect the result and kill the
VM and LVM snapshot. After that, take another snapshot of the original, 
clean machine and run the next test. Repeat.

It's a bit slower than cleaning up a continuously running machine during 
each test, but it also ensures everything is in a defined state.
Though we also have tests that perform internal cleanup if the changes
are only "small". LVM snapshots are fast anyway as the VM
is installed on a ramdisk (using sparse files for the HDD image).

We are still working on making this "framework" more generic,
so it can be used with different distributions and send
some of this stuff upstream. So I guess it could become
interesting for the horde project at a later point in time.

I just wanted to mention we are working on this so
it does not appear like a submarine out of the blue :)

Cheers,
Thomas