[dev] Branches (again), Horde 4.1/5, recent IMAP changes

[Michael J Rubinsky]

Quoting Jan Schneider <jan at horde.org>:

> Hi,
>
> I originally wanted to sum this up after all October releases are  
> finished, but the recent IMAP (mailbox) changes made this somewhat  
> higher priority.
>
> Even though there are still some complaints, I think the switch to  
> PEAR based releases was a great overall success. With a whopping 780  
> (seven hundred and eighty!) package releases since we started the  
> new PEAR server, we brought the paradigm of "release early, release  
> often" to new heights, at least for Horde's standards. That, and the  
> implicit dependency management makes me never want to go back.
>
> But that's not the only thing we changed for Horde 4, we also  
> intended to set new rules for release management  
> (http://wiki.horde.org/Doc/Dev/ReleaseCycle) and branch management  
> (http://wiki.horde.org/Doc/Dev/Branches). This is the area where we  
> still need to improve.
>
> We are currently in our 2nd release cycle since we released Horde 4,  
> and everybody noticed by now that there is no Horde 4.1 in the  
> works, let alone Horde 5. The reason is that we simply didn't have  
> any new features or larger changes in our stable code base that  
> hadn't been released yet anyway. That's why we focused on releasing  
> the "missing bits" during this release cycle, i.e. the until now  
> unreleased applications that had to be ported to Horde 4.
>
> The reason for the lack of new features is that we kept adding new  
> stuff to the master branch, i.e. to the stable branch, and still  
> shuffled things around with new stable releases. I let this happen,  
> despite this being against the release/branch rules, because I felt  
> like there still were some teething troubles with our young Horde 4  
> release (and Michael even explicitly expressed that he considered  
> our stable release too early at one point) We also had some  
> important things missing originally, that shouldn't had to wait  
> another half a year. But at the same time I was intent to get an  
> agreement to more strictly enforce these rules once the 2nd release  
> cycle is over.
> Because the flip-side of the coin is that (even though some of those  
> changes in stable releases were necessary to fix bugs), the "stable"  
> releases were much less stable than they should have been. Too often  
> some of those larger changes caused intermediate regressions.  
> Fortunately this didn't have such a high impact, thanks to our new  
> release model (see above). I still think we should stop this now  
> though. People should not be prepared to experience regressions due  
> to code restructuring (like the mailbox encoding in IMP), notable UI  
> changes (like in dynamic Kronolith) or new features suddenly popping  
> up (like in - everywhere), while they update within a minor version.  
> Such changes need more testing through RCs (yeah, I know, they won't  
> be tested properly anyway, but that's a different story), and with  
> piling them up for the next minor version, we actually *have* some  
> minor version to release.

While I agree with the importance of keeping master stable, as you  
stated, some of those changes were necessary to fix some high profile  
bugs. Going forward, what would be the protocol for similar bug fixes  
where we really shouldn't wait 6 months to release a fix? Hopefully  
master has settled down to a point where we shouldn't have this type  
of problem. Given how we just released a handful of new H4 apps, it's  
not out of the realm of possibilities that a similar situation could  
arise.

Part of this could indeed be that we released before the code was 100%  
ready, but I have no doubt that we *had* to do this. Had to finally  
pick a date and stick to it. Even if we *had* waited, some of these  
bugs would still not have been found until we had the level of use we  
only get after our code is put into production.

> So, I urge everyone to read the wiki pages linked above again. If  
> everyone is still fine with that direction, then let's all agree to  
> enforce these rules from now on.

I'm still ok with this direction overall, but I have some questions  
now that we have spent some time with this model.

(1) I know that we are to make no BC breaks until Horde 5, and I also  
know that we have made some semi-bc breaking changes during this cycle  
and simply changed the application's minimum required version of the  
dependency. I'm assuming these were unavoidable changes, but it brings  
up a broader question in my mind: When is it ok to require a newer  
version of some dependency? Only during major version releases? Minor  
versions?Never during bug fix releases?

(2) How do individual framework packages fit into the stable  
branch/dev branch. Since they are now released separately, do we still  
stick to the no-new-features-in-master paradigm for these? Since these  
have been (up until now, anyway) released on a more frequent cycle I'm  
not sure this makes sense. Horde_Foo may have some new feature, or  
major bug fix that required the minor number to be bumped - and I'm  
not sure I would want to wait the up to 6 months until the next time  
dev is merged into master for the next release cycle.

Similarly, what about *new* packages? Real world example, I started  
working on a Service_Weather package to replace the now mostly useless  
Services_Weather_Weatherdotcom(?). I really don't want to wait for  
another 6 months before it can be released, so I started it in master.  
This library would be used by both Horde (for a new weather block) and  
by Timeobjects (to feed Kronolith with weather data). Whether this can  
go in before the next minor/major release is another question, and I  
guess it depends on how big a deal failure of the existing weather  
data is. My vote would be before the next minor release. Yes, it's a  
"new" feature, but it's replacing an existing (now broken feature).

I've already hijacked this thread with some not-quite-related stuff,  
so we should also probably start a separate discussion about deciding  
on a Horde 5 vs 4.1 release and start a H5 BC breaking changes wiki  
page to give us a better idea as to what we have so far.

> Back to my original impetus for this write-up. I wanted to restart  
> this discussion after the current release cycle because 1) we would  
> have half a year starting now to pile up new features/changes for a  
> potential Horde 4.1 (or even Horde 5) release next April, which  
> would match the originally planned release cycle length, and 2) I  
> felt like our commits had settled a bit during the last few weeks  
> (though this could also be caused by a stronger focus on the  
> unreleased applications).
> Thus the recent Horde_Imap_Client_Mailbox changes make me very  
> strong headaches. This is exactly the kind of changes that has  
> potential of causing more trouble than solving problems (for a bug  
> fix release, that is). It introduces new dependencies, touches large  
> parts of important code in IMP, and even converts existing user  
> preferences. This is unacceptable for a simple bug fix release IMO.  
> I *strongly* suggest to revert this on master, and re-apply it to  
> the develop branch, so that it gets more and longer testing and  
> won't be released before a IMP 5.1 RC.


-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org