[dev] Branches (again), Horde 4.1/5, recent IMAP changes

[Michael J Rubinsky]

Quoting Jan Schneider <jan at horde.org>:

> Zitat von Michael J Rubinsky <mrubinsk at horde.org>:
>
>> Quoting Jan Schneider <jan at horde.org>:
>>
>>> Hi,
>>>
>>> I originally wanted to sum this up after all October releases are  
>>> finished, but the recent IMAP (mailbox) changes made this somewhat  
>>> higher priority.
>>>
>>> Even though there are still some complaints, I think the switch to  
>>> PEAR based releases was a great overall success. With a whopping  
>>> 780 (seven hundred and eighty!) package releases since we started  
>>> the new PEAR server, we brought the paradigm of "release early,  
>>> release often" to new heights, at least for Horde's standards.  
>>> That, and the implicit dependency management makes me never want  
>>> to go back.
>>>
>>> But that's not the only thing we changed for Horde 4, we also  
>>> intended to set new rules for release management  
>>> (http://wiki.horde.org/Doc/Dev/ReleaseCycle) and branch management  
>>> (http://wiki.horde.org/Doc/Dev/Branches). This is the area where  
>>> we still need to improve.
>>>
>>> We are currently in our 2nd release cycle since we released Horde  
>>> 4, and everybody noticed by now that there is no Horde 4.1 in the  
>>> works, let alone Horde 5. The reason is that we simply didn't have  
>>> any new features or larger changes in our stable code base that  
>>> hadn't been released yet anyway. That's why we focused on  
>>> releasing the "missing bits" during this release cycle, i.e. the  
>>> until now unreleased applications that had to be ported to Horde 4.
>>>
>>> The reason for the lack of new features is that we kept adding new  
>>> stuff to the master branch, i.e. to the stable branch, and still  
>>> shuffled things around with new stable releases. I let this  
>>> happen, despite this being against the release/branch rules,  
>>> because I felt like there still were some teething troubles with  
>>> our young Horde 4 release (and Michael even explicitly expressed  
>>> that he considered our stable release too early at one point) We  
>>> also had some important things missing originally, that shouldn't  
>>> had to wait another half a year. But at the same time I was intent  
>>> to get an agreement to more strictly enforce these rules once the  
>>> 2nd release cycle is over.
>>> Because the flip-side of the coin is that (even though some of  
>>> those changes in stable releases were necessary to fix bugs), the  
>>> "stable" releases were much less stable than they should have  
>>> been. Too often some of those larger changes caused intermediate  
>>> regressions. Fortunately this didn't have such a high impact,  
>>> thanks to our new release model (see above). I still think we  
>>> should stop this now though. People should not be prepared to  
>>> experience regressions due to code restructuring (like the mailbox  
>>> encoding in IMP), notable UI changes (like in dynamic Kronolith)  
>>> or new features suddenly popping up (like in - everywhere), while  
>>> they update within a minor version. Such changes need more testing  
>>> through RCs (yeah, I know, they won't be tested properly anyway,  
>>> but that's a different story), and with piling them up for the  
>>> next minor version, we actually *have* some minor version to  
>>> release.
>>
>> While I agree with the importance of keeping master stable, as you  
>> stated, some of those changes were necessary to fix some high  
>> profile bugs. Going forward, what would be the protocol for similar  
>> bug fixes where we really shouldn't wait 6 months to release a fix?  
>> Hopefully master has settled down to a point where we shouldn't  
>> have this type of problem. Given how we just released a handful of  
>> new H4 apps, it's not out of the realm of possibilities that a  
>> similar situation could arise.
>
> Yes, even we if set up some rules, those should not be written in  
> stone, and some flexibility is necessary to react properly in  
> important situations. But those should really only be exceptions  
> from the rule, well justified, with no alternative options, and with  
> side-effects kept as small as possible.
>
>> Part of this could indeed be that we released before the code was  
>> 100% ready, but I have no doubt that we *had* to do this. Had to  
>> finally pick a date and stick to it. Even if we *had* waited, some  
>> of these bugs would still not have been found until we had the  
>> level of use we only get after our code is put into production.
>>
>>> So, I urge everyone to read the wiki pages linked above again. If  
>>> everyone is still fine with that direction, then let's all agree  
>>> to enforce these rules from now on.
>>
>> I'm still ok with this direction overall, but I have some questions  
>> now that we have spent some time with this model.
>>
>> (1) I know that we are to make no BC breaks until Horde 5, and I  
>> also know that we have made some semi-bc breaking changes during  
>> this cycle and simply changed the application's minimum required  
>> version of the dependency. I'm assuming these were unavoidable  
>> changes, but it brings up a broader question in my mind: When is it  
>> ok to require a newer version of some dependency? Only during major  
>> version releases? Minor versions?Never during bug fix releases?
>
> Originally I thought we should not allow to require higher versions  
> of dependencies unless really, really required for serious reasons  
> (security fixes, important bug fixes). But during our last  
> conversation about that topic I got convinced (and that's what the  
> other Michael refers to in his reply) that this strictness is not  
> really necessary. So we should allow to raise a dependency version.  
> But this of course does *not* mean that this dependency is allowed  
> to break BC. I.e. the users must be able to upgrade a dependency  
> without breaking an older version of a module that depends on it.
>
>> (2) How do individual framework packages fit into the stable  
>> branch/dev branch. Since they are now released separately, do we  
>> still stick to the no-new-features-in-master paradigm for these?  
>> Since these have been (up until now, anyway) released on a more  
>> frequent cycle I'm not sure this makes sense. Horde_Foo may have  
>> some new feature, or major bug fix that required the minor number  
>> to be bumped - and I'm not sure I would want to wait the up to 6  
>> months until the next time dev is merged into master for the next  
>> release cycle.
>
> Good question. From a gut feeling I would say that it's fine to add  
> features to framework packages, as long as we keep bumping the minor  
> version, which we already did very disciplined so far. But thinking  
> about it again, I'm not sure why there should be a difference to the  
> applications. Any arguments to rationalize my feelings are welcome. :)

I was thinking mostly because the release cycles are so different. No  
reason to hold back an improvement in a framework package for 6 months  
just because are application release cycle is 6 months. This seems  
like an arbitrary rule to me. If it's stable, we should release it.  
Release early and often, as they say.

While we are talking about this, I have a problem with the branching model ...
I'm confused about what happens once we start releasing minor point  
releases of some applications, while other applications are still on  
the x.0 release. Add to the mix that we have a number of different  
minor version bumps already on master and things get even more  
confusing (at least to me), as to what version should be on what  
branch. What happens when we release e.g., IMP 5.1, but not Ansel 2.1  
yet? How do we merge the changes from IMP into master (since that is  
now the stable code for IMP), while not moving Ansel's 2.1 code to  
master since it is *not* yet stable? Are we going to synchronize the  
minor releases? That's not something we've done in the past, and don't  
think will work well.

-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org