[dev] Branches (again), Horde 4.1/5, recent IMAP changes

[Michael J Rubinsky]

Quoting Jan Schneider <jan at horde.org>:

> Zitat von Michael J Rubinsky <mrubinsk at horde.org>:
>
>> Quoting Jan Schneider <jan at horde.org>:
>>
>>> Zitat von Michael J Rubinsky <mrubinsk at horde.org>:
>>>
>>>> Quoting Jan Schneider <jan at horde.org>:
>>>>
>>>>> Hi,
>>>>>
>>>>> I originally wanted to sum this up after all October releases  
>>>>> are finished, but the recent IMAP (mailbox) changes made this  
>>>>> somewhat higher priority.
>>>>>
>>>>> Even though there are still some complaints, I think the switch  
>>>>> to PEAR based releases was a great overall success. With a  
>>>>> whopping 780 (seven hundred and eighty!) package releases since  
>>>>> we started the new PEAR server, we brought the paradigm of  
>>>>> "release early, release often" to new heights, at least for  
>>>>> Horde's standards. That, and the implicit dependency management  
>>>>> makes me never want to go back.
>>>>>
>>>>> But that's not the only thing we changed for Horde 4, we also  
>>>>> intended to set new rules for release management  
>>>>> (http://wiki.horde.org/Doc/Dev/ReleaseCycle) and branch  
>>>>> management (http://wiki.horde.org/Doc/Dev/Branches). This is the  
>>>>> area where we still need to improve.
>>>>>
>>>>> We are currently in our 2nd release cycle since we released  
>>>>> Horde 4, and everybody noticed by now that there is no Horde 4.1  
>>>>> in the works, let alone Horde 5. The reason is that we simply  
>>>>> didn't have any new features or larger changes in our stable  
>>>>> code base that hadn't been released yet anyway. That's why we  
>>>>> focused on releasing the "missing bits" during this release  
>>>>> cycle, i.e. the until now unreleased applications that had to be  
>>>>> ported to Horde 4.
>>>>>
>>>>> The reason for the lack of new features is that we kept adding  
>>>>> new stuff to the master branch, i.e. to the stable branch, and  
>>>>> still shuffled things around with new stable releases. I let  
>>>>> this happen, despite this being against the release/branch  
>>>>> rules, because I felt like there still were some teething  
>>>>> troubles with our young Horde 4 release (and Michael even  
>>>>> explicitly expressed that he considered our stable release too  
>>>>> early at one point) We also had some important things missing  
>>>>> originally, that shouldn't had to wait another half a year. But  
>>>>> at the same time I was intent to get an agreement to more  
>>>>> strictly enforce these rules once the 2nd release cycle is over.
>>>>> Because the flip-side of the coin is that (even though some of  
>>>>> those changes in stable releases were necessary to fix bugs),  
>>>>> the "stable" releases were much less stable than they should  
>>>>> have been. Too often some of those larger changes caused  
>>>>> intermediate regressions. Fortunately this didn't have such a  
>>>>> high impact, thanks to our new release model (see above). I  
>>>>> still think we should stop this now though. People should not be  
>>>>> prepared to experience regressions due to code restructuring  
>>>>> (like the mailbox encoding in IMP), notable UI changes (like in  
>>>>> dynamic Kronolith) or new features suddenly popping up (like in  
>>>>> - everywhere), while they update within a minor version. Such  
>>>>> changes need more testing through RCs (yeah, I know, they won't  
>>>>> be tested properly anyway, but that's a different story), and  
>>>>> with piling them up for the next minor version, we actually  
>>>>> *have* some minor version to release.
>>>>
>>>> While I agree with the importance of keeping master stable, as  
>>>> you stated, some of those changes were necessary to fix some high  
>>>> profile bugs. Going forward, what would be the protocol for  
>>>> similar bug fixes where we really shouldn't wait 6 months to  
>>>> release a fix? Hopefully master has settled down to a point where  
>>>> we shouldn't have this type of problem. Given how we just  
>>>> released a handful of new H4 apps, it's not out of the realm of  
>>>> possibilities that a similar situation could arise.
>>>
>>> Yes, even we if set up some rules, those should not be written in  
>>> stone, and some flexibility is necessary to react properly in  
>>> important situations. But those should really only be exceptions  
>>> from the rule, well justified, with no alternative options, and  
>>> with side-effects kept as small as possible.
>>>
>>>> Part of this could indeed be that we released before the code was  
>>>> 100% ready, but I have no doubt that we *had* to do this. Had to  
>>>> finally pick a date and stick to it. Even if we *had* waited,  
>>>> some of these bugs would still not have been found until we had  
>>>> the level of use we only get after our code is put into production.
>>>>
>>>>> So, I urge everyone to read the wiki pages linked above again.  
>>>>> If everyone is still fine with that direction, then let's all  
>>>>> agree to enforce these rules from now on.
>>>>
>>>> I'm still ok with this direction overall, but I have some  
>>>> questions now that we have spent some time with this model.
>>>>
>>>> (1) I know that we are to make no BC breaks until Horde 5, and I  
>>>> also know that we have made some semi-bc breaking changes during  
>>>> this cycle and simply changed the application's minimum required  
>>>> version of the dependency. I'm assuming these were unavoidable  
>>>> changes, but it brings up a broader question in my mind: When is  
>>>> it ok to require a newer version of some dependency? Only during  
>>>> major version releases? Minor versions?Never during bug fix  
>>>> releases?
>>>
>>> Originally I thought we should not allow to require higher  
>>> versions of dependencies unless really, really required for  
>>> serious reasons (security fixes, important bug fixes). But during  
>>> our last conversation about that topic I got convinced (and that's  
>>> what the other Michael refers to in his reply) that this  
>>> strictness is not really necessary. So we should allow to raise a  
>>> dependency version. But this of course does *not* mean that this  
>>> dependency is allowed to break BC. I.e. the users must be able to  
>>> upgrade a dependency without breaking an older version of a module  
>>> that depends on it.
>>>
>>>> (2) How do individual framework packages fit into the stable  
>>>> branch/dev branch. Since they are now released separately, do we  
>>>> still stick to the no-new-features-in-master paradigm for these?  
>>>> Since these have been (up until now, anyway) released on a more  
>>>> frequent cycle I'm not sure this makes sense. Horde_Foo may have  
>>>> some new feature, or major bug fix that required the minor number  
>>>> to be bumped - and I'm not sure I would want to wait the up to 6  
>>>> months until the next time dev is merged into master for the next  
>>>> release cycle.
>>>
>>> Good question. From a gut feeling I would say that it's fine to  
>>> add features to framework packages, as long as we keep bumping the  
>>> minor version, which we already did very disciplined so far. But  
>>> thinking about it again, I'm not sure why there should be a  
>>> difference to the applications. Any arguments to rationalize my  
>>> feelings are welcome. :)
>>
>> I was thinking mostly because the release cycles are so different.  
>> No reason to hold back an improvement in a framework package for 6  
>> months just because are application release cycle is 6 months. This  
>> seems like an arbitrary rule to me. If it's stable, we should  
>> release it. Release early and often, as they say.
>>
>> While we are talking about this, I have a problem with the  
>> branching model ...
>> I'm confused about what happens once we start releasing minor point  
>> releases of some applications, while other applications are still  
>> on the x.0 release. Add to the mix that we have a number of  
>> different minor version bumps already on master and things get even  
>> more confusing (at least to me), as to what version should be on  
>> what branch. What happens when we release e.g., IMP 5.1, but not  
>> Ansel 2.1 yet? How do we merge the changes from IMP into master  
>> (since that is now the stable code for IMP), while not moving  
>> Ansel's 2.1 code to master since it is *not* yet stable? Are we  
>> going to synchronize the minor releases? That's not something we've  
>> done in the past, and don't think will work well.
>
> But that's exactly what we decided to do in the release plan:  
> synchronized releases. Of course this would be much easier if we  
> didn't have a single repository for all applications. And I can see  
> how this is going to cause problems when we don't manage to get all  
> applications ready for a new release in the same time.
> Of course we don't necessarily have to release a new minor version  
> of each application if there simply aren't any new features.

Exactly. These are the two cases I'm thinking of. I doubt we will both  
(1) Always have a minor version upgrade for every application at the  
same time and (2) Even if we did, I doubt they would be ready at the  
same time...at least given our track record.

I guess given our monolithic repository, no matter how we decide to do  
it, there are going to be some issues like this to work around.

-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org