[dev] Git splitting

[Michael J Rubinsky]

Quoting Michael M Slusarz <slusarz at horde.org>:

> Quoting Michael J Rubinsky <mrubinsk at horde.org>:
>
>> Quoting Michael M Slusarz <slusarz at horde.org>:
>>
>>> Quoting Jan Schneider <jan at horde.org>:
>>>
>>>> Zitat von Michael J Rubinsky <mrubinsk at horde.org>:
>>>>
>>>>> Quoting Michael M Slusarz <slusarz at horde.org>:
>>>>>
>>>>>> Quoting Michael M Slusarz <slusarz at horde.org>:
>>>>>>
>>>>>>> The next step will be to determine how we can use PHP composer  
>>>>>>> to allow developers to assemble the disparate repos into a  
>>>>>>> format that we can actually use on a day-to-day basis.  Should  
>>>>>>> not be that difficult - we should only need to replace/alter  
>>>>>>> the install scripts in framework/bin.
>>>>>>
>>>>>> OK.  Verified that we can create composer.json with his library:
>>>>>>
>>>>>> https://github.com/claylo/conductor
>>>>>>
>>>>>> On the command line, it works via:
>>>>>>
>>>>>> # php package2composer.php
>>>>>>
>>>>>> in the same directory as a package.xml file.  One caveat: this  
>>>>>> needs to be added manually to the composer.json file:
>>>>>>
>>>>>> "repositories": [
>>>>>>    {
>>>>>>        "type": "pear",
>>>>>>        "url": "http://pear.horde.org"
>>>>>>    }
>>>>>> ],
>>>>>>
>>>>>> (This can be done via a config file going forward).
>>>>>>
>>>>>> This will install all dependencies for a package using Composer  
>>>>>> (using PEAR packages).
>>>>>>
>>>>>> It seems that Pirum does automatically create compose repos  
>>>>>> when building the PEAR repo, so maybe we can take advantage of  
>>>>>> that in the future.
>>>>>
>>>>> I'm still confused how this will help with people using our  
>>>>> split git repositories. This will work against the released PEAR  
>>>>> packages, right? So, how will we help people manage the various  
>>>>> dependencies when using git? If I'm checking out git to get the  
>>>>> latest and greatest code to work against (or help develop), I'm  
>>>>> also going to want to be able to checkout the git repository of  
>>>>> all of it's dependencies, not the PEAR packages - especially if  
>>>>> we are in development of a new major version where we won't have  
>>>>> any PEAR packages released yet that are compatible.
>>>>
>>>> Composer is capable of building packages from Git.
>>>
>>> This is besides the point anyway.  Above I am discussing how we  
>>> can create composer.json files for all packages as an alternative  
>>> to PEAR installation.  It has nothing to do with git  
>>> reorganization.  This is something that can be done TODAY.
>>
>> Maybe, but the point I was making, and Jan has addressed, is still  
>> valid. Before we can split the repos, we need to ensure we can help  
>> developers manage the dependencies while checking out any of our  
>> git repositories. I guess the context of the email caused me to  
>> misread the point you were making about composer if it did not have  
>> anything to do with splitting the repos.
>
> Yeah, this was what I was talking about in my first email.  Not sure  
> if composer will be able to do this for us or not, or if we have to  
> roll our own.  So I was looking into Composer in general and the  
> idea of auto-creating composer.json files was an offshoot of that.
>
> For the record, it looks like Composer can NOT do something like  
> download all git repos that are dependencies.  It can install from a  
> git repo, but the local copy is not a git repo - it's just a  
> collection of files.
>
> But that's fine.  And that's not really the point of of splitting  
> the repos anyway.  Instead, the idea is that someone wants to fix  
> something in Horde_Imap_Client they only have to download the  
> Horde_Imap_Client repo and make the fix.  composer.json comes in  
> handy if they want to run the unit tests for a particular package.   
> The latter makes it much more realistic that people who have no  
> interest in Horde applications, but want to use some of the  
> libraries, can use the code.
>
> One of the major issues right now, as I see it, is that is is pretty  
> much impossible for ordinary users to easily create patches.  We  
> essentially require them to run a completely separate installation  
> in order to do this.  By changing, we now dramatically lower the bar  
> for development entry.  People can install and run their  
> installations through the normal channels (i.e. PEAR), and then only  
> need to download a tiny subset of Horde (i.e. Horde_Core) when they  
> discover a bug.

Except that then they can't actually test said patch if they are not  
running the code they are patching. Of course they can always play  
with include_path to ensure the git version is included first  
(assuming they can figure out how to install the libraries properly)  
but that just opens up a whole other set of problems with  
dependencies. Not saying we shouldn't split, it's just something else  
to think about. I don't think this transition is going to be as easy  
as you are saying it will be for non-horde developers to make use of  
the split repo.

> Either way, I can't tell you how excited I am to have a split repo.
> Looking at the test repo I created, it is about 1,000x easier to  
> track both the changes, tags, and branches for a particular component.

While I agree it is *easier*, I never found it particularly  
troublesome with the monolithic repo.

> For me, large chunks of version 3 of Horde_Imap_Client have been  
> written already, and switching to this kind of development will  
> actually allow that code to see the light of day -- since it is  
> completely impossible right now to do this with the current setup  
> (creating a topic branch does NOT work for this kind of thing).

I'm curious why it doesn't work. While we were doing the Horde 5.1  
work, we kept a separate branch for each app/library. When I needed to  
test the whole, I just created a local test branch and merged all the  
public topic branches into it. This also allowed me to explicitly  
include/exclude certain application/libraries from the test branch to  
test things like BC with different combinations of package versions.  
Added only about an extra minute to run a shell script to merge the  
changes into the branch. Ideal? No. Workable? Yes.

-- 
mike

The Horde Project (www.horde.org)
mrubinsk at horde.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-keys
Size: 2200 bytes
Desc: PGP Public Key
URL: <http://lists.horde.org/archives/dev/attachments/20130621/109ddb37/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6062 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.horde.org/archives/dev/attachments/20130621/109ddb37/attachment-0001.bin>