[dev] Horde 5?

[Gunnar Wrobel]

Zitat von Michael J Rubinsky <mrubinsk at horde.org>:

> Quoting Vilius ?umskas <vilius at lnk.lt>:
>
>> Sveiki,
>>
>> Tuesday, February 28, 2012, 10:32:02 PM, you wrote:
>>
>>
>>> Zitat von Vilius ?umskas <vilius at lnk.lt>:
>>
>>>> Hi,
>>>>
>>>> Tuesday, February 28, 2012, 9:20:52 PM, you wrote:
>>>>
>>>>>>>> but personally me would
>>>>>>>> be  against spliting repository for every application or/and framework
>>>>>>>> library.  Usually  I  do  only  minor bug fixes, it would be a pain to
>>>>>>>> keep  50 or more repositories up to date in development environment,
>>>>>>>> because all Horde components are interconnected and you cannot develop
>>>>>>>> and test if one of them is outdated.
>>>>>>
>>>>>>> While I, too, am somewhat opposed to splitting the repo, I have to
>>>>>>> disagree with some of your reasoning. We are striving to make our
>>>>>>> components atomic. In fact, most of our framework libraries ARE
>>>>>>> actually usable as stand-alone components, and do not require a
>>>>>>> traditional horde install to work. That is one of the arguments for
>>>>>>> splitting the framework libraries - to make them appear more atomic
>>>>>>> and to relieve a developer from the chore of having to install the
>>>>>>> whole horde stack if he/she wants to help develop the code for a
>>>>>>> single library.
>>>>>>
>>>>>> Yes,  most of the libraries doesn't require a whole horde install, but
>>>>>> still  they  need  other libraries usually. At least Horde_Autoloader,
>>>>>> Horde_Exception,  Horde_Translation  and a few others. I don't see how
>>>>>> splitting  the  repository  would help here. Quite the opposite. Let's
>>>>>> say I want to work on Horde_View. For splitted repository I would have
>>>>>> to  actually know the dependency list and grab repositories one by one
>>>>>> for 5-7 libraries.
>>>>>>
>>>>>> Or  let's say I would want to work on a new Horde application.
>>>>>> Again, I would have
>>>>>> to  actually  know  all  the  features  I  want  to  implement  in new
>>>>>> application  in  advance,  clone  all needed libraries one by one, and
>>>>>> update them later during development.
>>>>>>
>>>>>> At  least  for  me  cloning one repository is a lot easier. Especially
>>>>>> when  git  is so great with big repositories and considering speed of
>>>>>> the internet nowaways.
>>>>
>>>>> This is a question of having the right tools for the job. Given the
>>>>> toolset we currently use you are of course right. That wouldn't make
>>>>> any sense and is way to cumbersome.
>>>>
>>>>> But other PHP frameworks are doing exactly the same thing: they start
>>>>> to split their software into components. This makes reuse a lot
>>>>> easier. Developers can choose the best parts from the frameworks they
>>>>> like. And having clearly delineated interfaces between your modules
>>>>> help the development in general.
>>>>
>>>>> One of the results from Symfony 2 is the "composer" tool: It is
>>>>> specifically designed for this "many-git-repositories" situation. They
>>>>> try to use it as package manager in general and I'm still skeptical if
>>>>> that will really work out. But for the development situation it may
>>>>> actually help.
>>>>
>>>> Even  with  improved  toolset  and  as  a  developer I don't buy this.
>>>> Library  dependencies and repository structure are two different things
>>>> for  me.  Having  one big repository doesn't prevent you from choosing
>>>> "best parts from the framework".
>>
>>> It does not prevent it but it makes it more difficult. With a "repo
>>> per component" strategy you can follow exactly the code you are
>>> interested in via the commit stream and will not need to try to
>>> isolate the interesting bits from the noise of the whole batch of
>>> components you are not interested in. This is something I still miss
>>> from the CVS days where the commit mail immediately told me if I want
>>> to take a close look or if it is just something I'll peek into if I
>>> have the time. Pull requests also go to exactly one component, testing
>>> runs with just this one component, and in the end all components are
>>> treated somewhat equally - it does not matter wether the path starts
>>> with "framework/" or not.
>>
>> Yes, I miss path in the commit email too. However I feel like it could
>> be  reparsed  from the email itself back to the subject with the right
>> piece of code.
>
> Not with the way the repository is currently set up, no. We've  
> discussed this at length when we first moved to git.Consider a  
> single push that has commits that touch multiple packages. A single  
> push (which generates a single email) != a single commit. Which  
> module would you want to include in the email subject if a single  
> push includes commits to e.g., 10 different packages? Truncating  
> package names is just as bad as not including them as you would  
> still need to look at the email to know what packages the commit  
> touches.
>
>
>> Pulling  and  testing  is  not really important to me, because I'm not
>> core developer (see below).
>>
>>> I definitely won't say that you can't work with a monolithic
>>> repository. We are doing it every day and it works just fine. But
>>> every time I hit something of the things I outlined above - I have the
>>> feeling that is not the correct model. We already do 100 components
>>> and these would better fit into 100 repos.
>>
>>>>
>>>> I  feel  like  emulating  package  manager  is too much "overhead" for
>>>> development.
>>
>>> No. Absolutely not. Package management is the key element and one of
>>> the reasons why I work with Horde. It is the one central element that
>>> I believe many, many web apps didn't get right so far. Why are
>>> distributions successful after all? You don't install Debian from a
>>> single tar archive and slap it onto your hard drive. The flexibility
>>> you get from a distribution is one of the key factors that make them
>>> attractive.
>>
>>> Yes, web applications are a different kind of beast. But everyone is
>>> moving into the cloud now and all these web application are getting
>>> more complex by the minute. I know installing Horde is not trivial -
>>> but updating is - at least since Horde 4.0. And I consider that
>>> extremely important. It allows us to have a fast paced development, it
>>> allows us to push releases quickly, it allows the admins to update
>>> immediately, to keep their systems secure. This is mandatory. There
>>> are Horde 3 installations of large universities out there that display
>>> "Copyright 1999-2006". Package management is what allows to update
>>> quickly while working with an extremely complex and flexible software.
>>> We don't care which apps you install - the update will work anyway. Of
>>> course PEAR is a crappy package manager - but it is at least something
>>> and I hope time will give us better tools.
>>
>> I  completely  agree  with everything you say here, however I was
>> talking  about  package  manager  (say  "components")  for development
>> environment.  IMHO  making  development environment harder to set just
>> pushes  other possible contributor away from Horde, just as much as  
>> difficult
>> APIs or unreadable code  does. Especially when we want someone to  
>> reuse our code.
>>>> Keep  in  mind  that  not  every developer uses Unix. I
>>>> personally  develop  on  Windows  with  TortoiseGUI/Editplus IDE.  I
>>>> would have to
>>>> specifically   set   my  tools to use "components" somehow, because in
>>>> splitted  repository "components" would become a requirement.
>>
>>> For every one of the core developers it is already a requirement.
>>> PHPUnit is a requirement for any Horde developer as well. These are
>>> PHP tools, they can be easily installed on any system that supports
>>> PHP. Granted - we still lack decent docs that would make that easy.
>>> But of course there are requirements to any development environment.
>>> And if I look at how these requirements have evolved during the time I
>>> have been PHP developer - there has been quite some changes since my
>>> initial days. PHPUnit, PHP Code Sniffer, Pdepend, PEAR, PHP documentor
>>>  and probably other stuff I can't think of at the moment.
>>
>> Well,  there  are  two types of developers:   core   Horde  developers
>> and  external  contributors  or
>> developers  who  just  reuses Horde code. I  think  we  would agree to
>> disagree  here  that  for  that  other (half?:)) it would be a pain to
>> install  and  learn  all  bunch  of  tools  they don't really need.  
>> As I said
>> earlier  I've  always  used  simple master/develop setup with 100% GUI
>> tools   on   top  of  it.  This allowed me to bugfix some Horde bugs I
>> was most concerned about and commit them easly,  also  keep  my   
>> in-house code in sync.
>> I  don't  really  run  tests  or  use  components for anything  
>> other than generate
>> package.xml.   To  tell  the  truth  I  don't  even run PHP on one of
>> my computers used for development, but maybe it's just me :)
>
> I have to agree here as well.

As I said in my other mail - me too :)

> None of the arguments I've heard yet for splitting the repository  
> are very convincing to me, including the argument of having a split  
> repository making it more attrative for developers. As Vilius said,  
> there is a difference between a developer using our component in one  
> of his (non-horde) projects and a developer that is actively  
> developing the horde component and submitting/commiting changes to  
> that component. The former *can* use only the components they want -  
> via PEAR - and all the dependencies are taken care of.

That depends on what the external developer is trying to do.

I think I already provided that example before: If I combine my brief  
experience with the z-push code that I had quite a while back with how  
I judge your coding style then I conclude that the Horde_ActiveSync  
library you wrote is far more attractive for other projects to extend  
than relying on z-push. At least assuming that Horde_ActiveSync gains  
the final few features that it still lacks in comparison to the z-push  
code. But chances for this are high.

At the moment there is of course no other project using  
Horde_ActiveSync. I'm not going to pretend that this will happen once  
we split our repo :) The most important point would be documenting the  
library and explaining where the extension points are.

But a splitted repo would help as well. I don't expect that any  
non-Horde project trying to adopt that package could do so right away.  
It is not so uncommon that you find minor problems and flaws with a  
library that has not yet been used by many people outside of the  
original context. In that case you definitely want a git checkout and  
not some PEAR package to rely on.

I picked a single package for this example - even one that might be  
dear to you to try to be convincing ;) - but honestly: I think many of  
our packages would have that problem. We code the stuff for the Horde  
context. Yes, we improved tremendously since the Horde 3 days. And  
there is actually a lot in our components that would be ripe for  
consumption in other projects.

Actually I did so myself: I did integrate several Horde 4 components  
with the Zend 1 framework. This works just fine. But of course you  
loose the version control context if you do so. Which is a pity in the  
long run.

> It's the latter that we are concerned about here. Sure, we can  
> benefit from attracting more developers that are willing to donate  
> time to our project, but I don't think that requiring them to clone  
> a monolithic repository is asking too much.

Try to bridge Horde with Zend framework 2 and Symfony 2 in the context  
of a larger coding project. Pulling a monolihic repository will not be  
an option. Working based on PEAR will mean that no patches will be  
contributed back. Or at least it will be harder.

> ..and might even lead to them wanting to contribute to more  
> components than the original one.
>
> I think it would be more work to ask them to figure out the needed  
> dependencies any single compononet would require and  
> download/install each of those. Sure, we could put the energy/time  
> into leveraging existing tools or implementing new tools to make  
> this somewhat easier for other developers looking to contribute to  
> Horde, but why? At some point I would think it would just become  
> easier to:
>
> git clone horde
> horde/framework/bin/install_dev
>
> in order to get everything...and who knows, they might even discover  
> components they were not aware of because they now have the full  
> checkout.
>
> Will we discourage some casual developers that might be interested  
> in contributing? I don't think so...the libraries are solid, well  
> thought out code - arguably the best in our core areas. That is what  
> should attract developers, not the repository structure.

The fact that the code is there and structurally sound is probably not  
enough to attract more devs. The repository structure is also  
insignificant. The main factor is documentation and publishing how to  
use the components.

I know I write way too much text in this thread for the fact that I  
don't see the repo structure as one of the major issues at the moment.  
It would help more if I'd write docs instead :) But I'm on the other  
hand happy that we are having this discussion as all because this has  
been rejected outright before.

And I'm still convinced it makes sense to split at some later point in  
time simply because I failed integration Horde code with other  
components outside of Horde context. Which does not make me happy :)

> Even so, Horde is non-trivial software and anyone writing code that  
> will be included in our code base should have no trouble using our  
> existing toolset/repository structure.

Not with the toolset but certainly with the repository structure. I  
still believe does not foster code reuse in an environment with many  
high level PHP coders.

Cheers,

Gunnar

>
> -- 
> mike
>
> The Horde Project (www.horde.org)
> mrubinsk at horde.org

-- 
Core Developer
The Horde Project

e: wrobel at horde.org
t: +49 700 6245 0000
w: http://www.horde.org

pgp: 9703 43BE
tweets: http://twitter.com/pardus_de
blog: http://log.pardus.de