[horde] How to move a pre-4.0 database to a new server

[Steve Campbell]

On 8/6/2014 2:35 AM, Steffen wrote:
> On Tue, 5 Aug 2014, Steve Campbell wrote:
>
>> Date: Tue, 05 Aug 2014 12:33:46 -0400
>> From: Steve Campbell <campbell at cnpapers.com>
>> To: horde at lists.horde.org
>> Subject: Re: [horde] How to move a pre-4.0 database to a new server
>>
>>
>> On 8/5/2014 11:23 AM, Steffen wrote:
>>> On Tue, 5 Aug 2014, Steve Campbell wrote:
>>>> On 8/5/2014 7:58 AM, Steve Campbell wrote:
>>>>> OK, I found the scripts in the old versions. So I'll start working 
>>>>> my way through them.
>>>>>
>>>>> steve
>>>>> On 8/5/2014 7:45 AM, Steve Campbell wrote:
>>>>>>
>>>>>> On 8/5/2014 6:19 AM, Steffen wrote:
>>>>>>> On Mon, 4 Aug 2014, Steve Campbell wrote:
>>>>>>>> On 8/4/2014 8:30 AM, Steffen wrote:
>>>>>>>>> On Mon, 4 Aug 2014, Steve Campbell wrote:
>>>>>>>>>
>>>>>>>>>> Just wondering what needs to be considered to move a pre-4.0 
>>>>>>>>>> MySQL database to a new server for IMP users. We only used 
>>>>>>>>>> IMP from the old framework.
>>>>>>>>>
>>>>>>>>>> Should I be able to export the old data and import it into 
>>>>>>>>>> the new database or has the database changed significantly?
>>>>>>>>>
>>>>>>>>> See http://www.horde.org/apps/horde/docs/UPGRADING
>>>>>>>>>
>>>>>>>>> I've exported the old database to a test system equally 
>>>>>>>>> configured to the current system, upgraded there to 3.3.5. On 
>>>>>>>>> a 2nd test machine I setup H5 with a config the final server 
>>>>>>>>> is to run with. On the H5 test machine I dropped the database, 
>>>>>>>>> imported the one from v3.3.5, then I ran webmail-install 
>>>>>>>>> again, which upgrades the old database, but also kills the 
>>>>>>>>> config. That database I imported to the production server.
>>>>>>>>>
>>>>>>>>> Maybe, you should run "horde-db-migrate" instead of 
>>>>>>>>> webmail-install, but the time I migrated my servers I wasn't 
>>>>>>>>> aware of that command.
>>>>>>>>>
>>>>>>>>> Note: I imported always the whole database by deleting the 
>>>>>>>>> existing one and re-creating the DB from scratch. I also used 
>>>>>>>>> the SQL export file to patch some stuff, e.g. migrated to UTF8 
>>>>>>>>> and changed the OWNER of the database.
>>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> I'm still not sure where to start. I've got 2 tables that have 
>>>>>>>> real data in them on the old system, horde_prefs and 
>>>>>>>> turba_objects. I suppose I should dump those two tables on the 
>>>>>>>> new server and create them with dumps from the old server using 
>>>>>>>> mysqldump. Maybe drop the entire database(?) but that seems 
>>>>>>>> real risky.
>>>>>>>>
>>>>>>>> After that, I'm lost. The link doesn't describe how to upgrade 
>>>>>>>> other than incrementally upgrading from one version to another 
>>>>>>>> to the next, and so on. I hope that doesn't mean I need to 
>>>>>>>> install old versions to get the upgrade scripts.
>>>>>>>
>>>>>>> As Mauricio writes, since PEAR all is automatic. That is you can 
>>>>>>> upgrade from H3.3.5 to any later version.
>>>>>>>
>>>>>>> pre3.3.5 you should upgrade step by step. I could not jump from 
>>>>>>> webmail-1.0 to Horde 3.3.5. The upgrade broke the database.
>>>>>>>
>>>>>>> And yes, Mauricio and I do recommend to replace the existing H5 
>>>>>>> database with the _complete_ old one and then run 
>>>>>>> horde-db-migrate. Do both, the upgrades of the old system and 
>>>>>>> the horde-db-migrate, on separate test servers, never on the 
>>>>>>> production servers.
>>>>>>>
>>>>>>> If you do not want to drop the original H5 table, just rename 
>>>>>>> it. _Never_ try to fiddle an old table into a current database, 
>>>>>>> that won't work.
>>>>>>>
>>>>>> Thanks all,
>>>>>>
>>>>>> Current version of Horde = 2.2.3-1, Imp = 3.3.1-1, turba = 1.2-1. 
>>>>>> I believe that is all that is running on the old server.
>>>>>>
>>>>>> Since the link points to upgrading Horde starting with 3.0 -> 
>>>>>> 3.1, how should I start to get myself to 3.5?
>>>>>>
>>>>>> I can't find the "scripts/upgrades" directory or files that the 
>>>>>> linked page references on my new server.
>>>>>>
>>>>>> steve
>>>>>
>>>> I ran through the updates on all the upgrades. When I attempt to 
>>>> "Update all DB schemas" from the adminstration/configuration 
>>>> screen, I get one error.
>>>>
>>>> The error indicates that object_photo does not exist in turba_objects.
>>>>
>>>> Does this sound familiar to anyone and is there an easy fix for 
>>>> this? Did I miss a step somewhere?
>>>
>>> OK, Horde 2 :-)
>>>
>>> I migrated from webmail-1.0, which is H3.
>>>
>>> I guess, maybe try lowest H3 version, then upgrade each step by step 
>>> 3.1.x -> 3.2.x -> 3.3.x. In H3 I always ran "scripts/setup.php" .
>>>
>>> And yes, I know this error. I migrated from webmail-1.1 to 1.2 some 
>>> columns had wrong types - however I use Postgres. I had to make 
>>> these changed manually:
>>>
>>> mapSqlCol turba_objects object_photo BYTEA
>>> mapSqlCol turba_objects object_logo BYTEA
>>> mapSqlCol horde_cache cache_data BYTEA
>>> ^^ script to do the conversation - Postgres depended
>>>
>>> I think their original type had been TEXT. The upgrade script was 
>>> unable to convert them, because the in-built converter was rejected 
>>> by Postgres, so that columns were lost.
>>>
>>> Note: I never made in-between version work fully. I just run the 
>>> particular setup / upgrade script of the version to upgrade the 
>>> database. Only the very first installation (the clone of the 
>>> production server) and the target installation (H5) did worked. The 
>>> command line setup script did all the stuff.
>>>
>> I'm running into so many errors during the execution of the scripts. 
>> The first one is the "php 
>> scripts/upgrades/2008-08-29_fix_mdb2_sequences.php" when upgrading 
>> from 3.2 to 3.3.x. Seems that any of the php scripts are to be run 
>> from the horde base (the real one, not the directory downloaded and 
>> expanded), so a lot of path issues arise. Then there's the "Horde" 
>> instead of "horde" it keeps looking for.
>
> ? You have upgrade from H2 to H3.2 and now you run in script errors 
> when upgrading to H3.3?
>
> Did you updated/copied over the basic config.php, the backends etc.pp. 
> to reflect the location and database?
>
> Maybe, you should explain what you've did.
>
>> I still have found where object_photo get created in the 
>> turba_objects table. And somewhere, the rules used for Imp users is 
>> not getting created in the upgrade, although I think the rules are 
>> still in the tables.
>
> What "rules for Imp" you mean?
>
>> At this point, I might have to just ask everyone if they can recreate 
>> their addressbook and rules. So frustrating.
>
> Along with the preferences I guess.
>
I probably should start this all over since it's become so long, but I'm 
sure I'm doing something wrong since it seems to be so straight forward 
for everyone else.

Let me ask a few questions first.

It's been suggested I create a test box with the original configuration. 
I'm not sure I can do that at this point in time, so should I be able to 
install the new version of Webmail, copy the old database to the new 
server, and then run the upgrades in order on that machine?

When you ask about the config.php are you talking about the new 
config.php or the one from the original box There is no backends.* file 
on the original box.

The rules are I'm speaking of are called "filters". They are the filters 
that do something like "if the email is from a particular person, then 
move the email to a particular folder". But there are Maintenance 
options like "purge the Trash folder of email that is older than 2 days 
old" that I can't find in the new system at all.

So far, I've ran the scripts involved in each of the major version 
upgrades since these major tar.gz files have all of the scripts for 
previous versions. The sql updates seem to do fine. The php scripts will 
usually cause problems.

What I guess I'll do next is await the reply on whether I can update all 
of this without creating a server copy of the original but instead use 
the new box with the latest horde installed to run the scripts. Then 
I'll work my way through this again and provide step-by-step job 
execution and errors I receive.

Thanks for all the help so far.

steve