FN19497

From TBwiki
(Difference between revisions)
Jump to: navigation, search
m (moved FN:19497 to FN19497: Remove colon)
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{DISPLAYTITLE:FN19497 - Procedure to Recover a Major Upgrade after Recovering Database from Newer Version to Older Version }}
+
{{DISPLAYTITLE:FN19497 - Major release upgrade fail if the database have data upgraded from a newer major release}}
  
 
= Affected Products =
 
= Affected Products =
 
* TMG800, TMG3200, TMG7800-CTRL running Toolpack software older than rel2.7.176
 
* TMG800, TMG3200, TMG7800-CTRL running Toolpack software older than rel2.7.176
 +
  
 
= Problem Description =
 
= Problem Description =
* Import a database backup from a newer version to an older version. e.g. Imported a database backup from rel2.9 to a TMG which is running rel2.7.175 (or older)
+
Importing a database from a system running a newer major release to a older major release is not supported (e.g. import a database from a system running 2.9 to 2.7). Previous to release 2.7.176, there was no mechanism in the web portal to prevent such action. From 2.7.176 onward, the validation and recover now block such action. However, upgrading a system to a newer version which have a database imported from a newer major release system, will result in a failure of the upgrade.
* Validate/Recover the database
+
* Do a major upgrade on the TMG
+
  
= Problem Symptoms =
+
== How to get into the issue ==
* Upgrade fails, always switch back to the old version
+
* Import a database backup from a newer major release to an older release previous to 2.7.176. e.g. Import a database backup from rel2.9 to a TMG which is running rel2.7.175 (or older)
* tboamapp logs shows: database_upgrade: [FAIL]    ERROR: data upgrade 26 failed: ERROR: data upgrade 26
+
* Validate/Recover the database backup (validation and recover would be performed)
 +
 
 +
== System database backup recover ==
 +
Each time a database backup is recovered, the system proceed to the following actions:
 +
* perform an automatic backup of the actual database (under Backups->Automatic database backups)
 +
* erase the actual database and replace it by the new database (recovered)
 +
* proceed to the database data upgrade to match the actual system's version
 +
 
 +
== How to validate your system database ==
 +
{| class="wikitable"
 +
|-
 +
! Release !! Database Name !! Unique data upgrade number
 +
|-
 +
| Release 2.6 || toolpack_0 || 3
 +
|-
 +
| Release 2.7 || toolpack_0 || 15
 +
|-
 +
| Release 2.8 || toolpack_2_8 || 44
 +
|-
 +
| Release 2.9 || toolpack_2_9 || 57
 +
|}
 +
 
 +
# mysql -u tbdb -ptbdbpw -e "select * from database_name.current_data_upgrades order by CAST(id as SIGNED INTEGER) asc;"
 +
To make sure your system does not run with a newer version database, you can look at the database data upgrades table and make sure that is does not include upgrades from newer releases. Note that some upgrade number are common to more than one release, for example the upgrade number '61', '63', '73', etc.. are common to both 2.8 and 2.9 releases. The table above provides unique data upgrade for a particular major release. For example, the data upgrade '57' can only be found if the database data have been upgraded from a 2.9 system.
 +
 
 +
For, example, if your system runs release 2.8, in the command above replace ''database_name'' by "toolpack_2_8" to get the list of upgrades performed on your database:
 +
  # mysql -u tbdb -ptbdbpw -e "select * from toolpack_2_8.current_data_upgrades order by CAST(id as SIGNED INTEGER) asc;"
 +
  [root@TB013496 ~]# mysql -u tbdb -ptbdbpw -e "select * from toolpack_2_8.current_data_upgrades order by CAST(id as SIGNED INTEGER) asc;"
 +
  +----+-----------------------------------------------------------------------------------------------------------------------+
 +
  | id | description                                                                                                          |
 +
  +----+-----------------------------------------------------------------------------------------------------------------------+
 +
  | 0  | Move fax behavior enable to profiles and fix default passthrough detection type.                                      |
 +
  | 1  | Remove unique constraint on gateway configurations for RADIUS server configurations.                                  |
 +
  | 2  | Remove the USING BTREE attribute from the RADIUS server configurations table.                                        |
 +
  | 3  | Change the RADIUS dictionary for attributes 1-29.                                                                    |
 +
  | 4  | Re-use the value from field nat_traversal as the new remote_nat_traversal_method_id on table nap_sip_cfgs            |
 +
  | 5  | Update the RADIUS dictionary to add attributes (30-33).                                                              |
 +
  ...
 +
  | 44 | Create the default tone definition profiles.                                                                          |
 +
  | 45 | Create custom tone definition profiles with current data.                                                            |
 +
  | 46 | Update the RADIUS dictionary to modify attributes (changed RemoteIp/RemotePort).                                      |
 +
  | 48 | Re-generate some unique ids that were previously improperly generated.                                                |
 +
  | 49 | Clear old relation between ip_interfaces and virtual_adapters                                                        |
 +
  | 55 | Update the default tone definition profiles (No change. Needed to fix tone definitions truncated at 256 characters ). |
 +
  | 61 | Fill reason cause action in profile cause maps                                                                        |
 +
  | 63 | Update the default tone definition profiles (re-apply default values)                                                |
 +
  | 73 | Update the RADIUS dictionary to add attribute (CollectedDigits).                                                      |
 +
  | 77 | Fixed radius associated_server column which could point to another configuration after copy                          |
 +
  | 78 | Generate UniqueID for host configurations                                                                            |
 +
  | 79 | Update the cause maps for SIP redirect (release 2.8 only)                                                            |
 +
  | 80 | Update the default tone definition profiles (fixed some profiles).                                                    |
 +
  | 84 | Fix leaked timeslots pointing to mtp2 links but wrong line service                                                    |
 +
  | 86 | Fix leaked NAP columns and custom Route Columns                                                                      |
 +
  +----+-----------------------------------------------------------------------------------------------------------------------+
 +
 
 +
In the example above, you can find the data upgrade number '44' in the list, which indicate that the data of this database have been upgraded by a 2.8 system. However, the data upgrade number '57' is not present, which meant that it was not upgrade by a 2.9 system. therefore, this is a valid database for a 2.8 system.
 +
 
 +
If you find out data upgrades from a newer releases, you should:
 +
# Recover from a valid database backup of a previous or similar release. The backup could also be imported from another system.
 +
# Since there is no conversion tools to migrate the data from a newer release database to an older release, you might have to manually reconfigure some items in your system (missing parts that was configured on your newer release database or that you have configured since you imported the newer release database on your system)
 +
 
 +
== Problem Symptoms ==
 +
If your database data have been previously upgraded by a newer release, performing a major upgrade on your system would lead to a failure of the software release upgrade.
 +
* Software release upgrade fails, always switch back to the old version
 +
* tboamapp logs would shows
 +
database_upgrade: [FAIL]    ERROR: data upgrade 26 failed: ERROR: data upgrade 26 and your system would not switch to the latest
  
 
= Solution =
 
= Solution =
 +
If your database was previously upgraded from a newer release (as described above) and you inadvertently upgraded your system to a newer software release which ended by a failure, you must recover your database with one from the release your were running previous to the upgrade. You can look for an automatic backup in Backups (web portal) and recover from it.
 +
 +
For example, lets say your database was recovered from a 2.9 system on a 2.7 system and you upgraded to the software release 2.8. After you activated the package 2.8, the system would report a failure and fall back on 2.7 (if running 2.7.148 onward). Here is the steps you should follow:
 +
 +
 +
1. Recover from a valid 2.7 database backup or a previous release backup. Skip to steps 4a or 4b.
 +
 +
 +
If for some reason you can't recover a valid database, erase the databases to force the system to create new ones:
 +
 +
 +
2. ssh to your TMG host
 +
 +
 +
3. Drop the 2.7 database (toolpack_0) and the 2.8 database (toolpack_2_8):
 +
 +
  # tbtoolpack stop
 +
  # mysql -u tbdb -ptbdbpw -e "drop database toolpack_0;"
 +
  # mysql -u tbdb -ptbdbpw -e "drop database toolpack_2_8;"
 +
 +
 +
4a. If you '''upgrade from a release previous to 2.7.148''', you must follow the instruction below:
  
1- ssh to the TMG host
+
# tbtoolpack stop
2- stop toolpack by shell command: tbtoolpack stop
+
  [root@TB007036 ~]# tbtoolpack stop
+
  0:tbtoolpack_12358
+
  Stopping tbtoolpack_12358:                                [  OK  ]
+
  Toolpack current state: Disconnected
+
  Ok
+
  
2- Drop both databases: toolpack_0 and toolpack_2_x
+
Edit the following file with "vi":  
  [root@TB007036 ~]# mysql -u tbdb -ptbdbpw
+
  Welcome to the MySQL monitor.  Commands end with ; or \g.
+
  Your MySQL connection id is 116
+
  Server version: 5.1.47-log Source distribution
+
  
   Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
+
   vi /lib/tb/toolpack/setup/12358/service/config.txt
  This software comes with ABSOLUTELY NO WARRANTY. This is free software,
+
  and you are welcome to modify and redistribute it under the GPL v2 license
+
  
  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
+
Make sure that the 'application=' line contain '2.8'.  
  
   mysql> drop database toolpack_0;
+
   application= /lib/tb/toolpack/setup/12358/2.8/apps/tboamapp/tboamapp_wd_cfg.ini
  mysql> drop database toolpack_2_9;
+
  
3- Check the config files to make sure it is pointing to the new release.
+
Restart toolpack
vi /lib/tb/toolpack/setup/12358/service/config.txt
+
  # tbtoolpack start
  
application= /mnt/.0/tb/toolpack/setup/12358/2.9/apps/tboamapp/tboamapp_wd_cfg.ini
 
  
vi /mnt/.0/tb/toolpack/setup/12358/2.9/apps/tboamapp/tboamapp_wd_cfg.ini
+
4b. If you upgrade from a release supporting major release upgrade from the web ('''2.7.148 and above'''), you must follow the instruction below:
  
TB_PKG_PATH                    = /lib/tb/toolpack/pkg/2.9.41
+
Restart toolpack
 +
  # tbtoolpack start
  
4- Manually start toolpack on the newer release
+
From the Web portal, '''activate package 2.8.'''
[root@TB007036 ~]# tbtoolpack start
+
0:tbtoolpack_12358
+
Starting tbtoolpack_12358:                                [  OK  ]
+
Toolpack current state: Ready
+
Ok
+

Latest revision as of 10:36, 6 July 2016


Contents

Affected Products

  • TMG800, TMG3200, TMG7800-CTRL running Toolpack software older than rel2.7.176


Problem Description

Importing a database from a system running a newer major release to a older major release is not supported (e.g. import a database from a system running 2.9 to 2.7). Previous to release 2.7.176, there was no mechanism in the web portal to prevent such action. From 2.7.176 onward, the validation and recover now block such action. However, upgrading a system to a newer version which have a database imported from a newer major release system, will result in a failure of the upgrade.

How to get into the issue

  • Import a database backup from a newer major release to an older release previous to 2.7.176. e.g. Import a database backup from rel2.9 to a TMG which is running rel2.7.175 (or older)
  • Validate/Recover the database backup (validation and recover would be performed)

System database backup recover

Each time a database backup is recovered, the system proceed to the following actions:

  • perform an automatic backup of the actual database (under Backups->Automatic database backups)
  • erase the actual database and replace it by the new database (recovered)
  • proceed to the database data upgrade to match the actual system's version

How to validate your system database

Release Database Name Unique data upgrade number
Release 2.6 toolpack_0 3
Release 2.7 toolpack_0 15
Release 2.8 toolpack_2_8 44
Release 2.9 toolpack_2_9 57
# mysql -u tbdb -ptbdbpw -e "select * from database_name.current_data_upgrades order by CAST(id as SIGNED INTEGER) asc;"

To make sure your system does not run with a newer version database, you can look at the database data upgrades table and make sure that is does not include upgrades from newer releases. Note that some upgrade number are common to more than one release, for example the upgrade number '61', '63', '73', etc.. are common to both 2.8 and 2.9 releases. The table above provides unique data upgrade for a particular major release. For example, the data upgrade '57' can only be found if the database data have been upgraded from a 2.9 system.

For, example, if your system runs release 2.8, in the command above replace database_name by "toolpack_2_8" to get the list of upgrades performed on your database:

 # mysql -u tbdb -ptbdbpw -e "select * from toolpack_2_8.current_data_upgrades order by CAST(id as SIGNED INTEGER) asc;"
 [root@TB013496 ~]# mysql -u tbdb -ptbdbpw -e "select * from toolpack_2_8.current_data_upgrades order by CAST(id as SIGNED INTEGER) asc;"
 +----+-----------------------------------------------------------------------------------------------------------------------+
 | id | description                                                                                                           |
 +----+-----------------------------------------------------------------------------------------------------------------------+
 | 0  | Move fax behavior enable to profiles and fix default passthrough detection type.                                      |
 | 1  | Remove unique constraint on gateway configurations for RADIUS server configurations.                                  |
 | 2  | Remove the USING BTREE attribute from the RADIUS server configurations table.                                         |
 | 3  | Change the RADIUS dictionary for attributes 1-29.                                                                     |
 | 4  | Re-use the value from field nat_traversal as the new remote_nat_traversal_method_id on table nap_sip_cfgs             |
 | 5  | Update the RADIUS dictionary to add attributes (30-33).                                                               |
 ...
 | 44 | Create the default tone definition profiles.                                                                          |
 | 45 | Create custom tone definition profiles with current data.                                                             |
 | 46 | Update the RADIUS dictionary to modify attributes (changed RemoteIp/RemotePort).                                      |
 | 48 | Re-generate some unique ids that were previously improperly generated.                                                |
 | 49 | Clear old relation between ip_interfaces and virtual_adapters                                                         |
 | 55 | Update the default tone definition profiles (No change. Needed to fix tone definitions truncated at 256 characters ). |
 | 61 | Fill reason cause action in profile cause maps                                                                        |
 | 63 | Update the default tone definition profiles (re-apply default values)                                                 |
 | 73 | Update the RADIUS dictionary to add attribute (CollectedDigits).                                                      |
 | 77 | Fixed radius associated_server column which could point to another configuration after copy                           |
 | 78 | Generate UniqueID for host configurations                                                                             |
 | 79 | Update the cause maps for SIP redirect (release 2.8 only)                                                             |
 | 80 | Update the default tone definition profiles (fixed some profiles).                                                    |
 | 84 | Fix leaked timeslots pointing to mtp2 links but wrong line service                                                    |
 | 86 | Fix leaked NAP columns and custom Route Columns                                                                       |
 +----+-----------------------------------------------------------------------------------------------------------------------+

In the example above, you can find the data upgrade number '44' in the list, which indicate that the data of this database have been upgraded by a 2.8 system. However, the data upgrade number '57' is not present, which meant that it was not upgrade by a 2.9 system. therefore, this is a valid database for a 2.8 system.

If you find out data upgrades from a newer releases, you should:

  1. Recover from a valid database backup of a previous or similar release. The backup could also be imported from another system.
  2. Since there is no conversion tools to migrate the data from a newer release database to an older release, you might have to manually reconfigure some items in your system (missing parts that was configured on your newer release database or that you have configured since you imported the newer release database on your system)

Problem Symptoms

If your database data have been previously upgraded by a newer release, performing a major upgrade on your system would lead to a failure of the software release upgrade.

  • Software release upgrade fails, always switch back to the old version
  • tboamapp logs would shows
database_upgrade: [FAIL]    ERROR: data upgrade 26 failed: ERROR: data upgrade 26 and your system would not switch to the latest 

Solution

If your database was previously upgraded from a newer release (as described above) and you inadvertently upgraded your system to a newer software release which ended by a failure, you must recover your database with one from the release your were running previous to the upgrade. You can look for an automatic backup in Backups (web portal) and recover from it.

For example, lets say your database was recovered from a 2.9 system on a 2.7 system and you upgraded to the software release 2.8. After you activated the package 2.8, the system would report a failure and fall back on 2.7 (if running 2.7.148 onward). Here is the steps you should follow:


1. Recover from a valid 2.7 database backup or a previous release backup. Skip to steps 4a or 4b.


If for some reason you can't recover a valid database, erase the databases to force the system to create new ones:


2. ssh to your TMG host


3. Drop the 2.7 database (toolpack_0) and the 2.8 database (toolpack_2_8):

 # tbtoolpack stop
 # mysql -u tbdb -ptbdbpw -e "drop database toolpack_0;"
 # mysql -u tbdb -ptbdbpw -e "drop database toolpack_2_8;"


4a. If you upgrade from a release previous to 2.7.148, you must follow the instruction below:

# tbtoolpack stop

Edit the following file with "vi":

 vi /lib/tb/toolpack/setup/12358/service/config.txt

Make sure that the 'application=' line contain '2.8'.

 application= /lib/tb/toolpack/setup/12358/2.8/apps/tboamapp/tboamapp_wd_cfg.ini

Restart toolpack

 # tbtoolpack start


4b. If you upgrade from a release supporting major release upgrade from the web (2.7.148 and above), you must follow the instruction below:

Restart toolpack

 # tbtoolpack start

From the Web portal, activate package 2.8.

Personal tools