Revision as of 03:01, 17 December 2020

This procedure shows how to configure a periodic database backup using a bash script (tbdbbackup) executed from a Tmedia host unix cron jobs. Cron allows Linux and Unix users to run commands or scripts at a given date and time.

Overview

The tbdbbackup script support the following features:

• Backup the TMG-CONTROL configuration database into a backup file
• Backup file name formatted as tbdbbackup_<hostname>_toolpack_<version>_<date>_<time>.sql.gz

 tbdbbackup_TB014369_toolpack_2_8_2017_01_11_15h52m.sql.gz

• Keep N days of local backup files in /lib/tb/toolpack/pkg/database_backups. Default is 7 days.
• Backup files are listed on the Web portal in Backups->Manual Database Backup
• Optional copy to a remote server supporting SSH Key-Based Authentication
  • Copy the backup file on a remote server (using scp) - optional
  • Keep M days of backup files on the remote server (using ssh) - optional

On latest product version, the script should be available from the /usr/bin. To validate if you have the script on your Tmedia:

• Connect to the Tmedia using SSH as root

[root@TB014369 ~]# which tbdbbackup
/usr/bin/tbdbbackup

Install

The tbdbbackup script is included in the tb_tools packages 1.2-32.el7 (CentOS7) and 1.2-32.el5 (CentOS5).

yum install tb_tools
yum list tb_tools
  tb_tools.noarch        1.2-32.el7.tb

If you don't have direct access to the repository from the Tmedia, you can manually install the package on your Tmedia.

1. Get the tb_tools rpm package from the repository.

• CentOS 7: tb_tools-1.2-32.el7.tb.noarch.rpm
• CentOS 5: tb_tools-1.2-32.el5.noarch.rpm

2. Copy the rpm package to the Tmedia (using a sftp client) and install:

• CentOS 7:

yum --nogpgcheck localinstall tb_tools-1.2-32.el7.tb.noarch.rpm

• CentOS 5:

yum --nogpgcheck localinstall tb_tools-1.2-32.el5.noarch.rpm

Configuration

The script is configured with user variables.

• CONFIG_FILE: Configuration file name and location. By default, it is set to /root/tbdbbackup.cfg.

CONFIG_FILE=/root/tbdbbackup.cfg

tbdbbackup.cfg file example:

LOCAL_BACKUP_MAX_DAYS=1
REMOTE_BACKUP_MAX_DAYS=1
REMOTE_HOST=192.168.101.170
REMOTE_PORT=22
REMOTE_DIR=/root/backup_test
REMOTE_USER=root

The configuration file can redefine any of the following user variables. To see the default variables:

head -n 50 /usr/bin/tbdbbackup

• LOCAL_DIR : Set this variable to change the local directory. By default the script uses /lib/tb/toolpack/pkg/database_backups.

For example to change the local backup dir to /home/tbdbbackup:

LOCAL_DIR=/home/tbdbbackup

• LOCAL_BACKUP_MAX_DAYS : Max number of days to keep local backup files. Set to 7 days by default.

# Assign no value to disable the local cleanup
LOCAL_BACKUP_MAX_DAYS=7

• REMOTE_BACKUP_MAX_DAYS : Max number of days to keep remote backup files (if REMOTE_HOST is set). Set to 7 days by default.

# Assign no value to disable the remote cleanup
REMOTE_BACKUP_MAX_DAYS=7

• REMOTE_HOST : Remote host name or Ip address. Remote host copy is disabled by default.

# Remote host IP or hostname for the remote copy
REMOTE_HOST=10.10.10.10

By default, no value is assigned. This disable the remote copy.

REMOTE_HOST=

• REMOTE_PORT : Remote host ssh port.

REMOTE_PORT=22

• REMOTE_DIR : Remote directory where to copy the backup files (/root/backup by default)

REMOTE_DIR=/root/backup

• REMOTE_USER : Remote host login user.

REMOTE_USER=root

• CONNECT_TIMEOUT : scp connection timeout

CONNECT_TIMEOUT=10

• NAME_PREFIX : Change this variable to customize the file backup prefix file name

# By default take the Tmedia hostname for the backup 'hostname'.
NAME_PREFIX=tbdbbackup_$(hostname)

Remote Server Copy

You must create an SSH key pair and copy it to the remote server, if the remote server copy is configured. This will enable the Tmedia to copy the backup files to the remote host without asking for a password:

• Connect to the Tmedia using SSH as root

• Create Tmedia public/private dsa key pair

ssh-keygen -t rsa

Note: Do not add a pass phrase, simply press enter until the key pair is generate.

• Append the content of the public key to the remote host authorized keys

Tmedia public key location

/root/.ssh/id_rsa.pub

Remote Host authorized keys location

~/.ssh/authorized_keys

More details here: Password-less SSH

Troubleshooting

1.Manually run the script

/usr/bin/tbdbbackup

2.Check if the backup is locally created:

ls -alt /lib/tb/toolpack/pkg/database_backups/

3.If the remote copy is configured, make sure the file was copied on the remote server

4.To test the "LOCAL_BACKUP_MAX_DAYS" (if configured), you can perform the following:

• run the script a few times to create multiple files
• use "touch" to change the date of a file to a date older then "LOCAL_BACKUP_MAX_DAYS" days ago. For example, if "LOCAL_BACKUP_MAX_DAYS" is set to 15 days, set the date of the file as "16 days ago".

[root@TB014369]# ls -alt /lib/tb/toolpack/pkg/database_backups/
total 820
-rw-r--r--  1 root  root  206638 Jan 11 16:40 tbdbbackup_TB014369_toolpack_2_8_2017_01_11_16h40m.sql.gz
-rw-r--r--  1 root  root  206612 Jan 11 15:52 tbdbbackup_TB014369_toolpack_2_8_2017_01_11_15h52m.sql.gz

[root@TB014369 database_backups]# touch -d "16 days ago" /lib/tb/toolpack/pkg/database_backups/tbdbbackup_TB014369_toolpack_2_8_2017_01_11_16h40m.sql.gz

[root@TB014369]# ls -alt /lib/tb/toolpack/pkg/database_backups/
total 820
-rw-r--r--  1 root  root  206612 Jan 11 15:52 tbdbbackup_TB014369_toolpack_2_8_2017_01_11_15h52m.sql.gz
-rw-r--r--  1 root  root  206638 Dec 26 16:43 tbdbbackup_TB014369_toolpack_2_8_2017_01_11_16h40m.sql.gz

• run the script again; the file tbdbbackup_TB014369_toolpack_2_8_2017_01_11_16h40m.sql.gz should have been removed from /lib/tb/toolpack/pkg/database_backups/

5. Look at the script log file in /lib/tb/toolpack/pkg/database_backups/tbdbbackup.log

# cat /lib/tb/toolpack/pkg/database_backups/tbdbbackup.log 
Start scheduled backup: Thu Jan 12 11:53:12 EST 2017
+ mysqldump -u tbdb -ptbdbpw --opt toolpack_2_10 > tbdbbackup_TB014369_toolpack_2_10_2017_01_12_11h53m.sql
  => OK
+ gzip -f tbdbbackup_TB014369_toolpack_2_10_2017_01_12_11h53m.sql
  => OK
+ Delete local files older then 7 days:
  Nothing to delete
Stop scheduled backup: Thu Jan 12 11:53:21 EST 2017

Schedule the Script

Modify the system crontab to execute the backup on a regular bases.

• Create or modify the crontab on the Tmedia

crontab -e

• Add new line with a cron configuration

Example:

0 0 * * * /usr/bin/tbdbbackup > /dev/null 2>&1

Sending Backup Folder to Google Drive

You can upload the backup folder to Google Drive.

• Login to CLI via SSH and download the gdrive with the following command

  sudo wget -O drive https://drive.google.com/uc?id=0B3X9GlR6EmbnMHBMVWtKaEZXdDg

• Move GDrive to /usr/sbin/drive folder by entering this command:

 sudo mv drive /usr/sbin/drive        

• Set permissions:

 sudo chmod 755 /usr/sbin/drive

• run the drive command to start the authentication process. You will a link like the following;

  https://accounts.google.com/o/oauth2/auth?client_id=12345678901203-7n0vf5serdar7on...

• Copy and paste the link in the browser. Accept the permission and you will get a verification code. Copy the verification and paste back into your shell

Enter verification code: 4/9gKYAFAJ326XIP6JJHAEhs342t35LPiA5QGW0935...
You have installed Google Drive and Connected to your Google Drive account.

• We need to create a bash script to upload files to Google Drive. Go to your Google Drive and create a folder. Go to the folder. Now, look at the URL. At the end of the URL, you can see the folder’s path. Look at the screenshot to be clear:

So, 0B02roDDQi5I8SUpLZ29RZ1A2YWc is the Mix folder’s path. Copy this path.

• Now create a file named “backup.sh” in any folder of your server. The following file compress the /home/tbdbbackup folder and upload to the Google Drive. You can find other examples in the internet.

1. compress the directory

echo "Compressing directory..." tar -zcvf "tbdbbackup-backup-$(date '+%d-%m-%Y').tgz" /home/tbdbbackup echo "Directory compressed." echo ""

1. upload to google drive

echo "Uploading to Google Drive..." drive upload --file "tbdbbackup-backup-$(date '+%d-%m-%Y').tgz" -p 1F4j8gAM1OnJHsUJn0lamcQDiXci4GAUH echo "Uploaded to Google Drive." echo

1. remove the compressed file

echo "Clearing temporary files..." sudo rm "tbdbbackup-backup-$(date '+%d-%m-%Y').tgz" echo "Cleared temporary files." echo ""

• After creating backup.sh file you need to add cron job to run after tbdbbackup and upload generated backup to google drive. Cron job example can be found in this wiki.