From TBwiki

(Difference between revisions)

Latest revision as of 13:30, 26 August 2020

This procedure shows how to configure a Tmedia to send its CDR files to a remote server automatically and perform daily (or monthly) file archives.

Overview

The tbcopycdr script support the following features:

Move cdr files from TMG_CDR_DIR to TMG_TEMP_DIR
Send files to remote server
Move sucessfully sent files (rename to .sent.gz) from TMG_TEMP_DIR to TMG_ARCHIVE_DIR (ARCHIVE=true)
Rename sucessfully sent files (.sent.gz) in TMG_TEMP_DIR (ARCHIVE=false; ERASE_AFTER_SEND=false)
Delete sucessfully sent files from TMG_TEMP_DIR (ARCHIVE=false; ERASE_AFTER_SEND=true)
If a file failed to be sent to remote server, it is kept in TMG_TEMP_DIR until it is successfully sent

The tbarchivecdr script create a gzip tar archive with all gz files copied by tbcopycdr in TMG_ARCHIVE_DIR

Change directory to TMG_ARCHIVE_DIR
Create a temporary folder with the current date (ie. 2016-01-30)
Sleep for 90 second
Move all *.gz files and $TMG_TEMP_DIR/$LOG_FILE to this folder
Archive the folder in a gzipped tar file (cdr_2016_01_30_$hostname.tgz)

Install

Installation with access to internet

The tbcopycdr and tbarchivecdr scripts are included in the tb_tools packages 1.2-32.el7 (CentOS7) and 1.2-32.el5 (CentOS5).

yum install tb_tools
yum list tb_tools
  tb_tools.noarch        1.2-32.el7.tb

Offline Install

If you don't have direct access to the repository from the Tmedia, you can manually install the package on your Tmedia.

1. Get the tb_tools rpm package from the repository.

CentOS 7: tb_tools-1.2-32.el7.tb.noarch.rpm
CentOS 5: tb_tools-1.2-32.el5.noarch.rpm

2. Copy the rpm package to the Tmedia (using a sftp client) and install:

CentOS 7:

yum --nogpgcheck localinstall tb_tools-1.2-32.el7.tb.noarch.rpm

CentOS 5:

yum --nogpgcheck localinstall tb_tools-1.2-32.el5.noarch.rpm

Configuration

The script is configured with user variables.

CONFIG_FILE: Configuration file name and location. By default, it is set to /root/tbcopycdr.cfg

CONFIG_FILE=/root/tbcopycdr.cfg

tbcopycdr.cfg file example:

TMG_TEMP_DIR=/home/tbcopycdr
TMG_CDR_DIR=/lib/tb/toolpack/setup/12358/cdr_logs
ARCHIVE=true
TMG_ARCHIVE_DIR=/lib/tb/toolpack/setup/12358/cdr_archive
ARCHIVE_MAX_DAYS=180
REMOTE_HOST=192.168.101.170
REMOTE_PORT=22
REMOTE_DIR=/root/cdr_backups
REMOTE_USER=root

The configuration file can redefine any of the following user variables. To see the default variables:

head -n 50 /usr/bin/tbcopycdr

TMG_CDR_DIR: Tmedia directory where the CDR are located. Default is /lib/tb/toolpack/setup/12358/cdr_logs

TMG_CDR_DIR=/lib/tb/toolpack/setup/12358/cdr_logs

REMOTE_HOST : Remote hostname or Ip address. Remote host copy is disabled by default.

REMOTE_HOST=10.10.10.10

REMOTE_PORT : Remote host ssh port.

REMOTE_PORT=22

REMOTE_DIR : Remote directory where to copy the backup files (/root/backup by default)

REMOTE_DIR=/root/backup

REMOTE_USER : Remote host login user.

REMOTE_USER=root

CONNECT_TIMEOUT : scp connection timeout

CONNECT_TIMEOUT=10

ARCHIVE: true or false. If set to true, move sucessfully sent files (rename to .sent.gz) from TMG_TEMP_DIR to TMG_ARCHIVE_DIR. Default is false.

ARCHIVE=false

TMG_ARCHIVE_DIR: Directory where the cdrs are moved if ARCHIVE=true

TMG_ARCHIVE_DIR=/lib/tb/toolpack/setup/12358/cdr_archive

ARCHIVE_MAX_DAYS: Define how many days archive file (.tgz) should be kept in TMG_ARCHIVE_DIR. This user variable is used by tbarchivecdr only.

# Assign no value to disable
ARCHIVE_MAX_DAYS=

ERASE_AFTER_SEND: Take effect only if ARCHIVE is false. If ERASE_AFTER_SEND is true, erase file from TMG_TEMP_DIR, else, rename and keep files in working directory.

ERASE_AFTER_SEND=false

Note: if you set ERASE_AFTER_SEND to false the CDR text file will be available in the /home/telcobridges/cdr/ directory and rename *.gz.already_send.

Remote Server Copy

You must create an SSH key pair and copy it to the remote server, if the remote server copy is configured. This will enable the Tmedia to copy the backup files to the remote host without asking for a password:

Connect to the Tmedia using SSH as root

Create Tmedia public/private dsa key pair

ssh-keygen -t rsa

Note: Do not add a pass phrase, simply press enter until the key pair is generate.

Append the content of the public key to the remote host authorized keys

Tmedia public key location

/root/.ssh/id_rsa.pub

Remote Host authorized keys location

~/.ssh/authorized_keys

More details here: Password-less SSH

Troubleshooting

Test

Run the tbcopycdr script with the -v option

tbcopycdr -v

If no error were seen on the shell, your configuration is correct. Also, you should have a hello_world.log file on the remote host.

Log File

Transfer results are written in $TMG_TEMP_DIR/$LOG_FILE

Schedule the Scripts

Create a new crontab on the Tmedia

crontab -e

Add new line with the cron configuration

tbcopycdr:

Frequency	Cron Configuration
Every 10 minutes	`/10 * * * /usr/bin/tbcopycdr > /dev/null 2>&1`
hourly (top of the hour)	`0 * * * * /usr/bin/tbcopycdr > /dev/null 2>&1`
daily (midnight)	`0 0 * * * /usr/bin/tbcopycdr > /dev/null 2>&1`

tbarchivecdr:

Frequency	Cron Configuration
daily@23h59	`59 23 * * * /usr/bin/tbarchivecdr > /dev/null 2>&1`
weekly (sunday@23h59)	`59 23 * 7 * /usr/bin/tbarchivecdr > /dev/null 2>&1`

Cron table example for a cdr copy to an external server every 10 minutes with a daily archive of all the transmitted files:

*/10 * * * * /usr/bin/tbcopycdr > /dev/null 2>&1
59 23 * * * /usr/bin/tbarchivecdr > /dev/null 2>&1

@@ Line 1: / Line 1: @@
-This procedure shows how to configure a Tmedia to send its CDR files to a remote server automatically.
+This procedure shows how to configure a Tmedia to send its CDR files to a remote server automatically and perform daily (or monthly) file archives.
-Warning: If the destination is a Windows server, you require to install software that supports scp (i.e. freeSSHd). Not tested by TelcoBridges personnel.
+= Overview =
+The ''tbcopycdr'' script support the following features:
+*Move cdr files from TMG_CDR_DIR to TMG_TEMP_DIR
+*Send files to remote server
+*Move sucessfully sent files (rename to .sent.gz) from TMG_TEMP_DIR to TMG_ARCHIVE_DIR (ARCHIVE=true)
+*Rename sucessfully sent files (.sent.gz) in TMG_TEMP_DIR (ARCHIVE=false; ERASE_AFTER_SEND=false)
+*Delete sucessfully sent files from TMG_TEMP_DIR (ARCHIVE=false; ERASE_AFTER_SEND=true)
+*If a file failed to be sent to remote server, it is kept in TMG_TEMP_DIR until it is successfully sent
+<br>
+The ''tbarchivecdr'' script create a gzip tar archive with all gz files copied by ''tbcopycdr'' in TMG_ARCHIVE_DIR
+*Change directory to TMG_ARCHIVE_DIR
+*Create a temporary folder with the current date (ie. 2016-01-30)
+*Sleep for 90 second
+*Move all *.gz files and $TMG_TEMP_DIR/$LOG_FILE to this folder
+*Archive the folder in a gzipped tar file (cdr_2016_01_30_$hostname.tgz)
-The shell script used by this procedure is included in the [[Tmedia]]'s distribution. For other distribution the tbcopycdr script can be decompressed from the [http://repo.telcobridges.com/centos/5.5/updates/SRPMS/tb_tools-1.1-3.src.rpm tb_tools.src.rpm] SRPM.
+= Install =
+== Installation with access to internet ==
+The ''tbcopycdr'' and ''tbarchivecdr'' scripts are included in the ''tb_tools'' packages 1.2-32.el7 (CentOS7) and 1.2-32.el5 (CentOS5).
+ yum install tb_tools
+ yum list tb_tools
+   tb_tools.noarch        '''1.2-32'''.el7.tb
-= Setup SSH Public/Private Key =
+== Offline Install ==
+If you don't have direct access to the repository from the Tmedia, you can manually install the package on your Tmedia.<br><br>
+. Get the ''tb_tools'' rpm package from the repository.
+*CentOS 7:  [http://repo.telcobridges.com/centos/7_dev/os/x86_64/Packages/tb_tools-1.2-32.el7.tb.noarch.rpm tb_tools-1.2-32.el7.tb.noarch.rpm]
+*CentOS 5:  [http://repo.telcobridges.com/centos/5_dev/os/x86_64/CentOS/tb_tools-1.2-32.el5.noarch.rpm tb_tools-1.2-32.el5.noarch.rpm]
+<br>
-This will enable the [[Tmedia]] to send files to the remote host without asking for a password:
+. Copy the rpm package to the Tmedia (using a sftp client) and install:
+*CentOS 7:
+ yum --nogpgcheck localinstall tb_tools-1.2-32.el7.tb.noarch.rpm
+*CentOS 5:
+ yum --nogpgcheck localinstall tb_tools-1.2-32.el5.noarch.rpm
-*Connect to the [[Tmedia]] using SSH as root
+= Configuration =
+The script is configured with '''user variables'''.
+*''CONFIG_FILE'': Configuration file name and location. By default, it is set to /root/tbcopycdr.cfg
+ CONFIG_FILE=/root/tbcopycdr.cfg
-*Create [[Tmedia]] public/private dsa key pair
+''tbcopycdr.cfg'' file example:
-  ssh-keygen -t dsa
+ TMG_TEMP_DIR=/home/tbcopycdr
-Note: Do not add a pass phrase, simply press enter until the key pair is generate.
+  TMG_CDR_DIR=/lib/tb/toolpack/setup/12358/cdr_logs
-*Append the content of the public key to the remote host authorized keys
+ ARCHIVE=true
+ TMG_ARCHIVE_DIR=/lib/tb/toolpack/setup/12358/cdr_archive
+ ARCHIVE_MAX_DAYS=180
+ REMOTE_HOST=192.168.101.170
+ REMOTE_PORT=22
+ REMOTE_DIR=/root/cdr_backups
+ REMOTE_USER=root
-[[Tmedia]] public key location
+The configuration file can redefine any of the following user variables.
-  /root/.ssh/id_dsa.pub
+To see the default variables:
+  head -n 50 /usr/bin/tbcopycdr
-Remote Host authorized keys location
+*''TMG_CDR_DIR'': Tmedia directory where the [[Text_Call_Detail_Records_(CDR)|CDR]] are located. Default is /lib/tb/toolpack/setup/12358/cdr_logs
-  ~/.ssh/authorized_keys
+  TMG_CDR_DIR=/lib/tb/toolpack/setup/12358/cdr_logs
-= Configure the Tmedia to Send files =
+*''REMOTE_HOST'' : Remote hostname or Ip address. Remote host copy is disabled by default.
+ REMOTE_HOST=10.10.10.10
+*''REMOTE_PORT'' : Remote host ssh port.
+ REMOTE_PORT=22
+*''REMOTE_DIR'' : Remote directory where to copy the backup files (/root/backup by default)
+ REMOTE_DIR=/root/backup
+*''REMOTE_USER'' : Remote host login user.
+ REMOTE_USER=root
+*''CONNECT_TIMEOUT'' : ''scp'' connection timeout
+ CONNECT_TIMEOUT=10
-*Open the file configuration
+*''ARCHIVE'': true or false. If set to true, move sucessfully sent files (rename to .sent.gz) from TMG_TEMP_DIR to TMG_ARCHIVE_DIR. Default is false.
-  vim /usr/bin/tbcopycdr
+ ARCHIVE=false
+*''TMG_ARCHIVE_DIR'': Directory where the cdrs are moved if ARCHIVE=true
+  TMG_ARCHIVE_DIR=/lib/tb/toolpack/setup/12358/cdr_archive
+*''ARCHIVE_MAX_DAYS'': Define how many days archive file (.tgz) should be kept in TMG_ARCHIVE_DIR. This user variable is used by ''tbarchivecdr'' only.
+ # Assign no value to disable
+ ARCHIVE_MAX_DAYS=
-*Set the following variables in the file
+*''ERASE_AFTER_SEND'': Take effect only if ARCHIVE is false. If ERASE_AFTER_SEND is true, erase file from TMG_TEMP_DIR, else, rename and keep files in working directory.
-**TMG_CDR_DIR (Tmedia directory where the [[Text_Call_Detail_Records_(CDR)|CDR]] are located)
+ ERASE_AFTER_SEND=false
-**REMOTE_DIR  (Remote host directory where the [[Text_Call_Detail_Records_(CDR)|CDR]] will be send, the directory MUST exist and directory MUST end with a '/')
-**REMOTE_USER (Remote host user)
-**REMOTE_HOST (Remote host name or Ip address)
-**ERASE_AFTER_SEND (TMedia will erase the [[Text_Call_Detail_Records_(CDR)|CDR]] files locally or not after a successful send, default true)
 Note: if you set ERASE_AFTER_SEND to false the CDR text file will be available in the /home/telcobridges/cdr/ directory and rename *.gz.already_send.
-Example: Send the files to the home directory of the root user of host 10.10.10.10
+= Remote Server Copy =
-  TMG_CDR_DIR=/lib/tb/toolpack/setup/12358/2.5/apps/gateway/cdr_logs/
+You must create an SSH key pair and copy it to the remote server, if the remote server copy is configured. This will enable the Tmedia to copy the backup files to the remote host without asking for a password:
- REMOTE_DIR=~/
- REMOTE_USER=root
+*Connect to the Tmedia using SSH as root
-  REMOTE_HOST=10.10.10.10
- ERASE_AFTER_SEND=true
+*Create Tmedia public/private dsa key pair
+ ssh-keygen -t rsa
+Note: Do not add a pass phrase, simply press enter until the key pair is generate.
+*Append the content of the public key to the remote host authorized keys
+Tmedia public key location
+  /root/.ssh/id_rsa.pub
+Remote Host authorized keys location
+  ~/.ssh/authorized_keys
+More details here: [[Password_less_ssh|Password-less SSH]]
-=== Validate Setup Configuration ===
+=Troubleshooting =
+== Test ==
 *Run the tbcopycdr script with the -v option
   tbcopycdr -v
@@ Line 50: / Line 110: @@
 If no error were seen on the shell, your configuration is correct. Also, you should have a hello_world.log file on the remote host.
+== Log File ==
+Transfer results are written in $TMG_TEMP_DIR/$LOG_FILE
-= Configure Tmedia to send the files automatically =
+= Schedule the Scripts =
 *Create a new crontab on the [[Tmedia]]
@@ Line 57: / Line 119: @@
 *Add new line with the cron configuration
-Example:
+'''''tbcopycdr''''':
 {| cellpadding="5" border="1" class="wikitable"
 |-
 ! width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | Frequency
-! width="350" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | Cron Configuration
+! width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | Cron Configuration
 |-
-! width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | Every 10 minutes
+| width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); text-align:center;" | Every 10 minutes
-! width="350" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>*/10 * * * * /usr/bin/tbcopycdr</code>
+| width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>*/10 * * * * /usr/bin/tbcopycdr > /dev/null 2>&1</code>
 |-
-! width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | hourly (top of the hour)
+| width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); text-align:center;" | hourly (top of the hour)
-! width="350" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>0 * * * * /usr/bin/tbcopycdr</code>
+| width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>0 * * * * /usr/bin/tbcopycdr > /dev/null 2>&1</code>
 |-
-! width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | daily (midnight)
+| width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); text-align:center;" | daily (midnight)
-! width="350" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>0 0 * * * /usr/bin/tbcopycdr</code>
+| width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>0 0 * * * /usr/bin/tbcopycdr > /dev/null 2>&1</code>
 |}
+'''''tbarchivecdr''''':
+{| cellpadding="5" border="1" class="wikitable"
+|-
+! width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | Frequency
+! width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | Cron Configuration
+|-
+| width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); text-align:center;" | daily@23h59
+| width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>59 23 * * * /usr/bin/tbarchivecdr > /dev/null 2>&1</code>
+|-
+| width="210" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); text-align:center;" | weekly (sunday@23h59)
+| width="400" style="background: none repeat scroll 0% 0% rgb(255, 255, 255);" | <code>59 23 * 7 * /usr/bin/tbarchivecdr > /dev/null 2>&1</code>
+|}
+Cron table example for a cdr copy to an external server every 10 minutes with a daily archive of all the transmitted files:
+ */10 * * * * /usr/bin/tbcopycdr > /dev/null 2>&1
+23 * * * /usr/bin/tbarchivecdr > /dev/null 2>&1

TMG:Automatic Text CDR Retrieval

Latest revision as of 13:30, 26 August 2020

Contents

Overview

Install

Installation with access to internet

Offline Install

Configuration

Remote Server Copy

Troubleshooting

Test

Log File

Schedule the Scripts

Views

Personal tools

Navigation

Search

Toolbox