Support:How To Run Toolpack Application In gdb - Revision history

Cbilodeau: split gdb command in 2 steps

2011-02-18T21:30:05Z

split gdb command in 2 steps

Abrassard at 12:40, 25 January 2011

2011-01-25T12:40:23Z

Nicole Tan at 15:59, 13 January 2011

2011-01-13T15:59:42Z

Abrassard at 12:19, 17 December 2010

2010-12-17T12:19:31Z

Abrassard: Created page with 'In case of Toolpack application crash, it's very useful to run the application inside the debugger (gdb) to catch the fault (even if the application binaries are compiled in 'rel…'

2010-12-17T12:18:54Z

Created page with 'In case of Toolpack application crash, it's very useful to run the application inside the debugger (gdb) to catch the fault (even if the application binaries are compiled in 'rel…'

New page

In case of Toolpack application crash, it's very useful to run the application inside the debugger (gdb) to catch the fault (even if the application binaries are compiled in 'release' without symbols, as gdb will still be able to provide call stack of all threads).
 
 
Notes:
 
For Toolpack setups with two redundant hosts, this procedure can be done on a "live" system without interrupting the service (no active calls will be lost, though few transient (ringing) calls will be dropped during the forced transitions from one application to it's standby).
 
For Toolpack setups without redundant hosts, service will be interrupted while the application to debug is not running (no new calls can be received in that period, though active calls continue to work).
 
 
Unfortunately, running a Toolpack application outside the Toolpack framework is not as simple as starting the application binary. A few environment variables need to be set first.
 
In that procedure, please replace the following values by those that correspond to your setup:
'2.5.42' : Replace that value by the package you are actually running
'toolpack_engine' : Replace by the application you wish to debug (gateway, toolpack_sys_mgr, etc.)
'toolpack_engine_1' : Replace by the application instance name in your database
(as seen in the Web Portal "Application->Instancea" or tboamapp's "Application launch manager" page)
Procedure:
First part of the procedure: stop the primary gateway application from running, and being re-launched by tboamapp
1- Log to the primary server (ssh)
2- cd /lib/tb/toolpack/pkg/2.5.42/bin/release/x86_64-linux64/
3- start tbx_cli_tools_remote application (./tbx_cli_tools_remote)
4- connect to "tboamapp" application
5- Select "Application launch manager" page (using TAB if appropriate)
6- press letter 's', will be prompted to choose 'Db id' of the application to stop manage
Enter the 'Db id' of the toolpack_engine application (see list of application in the page)
=> This will prevent toolpack_engine to relaunch itself automatically when we stop it
7- press letter 'd', will be prompted to choose the 'hProc' of the process to destroy.
Enter the value for toolpack_engine application
=> This will terminate toolpack_engine application, secondary host will take over
8- Press escape-escape, then 'control-C' to quit tbx_cli_tools_remote application

Then Set some environment variables to simulate Toolpack environment:
9- export TBDB_DRIVER=ODBC
export TBDB_ODBC_DRIVER=MySQL
export LD_LIBRARY_PATH=/lib/tb/toolpack/pkg/2.5.24/lib/release/x86_64-linux64/

Then start toolpack_engine application in the debugger:
10- gdb ./toolpack_engine
run -db -name toolpack_engine_1
11- Quit toolpack_engine on secondary host to force pass control back to primary in the debugger
(use tbx_cli_tools_remote to connect to the secondary toolpack_engine, press letter 'q' then 'y' then 'enter' to quit)

Wait for the crash to occur, then get the call stack of the crashed thread:
12- bt
13- thread apply all bt
(press enter until backtrace of all thread has been printed)
14- Take the backtraces, and send that to TelcoBridges support
(during that time, don't worry, secondary toolpack_engine has taken control of calls while debugger has paused the app)

Then restart gateway inside Toolpack as usual
15- Quit debugger, kill application
16- connect again to tboamapp using tbx_cli_tools_remote
17- Press 'a' then 'enter' to re-activate auto-launch of toolpack_engine
tboamapp should restart it within few seconds

@@ Line 6: / Line 6: @@
    'toolpack_engine_1'&nbsp;: Replace by the application instance name in your database
                         (as seen in the Web Portal "Application-&gt;Instancea" or tboamapp's "Application launch manager" page)
 Procedure:

@@ Line 35: / Line 35: @@
   Then start toolpack_engine application in the debugger:
-- gdb ./toolpack_engine run -db -name toolpack_engine_1
+- gdb ./toolpack_engine
-- Quit toolpack_engine on secondary host to force pass control back to primary in the debugger
+- run -db -name toolpack_engine_1
      (use tbx_cli_tools_remote to connect to the secondary toolpack_engine, press letter 'q' then 'y' then 'enter' to quit)
   Wait for the crash to occur, then get the call stack of the crashed thread:
 - bt
 - thread apply all bt
       (press enter until backtrace of all thread has been printed)
 - Take the backtraces, and send that to TelcoBridges support
       (during that time, don't worry, secondary toolpack_engine has taken control of calls while debugger has paused the app)
   Then restart toolpack_engine inside Toolpack as usual
 - Quit debugger, kill application
 - connect again to tboamapp using tbx_cli_tools_remote
 - Press 'a' then 'enter' to re-activate auto-launch of toolpack_engine
       tboamapp should restart it within few seconds

@@ Line 1: / Line 1: @@
-In case of Toolpack application crash, it's very useful to run the application inside the debugger (gdb) to catch the fault (even if the application binaries are compiled in 'release' without symbols, as gdb will still be able to provide call stack of all threads).
+In case of Toolpack application crash, it's very useful to run the application inside the debugger (gdb) to catch the fault (even if the application binaries are compiled in 'release' without symbols, as gdb will still be able to provide call stack of all threads). <br> <br> Notes: <br> For Toolpack setups with two redundant hosts, this procedure can be done on a "live" system without interrupting the service (no active calls will be lost, though few transient (ringing) calls will be dropped during the forced transitions from one application to it's standby). <br> For Toolpack setups without redundant hosts, service will be interrupted while the application to debug is not running (no new calls can be received in that period, though active calls continue to work). <br> <br> Unfortunately, running a Toolpack application outside the Toolpack framework is not as simple as starting the application binary. A few environment variables need to be set first. <br>
 <br>
 <br>
 Notes:
 <br>
 For Toolpack setups with two redundant hosts, this procedure can be done on a "live" system without interrupting the service (no active calls will be lost, though few transient (ringing) calls will be dropped during the forced transitions from one application to it's standby).
 <br>
 For Toolpack setups without redundant hosts, service will be interrupted while the application to debug is not running (no new calls can be received in that period, though active calls continue to work).
 <br>
 <br>
 Unfortunately, running a Toolpack application outside the Toolpack framework is not as simple as starting the application binary. A few environment variables need to be set first.
 <br>
   In that procedure, please replace the following values by those that correspond to your setup:
-   '2.5.42'            : Replace that value by the package you are actually running
+   '2.5.42'           &nbsp;: Replace that value by the package you are actually running
-   'toolpack_engine'   : Replace by the application you wish to debug (gateway, toolpack_sys_mgr, etc.)
+   'toolpack_engine'  &nbsp;: Replace by the application you wish to debug (gateway, toolpack_sys_mgr, etc.)
-   'toolpack_engine_1' : Replace by the application instance name in your database
+   'toolpack_engine_1'&nbsp;: Replace by the application instance name in your database
-                        (as seen in the Web Portal "Application->Instancea" or tboamapp's "Application launch manager" page)
+                        (as seen in the Web Portal "Application-&gt;Instancea" or tboamapp's "Application launch manager" page)
 Procedure:
   First part of the procedure: stop the primary toolpack_engine application from running, and being re-launched by tboamapp
 - Log to the primary server (ssh)
@@ Line 25: / Line 17: @@
 - press letter 's', will be prompted to choose 'Db id' of the application to stop manage
      Enter the 'Db id' of the toolpack_engine application (see list of application in the page)
-     => This will prevent toolpack_engine to relaunch itself automatically when we stop it
+     =&gt; This will prevent toolpack_engine to relaunch itself automatically when we stop it
 - press letter 'd', will be prompted to choose the 'hProc' of the process to destroy.
      Enter the value for toolpack_engine application
-     => This will terminate toolpack_engine application, secondary host will take over
+     =&gt; This will terminate toolpack_engine application, secondary host will take over
 - Press escape-escape, then 'control-C' to quit tbx_cli_tools_remote application
@@ Line 37: / Line 29: @@
   Then start toolpack_engine application in the debugger:
-- gdb ./toolpack_engine
+- gdb ./toolpack_engine run -db -name toolpack_engine_1
-     run -db -name toolpack_engine_1
 - Quit toolpack_engine on secondary host to force pass control back to primary in the debugger
      (use tbx_cli_tools_remote to connect to the secondary toolpack_engine, press letter 'q' then 'y' then 'enter' to quit)