Sitworld: TEMS Audit Tracing Guide Appendix

ArtDecoFelineimage

Version 1.64000 31 May 2017

John Alvord, IBM Corporation

jalvord@us.ibm.com

Follow on twitter

Following is the information on tracing for TEMS Audit

There was a document size limit but is logically a part of it the document.

Appendix 1

Some of the advisories and report sections require diagnostic tracing. We will use a standard workload tracing for these examples. Independent of the implementation the diagnostic trace string looks like this:

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)

You always need error. The rest are additions to trace more.

There are multiple ways to set this tracing:

Static Trace Definitions – requires a process recycle

Linux/Unix

The best way is to add a file ms.environment to the <installdir>/config directory which has the same attributes/owner/group as ms.ini. Use touch/chmod/chown/chgrp to create such a file. If one already exists use it. Add the following to that file – one long line.

KBB_RAS1= error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)

Windows

Using MTEMS, right click on the TEMS line, click Advanced, click Edit Trace Parms…

In the Ras1 Filter enter

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)

Note there is no KBB_RAS1=  in this context.

z/OS

Add the following to the RKANPARU(KDSENV) file

KBB_RAS1= error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)

At that point recycle the TEMS and collect the data.

The ms.environment technique works from ITM 623 GA onward. Before that you can achieve the same goal by updating the TEMS config file

hostname_ms_temsnode.config

with the configuration string added in single quotes like this:

KBB_RAS1= ‘error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)’

Such a temporary update will be lost during a TEMS configuration, so it is fine for cases like this.

Dynamic – tacmd settrace

The best modern way to run such a command is via tacmd settrace. The sequence looks like this when tacmd is run from Linux/Unix. Note that the <temsnodeid> is the TEMS nodeid and not the hostname running the TEMS. This what would be seen in a tacmd listsystems output.

Linux/Unix

cd <installdir>/bin

./tacmd login -s ….   [login to hub TEMS]

./tacmd settrace -m <temsnodeid>  -p KBB_RAS1 -o ‘error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)’

Note the single quotes around the diagnostic trace parameters.

After some time – usually a couple hours – you can disable the trace like this:

cd <installdir>/bin

./tacmd login -s ….   [login to hub TEMS]

./tacmd settrace -m <temsnodeid>  -p KBB_RAS1 -r

Windows

The commands are largely the same but quoting is different. External quotes are double quotes and embedded double quotes are tripled.

cd <installdir>bbin

tacmd login -s ….   [login to hub TEMS]

tacmd settrace -m <temsnodeid>  -p KBB_RAS1 -o “error (unit:kpxrpcrq,Entry=”””IRA_NCS_Sample””” state er)(unit:kshdhtp,Entry=”””getHeaderValue”””  all) (unit:kshreq,Entry=”””buildSQL””” all)(unit:kfastpst,Entry=”””KFA_PostEvent””” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”””ProcessTable””” all er)(unit:kraafira,Entry=”””runAutomationCommand””” all)(unit:kglhc1c all)”Note the single quotes around the diagnostic trace parameters.

After some time – usually a couple hours – you can disable the trace like this:

cd <installdir>\bin

tacmd login -s ….   [login to hub TEMS]

tacmd settrace -m <temsnodeid>  -p KBB_RAS1 -r

 

Example Trace Strings

Hub TEMS basic workload:

./tacmd settrace -m <temsnodeid>  -p KBB_RAS1 -o “error (unit:kpxrpcrq,Entry=”””IRA_NCS_Sample””” state er)(unit:kshdhtp,Entry=”””getHeaderValue”””  all) (unit:kshreq,Entry=”””buildSQL””” all)(unit:kfastpst,Entry=”””KFA_PostEvent””” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”””ProcessTable””” all er)(unit:kraafira,Entry=”””runAutomationCommand””” all)(unit:kglhc1c all)”

Remote TEMS basic workload:

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er) (UNIT:kfaprpst ER ST) (UNIT:kfastinh,ENTRY:”KFA_InsertNodests” ALL)(unit:kdssqprs metrics in er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)

Hub TEMS Workload plus Heartbeat:

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er) (UNIT:kfaprpst ER ST) (UNIT:kfastinh,ENTRY:”KFA_InsertNodests” ALL)(unit:kdssqprs metrics in er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)

Remote TEMS Workload plus Heartbeat:

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er) (UNIT:kfaprpst ER ST) (UNIT:kfastinh,ENTRY:”KFA_InsertNodests” ALL)(unit:kdssqprs metrics in er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)(UNIT:kfaprpst ST ER)

Hub TEMS plus heartbeat plus KPX traces to watch input in detail:

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er)(unit:kshdhtp,Entry=”getHeaderValue”  all) (unit:kshreq,Entry=”buildSQL” all)(unit:kfastpst,Entry=”KFA_PostEvent” all er)(unit:kdssqprs in metrics er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)(UNIT:kfaprps ST ER)

Remote TEMS plus heartbeat plus KPX traces to watch input in detail:

error (unit:kpxrpcrq,Entry=”IRA_NCS_Sample” state er) (UNIT:kfaprpst ER ST) (UNIT:kfastinh,ENTRY:”KFA_InsertNodests” ALL)(unit:kdssqprs metrics in er)(unit:kdsstc1,Entry=”ProcessTable” all er)(unit:kraafira,Entry=”runAutomationCommand” all)(unit:kglhc1c all)(UNIT:kfaprps ST ER)(UNIT:kfastinh,ENTRY:”KFA_InsertNodests” ALL)(UNIT:kpxreq ALL)(UNIT:kpxreqds ALL)

Rarer dynamic options

There was an earlier way to make dynamic tracing changes documented here:

Dynamically modify trace settings for an IBM Tivoli Monitoring component

http://www-1.ibm.com/support/docview.wss?rs=0&uid=swg21266129

It is sometimes blocked by firewall restrictions and lack of login credentials to the system running the TEMS.

There is also a z/OS TEMS option which looks a bit like

CTDS TRACE ADD FILTER ID=001 UNIT=KOCACHE CLASS(ALL)

If  must use that please contact the author for details.

  Versions:

Here are recently published versions, In case there is a problem at one level you can always back up.

1.64000 – first publication of the trace appendix

Sitworld: Table of Contents

Note: Art Deco Cat sculpture

 

One thought on “Sitworld: TEMS Audit Tracing Guide Appendix

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: