John Alvord, IBM Corporation
Introduction
ITM 6 has a marvelous ability to collect historical data. Best practice is to collect the historical data at the TEMA or Agent and then export the data to the Warehouse Proxy Agent which then forwards the data to the data warehouse. With a large number of agents almost anything can go wrong and require fixing. Identifying the problem cases has been challenging. A few years ago a new TEMA attribute group was added to expose the last export status. This can be used in a situation formula and generate situation alerts to problem cases. This post shows exactly how to do that. An appendix at the end lists all the current status codes. In some cases you can resolve it yourself, in other cases IBM Support will be involved.
Step by Step Situation Development
Right click on a TEP navigation node such as Linux OS under a test Linux System. Select Situations… Click on the new Situation action. Enter a situation name in the dialog box. If not matching what you want, also enter Monitored Application.
Next click OK and define the attribute group [ITM Historical Exports] and attribute item [Last Export Status].
Click OK. For the first experiment set the test to be == 0, meaning alert when things are working as expected.
Click on Advanced,Display Item and select Collection Identifier.
Now make sure situation is distributed to your test system and OK out. The situation should start immediately and in the Situation Event Console you will see
For the next steps you will likely want to test for Last Export Status not equal to zero. Next you will expand the distribution to more agents, like all Linux agents.
Last Export Status – what it means
The appendix has a list of all currently known export status values. 0 means success and can be ignored for this purpose.
One common one is 26
CTX_MetafileNotfound, 26
In general this means that historical data was configured, but no data was every collected. The example studied closely was some Linux LPAR attribute group, but the Linux system being looked at did not have any LPAR capability. For this site, we recommended that the attribute group should not be collected. Another way to avoid alerting would be to extend the formula to exclude the 26 status.
Other errors may lead to obvious conditions – like an inability for the agent to connect to the WPA or maybe a nearly full mount point. It any case you need to investigate and resolve… with IBM Support if need.
Other possibilites
Another common issue involves unhealthy agents – online but not responding and not running situations. Here is a blog post and program to help track them down:
Sitworld: ITM Agent Health Survey
These cases will not be running the historical data UADVISOR situations, which have the same effect. However they will not alert because no situations are running.
Summary
This shows how to create a situation to alert on some Historical Data collection and export problems.
Sitworld: Table of Contents
Appendix 1: Historical Data Export Error codes
KRAHIST – ITM History Exort – Last Export Status
CTX_Success = 0, 0
CTX_InvalidParameter, 1
CTX_InvalidOutputFormat, 2
CTX_NoMemory, 3
CTX_InternalError, 4
CTX_LogonFailed, 5
CTX_InvalidUserid, 6
CTX_InvalidPassword, 7
CTX_ConnectionFailed, 8
CTX_TargetTypeUndetectable, 9
CTX_EmailSendFailed = 10, 10
CTX_InvalidRecipType, 11
CTX_SMTPError, 12
CTX_OLEInitializationFailed, 13
CTX_InvalidInitCall, 14
CTX_SessionStartupFailed, 15
CTX_CMSConnectFailed, 16
CTX_DSError, 17
CTX_EndOfData, 18
CTX_InvalidDataType, 19
CTX_ODBCError = 20, 20
CTX_TableNotFound, 21
CTX_ParmDataLengthError, 22
CTX_InvalidParameterFormat, 23
CTX_InvalidExportType, 24
CTX_MetafileFormatError, 25
CTX_MetafileNotfound, 26
CTX_MetafileIOError, 27
CTX_MetafileCloseError, 28
CTX_HistoryIOError, 29
CTX_HistoryFileNotfound = 30, 30
CTX_HistoryCloseError, 31
CTX_SocketIOError, 32
CTX_GetHostnameError, 33
CTX_SMTPConnectError, 34
CTX_GetHostByNameError, 35
CTX_GetServByNameError, 36
CTX_SocketError, 37
CTX_SocketServerResponseError, 38
CTX_ColumnsNotBound, 39
CTX_ColumnsAlreadyBound = 40, 40
CTX_NotExporterMessage, 41
CTX_InvalidSocketBufferLength, 42
CTX_SocketBufferOverflow, 43
CTX_MessageParseError, 44
CTX_SchemaFormatError, 45
CTX_OLEError, 46
CTX_RequestRouted, 47
CTX_NoListenTask, 48
CTX_RPCError, 49
CTX_SpreadsheetNotFound = 50, 50
CTX_IncompatibleSpreadsheet, 51
CTX_NoObjectResolution, 52
CTX_ServerDied, 53
CTX_PDSLoadError, 54
CTX_PDSNotAvailable, 55
CTX_RPCRequestHandleError, 56
CTX_PropertyNotFound, 57
CTX_NoProperties, 58
CTX_EnvNo, 59
CTX_InitJVMError = 60, 60
CTX_JavaError, 61
CTX_JDBCError, 62
CTX_TempFileError, 63
CTX_ColumnLengthError, 64
CTX_PThreadError, 65
CTX_CfgFileError, 66
CTX_ConfigRecNotFound, 67
CTX_SkipRecord, 68
CTX_InvalidRPCFunction, 69
CTX_ConfigOpenError = 70, 70
CTX_ConfigCloseError, 71
CTX_RedriveExport, 72
CTX_WarehouseProxyNotRegistered, 73
CTX_GLBUnavailable, 74
CTX_InitializationFailed, 75
CTX_ParseError, 76
CTX_SQLFileNotFound, 77
CTX_RTNCustIDNotSet, 78
CTX_NotConnected, 79
CTX_GetJavaVMInitArgsFailed = 80,80
CTX_CreateJavaVMFailed, 81
CTX_GetJavaMethodFailed, 82
CTX_FindJavaClassFailed, 83
CTX_CompressionError, 84
CTX_InvalidOracleODBCDriver, 85
CTX_DataTypenameUnavailable, 86
CTX_RPCInterfaceRegisterError, 87
CTX_RouteNotifyMismatch, 88
CTX_PDSOpenInputNoRecords, 89
CTX_NotFound = 90, 90
CTX_NameConversionFailed, 91
CTX_NameCompatibility, 92
CTX_HistorySaveError, 93
CTX_HistoryDeleteError, 94
CTX_MetaSaveError, 95
CTX_MetaDeleteError, 96
CTX_PDSSetupError, 97
CTX_DecodeError, 98
CTX_BeginExport, 99
CTX_ExportInProgress = 100, 100
CTX_NoDataFound, 101
CTX_ShutdownRequested, 102
CTX_UCS2TranslationError = 200, 200
CTX_InvalidDatabaseEncoding, 201
CTX_ColumnNotFound, 202
CTX_ExistAlready, 203
CTX_InvalidList, 204
CTX_Reserved, 205
CTX_Not_Reserved, 206
CTX_No_More_Connection = 210, 210
CTX_Not_Initialized , 211
CTX_DB_Not_Connected, 212
CTX_AggregationTable, 213
CTX_OracleInternalTable, 214
CTX_Cnx_Null, 215
CTX_ServerTimeout, 216
CTX_ServerTimeout_BeforeCommit, 217
CTX_Table_Altered, 218
CTX_DBError, 219
CTX_SampleCommitError, 220
CTX_SampleLogStatusError, 221
CTX_QueueNotInitialized, 222
CTX_QueueFull, 223
CTX_QueueStopped, 224
CTX_JavaEnv_Null, 225
CTX_NoNeedToRenameFile, 226
CTX_SkipWrite, 227
CTX_InitializationWarning = 300,300
CTX_EncodingError, 301
CTX_AddressError, 302
CTX_InitFailedRecently, 303
CTX_Table_Deleted, 304
CTX_Wrong_Schema, 305
CTX_RenameConfigFileError, 306
CTX_RemoveConfigFileError, 307
CTX_ConfigFileEmptyError, 308
CTX_SelectSiteError = 320, 320
CTX_GetCurrentCMSAddressError, 321
CTX_RemoveMetaFileError = 325, 325
CTX_MetaFileSizeError, 326
CTX_Failed_Batch_Row_Not_Found=350, 350
CTX_RowExistAlready, 351
CTX_ODBC_DSN_Exceed_Max_Len, 352
CTX_Compress_Failed, 353
CTX_UnCompress_Failed, 354
CTX_Compression_Warning, 355
CTX_TableSpaceNotFound=400, 400
CTX_TableSpaceTooSmall, 401
CTX_TableSpaceNotOnline, 402
CTX_IndexTableSpaceInvalid, 403
CTX_SequenceNotFound=450, 450
CTX_HistoryRenameError=500, 500
CTX_Restore_Primary_WPA=550 550
Photo Note: New Cruise Ship Buffet 2016
The Wonderful World of Situations 