Sitworld: Use and Misuse of EPHEMERAL:Y

Moonset_3am

John Alvord, IBM Corporation

jalvord@us.ibm.com

Draft #1 – 14 December 2020 – Level 1.00000

Follow on twitter

Introduction

The ITM protocol modifier EPHEMERAL:Y is wonderful when it is used correctly. When used badly it can cause terrible issues such as communications malfunctioning and causing ITM processes like TEMS to fail and require a recycle to continue.

If you need a refresher on Protocol Modifiers, please see this document

Sitworld: ITM Protocol Usage and Protocol Modifiers

And how to make such changes

Sitworld: ITM Permanent Configuration Best Practices

The marvelous feature of a EPHEMERAL:Y configuration is that the ITM agent does not require any ITM listening ports. This greatly increases security and also insulates the ITM process from port scanning tests, which sometimes damage the ITM processing such that process recycles are needed.

Background

Most ITM Agents require TCP listening ports. In a default configuration, an agent will make a TCP connection to the TEMS it is configured to. It will also make a TCP connection when sending historical data to a Warehouse Proxy Agent. Finally it will open a listening port to allow the TEMS to give it alerts [like location of WPA] and requests for real time data. These are the port requirements for ITM Agent processes. Of course an agent may have non-ITM communication ports which are not covered here.

EPHEMERAL:Y Configuration

EPHEMERAL:Y is added to the KDC_FAMILIES or [z/OS KDE_TRANSPORT] environment variable. Usually it is added at the beginning as a global modifier. In this mode, a single Agent to TEMS TCP socket connection is made. There is no connection to a WPA and there is no listening port.

This logic is performed like a software router. The three connections are multiplexed between the usages. In certain diagnostic messages, you can make out virtual ports which are used to manage the data transmission.

EPHEMERAL:Y Requirements

The Warehouse Proxy Agent must be installed on the same system as the TEMS the agent connects to. The agent is told to connect to a WPA at a certain ip_address and a certain port… however the ip_address is ignored and the communication goes to the TEMS system. That isn’t bad since large environment best practice is to install a WPA with every hub and remote TEMS. It provides a nice workload balance. The alternative is to collect historical data at the TEMS [not great since slower and a single point of failure if the TEMS is down or communications outage. You can always not collect historical data but many customers view that as critical.

A second issue, the rule for EPHEMERAL:Y is ALL or NONE. If you have mixed usage, you will experience communication failures.

A third issue, the TEPS and the TEMS cannot run with EPHEMERAL:Y. That is because such servers need to contact the agents requesting service. Because of the ALL or NONE rule, no other ITM process on the same system as the TEMS/TEPS can use EPHEMERAL:Y. In this case the only alternative is to not run the other processes on the same system. This is true event if the other ITM process uses a different installation directory.

The communication outages are not frequent, like every few minutes. Instead they usually show as an issue every couple of days or once a week. A recycle often temporarily clears things up… but it will always return. Follow the ALL or NONE rule and the issue is avoided.

EPHEMERAL:Y Bad Experience

This mixed usage mal-configuration often occurs when different teams separately install agents. The original agents are all without EPHEMERAL:Y. The new agents use that setting and processing gradually destabilize. There was one case with 5,500 Windows systems that needed one by one correction.

EPHEMERAL:Y Detection

On Linux/Unix one easy way is to login to the system and do

cd <installdir>/logs
grep -i KDC_FAMILIES *.env

If you see some with and some without EPHEMERAL:Y you know the trouble exists. On occasion this gives a false alert because the xxx.env file is from an agent no longer running.

On Windows you usually have to review the agent diagnostic log files.

There is a limited detection capability using TEMS Audit

Sitworld: TEMS Audit Process and Tool

If you set the following trace at the TEMS the agents are connecting to

cd <installdir>/bin
./tacmd login -s ….. [to hub TEMS]
./tacmd settrace -m <remotetemsnodeid> -p KBB_RAS1 -o ‘error (comp:kde,unit:kdebp0r,Entry=”receive_vectors” all er)(comp:kde,unit:kdeprxi,Entry=”KDEP_ReceiveXID” all er)’

and to turn off or reset later

cd <installdir>/bin
./tacmd login -s ….. [to hub TEMS]
./tacmd settrace -m <remotetemsnodeid> -p KBB_RAS1 -r

TEMS Audit will detect some of the cases. It usually depends on whether the agent(s) are making new connections or not.

In practical terms you will discover a few agents mal-configured and then need to determine why it happened [like another group installing agents] and then correct them at each agent system. And of course educating the teams doing agent installs

Summary

This documents how to correctly use EPHEMERAL:Y protocol modifier.

Sitworld: Table of Contents

History and Earlier versions

1.00000

Initial publication

Photo Note:  Moon Setting 3am

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

<span>%d</span> bloggers like this: