Understanding agents and agent queues is critical to managing your Pega^® deployment effectively. This article explains Pega Platform 7.3 agent queues and how they work, particularly with the two new features supporting clustered (multi-node) environments: Node Classification and Node Types.

If your Pega deployment has one-to-many nodes in a cluster, you can manage the processing of queue items for Data-Agent-Queue and Rule-Agent-Queue class instances across the entire cluster. Learn how classifying nodes and specifying Node Types simplifies how you manage a Pega cluster of servers.

• Prerequisites
• Node Classification
• Node Types
• Classifying nodes using Pega Cluster Visualization (Application Server)
  • WebUser
  • BackgroundProcessing
  • Search
  • RunOnAllNodes
• Node Types provided by Pega 7.3
• Defining Nodes Types and specifying JVM parameter syntax (Application Server)
• Updating Agents rule and data instances (Designer Studio)
  • Mapping Rule-Agent-Queue instances to Node Types (Designer Studio)
  • Assigning Rule-Agent-Queue instances to Categories (Designer Studio)
• Agent Queues: Before and After examples (Pega Cluster Virtualization)
  • Pre-Pega 7.3 example
  • Pega 7.3 example
• Deleting old Data-Agent-Queue instances after upgrading to Pega 7.3
• Pega 7.3 changes to Agents rules: New, Obsolete, and Deprecated rules
  • New rules introduced in Pega 7.3
  • Obsolete rule
  • Deprecated rules

Prerequisites

Before you read this article, you should have fundamental knowledge of the Pega Platform 7, especially the following Pega 7 concepts, tasks, and reference information:

• How to develop rules for an application
• Pega Platform 7.2 Agent concepts, terms, and related topics
• What agents are available in Pega 7.2.2 and how to upgrade to Pega 7.3
• How to create and schedule Agents in Pega 7.2.2 on multiple nodes of a cluster (two or more Web Application Servers)
• How to create, modify, and schedule Pega 7 Agent Queues

Node Classification

Beginning with Pega 7.3, you can optimize performance and provide higher scalability and stability in a Pega cluster of servers by using node classification. When you classify nodes, you separate the nodes, segregating them by purpose and defining their behavior.

For example, determine whether a node is serving up requests for real-time application users, handling integration service calls, performing indexing (Elasticsearch), or managing the background processing that is needed for a given application.

To effectively use node classification, you must map agents and listeners to Node Types.

Node Types

While similar to node classification, specifying a Node Type fundamentally changes what happens internally on Pega 7.3 by augmenting the startup process and controlling where (on which node) agents persist. The example that follows illustrates the use of four Node Types.

A BackgroundProcessing Node Type will have most of the agents enabled, whereas the WebUser Node Type will have fewer agents enabled. And, as its name implies (“run on all the nodes in a cluster”), the RunOnAllNodes Node Type will have a subset of the two that do not need to be present for WebUser or BackgroundProcessing Node Types. The fourth Node Type is Search, which is specified for all nodes in a cluster that have the FTSIncrementalIndexer Agent enabled.

The benefit is increased performance for the real-time application users because nearly 95% of the resources are dedicated to that role.

The rest of this article explains the best practices for classifying and typing nodes in a Pega cluster to achieve optimal performance with great administrative efficiency.

Classifying nodes using Pega Cluster Visualization

Classifying the nodes of a Web Application Server is a simple process if you adhere to an effective naming convention. (A universal best practice!)

For example, you could refer to Node A, Node B, Node C, and so on. But these names are not descriptive and quickly becomes meaningless, particularly when a cluster is managing 50, 100, or more nodes.

Defining meaningful names before you get too far into configuring your Pega clustered environment is critical to your success. This process, informally referred to as Pega Cluster Visualization, is a way for you look at the system administration portion of a Pega 7 environment. Pega Cluster Visualization provides all interested administrative parties with a unique identifier for one-to-many servers in a Pega cluster.

WebUser

For example, if you plan to have a Virtual, Physical, or Logical Server that will host the Pega 7 application server and also host 5, 10, or more Web Application Servers, this grouping of servers could be classified for WebUser. The role of all the nodes running on it will be identical and share resources only to the user community accessing them. Visually, you now know what node class this server belongs to and can be applied to more servers. That is, more servers can be added to this group, extending the cluster to support more users and services as needed.

BackgroundProcessing

For the next Virtual, Physical, or Logical Server in your Pega deployment, you plan to host another 5, 10, or more Pega 7 application servers whose primary purpose is to handle all of the background processing and other agents to support the entire cluster and the application. Visually, you name, and thereby classify, this server as a BackgroundProcessing server.

Search

Another Virtual, Physical, or Logical Server in your Pega deployment might host the Elasticsearch indexing and index sharing. Visually, you name, and thereby classify, this server as a Search server.

RunOnAllNodes

The final set of servers is classified as RunOnAllNodes, which informs Pega 7.3 to run the agent on all nodes (either simultaneously or on one node only at a time) to properly handle the volume of requests. For example, in a high transactional environment where thousands of transactions take place every hour or even every minute, you may require additional processing steps to complete the transaction and also to keep up with the volume. Load balancing this process across many nodes ensures that each transaction completes on time. Splitting the workload across many nodes may often be needed; the use of RunOnAllNodes is specifically for this purpose.

Now, you have four distinct classifications, which are easily understood because they are visually and semantically named:

• WebUsers
• BackgroundProcessing
• Search
• RunOnAllNodes

Next, learn how these example node classifications and other node classifications rely on Node Types to which you map agents and agent listeners.

Node Types provided by Pega 7.3

Starting with Pega 7.3, there are a total of 11 different Node Types that you can use to define the nodes in the Pega cluster of your deployment.

There is no effect on the Dynamic System Settings or prconfig.xml settings when you use a Node Type.

1. Universal
2. WebUser
3. BackgroundProcessing
4. BIX
5. Search
6. RunOnAllNodes
   You cannot start a node with this Node Type. Agents and Listeners can be mapped to this Node Type so that they run on all nodes in a cluster.
7. Custom1
8. Custom2
9. Custom3
10. Custom4
11. Custom5

Defining Nodes Types and specifying JVM parameter syntax (Application Server)

As a best practice, ensure that the Web Application Server startup process has a new parameter passed into the JVM to force the node to start up as a specific Node Type. Adding this parameter at the JVM level eliminates the need to keep track of which node starts up first and in what order. This new parameter helps Pega 7.3 identify the Node Type and adds it into the Pega cluster automatically.

Continuing with the example node classifications defined above, here is how you specify the JVM parameters.

Other Node Types could be used for more servers of your cluster. For example, a BIX Node Type could be dedicated to processing BIX transactions.

Updating Agents rule and data instances (Designer Studio)

Now that the node is configured to start up with the specified Node Type, you must make some data and rule changes using the Designer Studio.

1. Mapping Rule-Agent-Queue instances to Node Types
2. Assigning Rule-Agent-Queue instances to Categories

Both of these changes give you the Pega Cluster Visualization (PCV) at a high level to help with administration of your cluster.

Mapping Rule-Agent-Queue instances to Node Types (Designer Studio)

First, for each agent, you need to map the Rule-Agent-Queue instance to one of the Node Types. See the Node Classifications and Node Types created in the example above. This mapping creates the associations for the scheduled agents when the nodes start up. In the example shown in the image below, the Agent activity is associated to the BackgroundProcessing Node Type.

Example of mapping an agent queue to a Node Type

If you open the Pega-SearchEngine Rule-Agent-Queue instance, you notice the FTSIncrementalIndexer Activity for that agent is associated with the Search Node Type.

Assigning Rule-Agent-Queue instances to Categories (Designer Studio)

The second change that you need to make using the Designer Studio is to assign each scheduled agent (Rule-Agent-Queue instance) to a Category that the agent has an impact on. There are currently eight (8) Categories that each agent can be mapped to.

Do not worry: There is no change to behavior or results of the agent if it is not assigned the correct category. The Category is used later on, when you work in the Landing Pages.

The Category you select relates to any agent processing a work item or assignment. All of these agents get grouped together under the same category, making it easier to manage when your application scales up.
Select a Category, for example, Case Management

Assign the agent queue instance to the selected Category

Agent Queues: Before and After examples (Pega Cluster Virtualization)

Now for the biggest benefit of them all.

After you have defined the Node Types and used them to specify the Web Application Server JVM Parameters and mapped the Agent rule and data instances to Node Types and Categories, you can see amazing results in the Pega 7.3 Agent Queue pages compared to the Pega 7.2.2 Agent Queue pages!

Pre-Pega 7.3 example

Assume, for example, that you have 10 nodes and 80 Agent Queues in your pre-Pega 7.3 environment. You will have a list of 800 Data-Agent-Queue instances (rows) that you will have to page through to locate the agent you need. The example image below illustrates only one (of the 80) Agent Queue rulesets that is configured on all 10 nodes, which are cryptically identified.

Example list of pre-Pega 7.3 Agent Queue rulesets

There is no way to look at the rows and know for certain visually where the agent is running, other than to assume that it is running on all 10 nodes. Not precise. Not good.

Because hashed (cryptic) Node IDs are used here, it is not clear which node is on which server! This problem will clearly escalate when you need to troubleshoot an issue.

Pega 7.3 example

The example list above shows only the Pega-AppDefinition ruleset in a release prior to Pega 7.3. Multiply that example by all of the rulesets provided in Pega 7.3 and your application rulesets, where agents are defined, and the topology snapshot -- along with the administrative challenge -- becomes overwhelming very quickly.

However, the results shown for the example above are not the same with Pega 7.3. Now, with Pega 7.3, the same ruleset is mapped to the BackgroundProcessing Node Type. Having mapped the ruleset to the BackgroundProcessing Node Type, you only need to update one of the Data-Agent-Queue instances to specify the Application Access Group to use, just as you have done in earlier releases.

See the example image below: What a difference!
Less editing means faster deployments, startups, and performance.

Example list of Pega 7.3 Agent Queue rulesets

Deleting old Data-Agent-Queue instances after upgrading to Pega 7.3

If you are upgrading from an earlier release to Pega 7.3, after the nodes in your Pega cluster start up for the first time and the newly defined Node Types have been created, you can delete all of the older Data-Agent-Queue instances shown with the hashed (cryptic) Node ID because they will no longer be valid or used.

Pega 7.3 has no capability to automatically remove the older Data-Agent-Queue instances, but you can use SQL to remove them directly from the database table. Once you have deleted the older instances, you are left with one (1) or more pages of Data-Agent-Queue instances compared to hundreds of pages.

Example: Run the following query to retrieve all of the old Data-Agent-Queue instances that match the node name for Pega- AppDefinition. After the old instances are found, you can delete them.

SQL> select * from pr_data where pxObjClass='Data-Agent-Queue' and pyNodeName like '%6599d77b%';

Alternative: The alternative way of deleting the now-obsolete agent queue instances is to open each one and delete it. If you do use SQL to delete the agent queue instances, follow up with a COMMIT statement to ensure that the obsolete nodes are removed.

In the System Management application (SMA), the list of agents is cleaner and smaller. The agents that have been disabled and are not running on a given node are removed. Now all you see are the agents that you associated with Node Types and Categories in the Rule-Agent-Queue instances. The display is less noisy and easier for you to locate problems faster. You can manage your Pega clustered environment more efficiently.

Pega 7.3 changes to Agents rules: New, Obsolete, and Deprecated rules

There are a number of changes to Agents rules in Pega 7.3. In addition to obsolete, deprecated, and new Agents, some Agents rules were renamed and repurposed in Pega 7.3.

Agents rules not listed here have been verified as the same (no change) between Pega 7.2.2 and Pega 7.3.

Some rulesets containing Agents rules are not documented in Pega 7.3 Help topics.

New rules introduced in Pega 7.3

The following Agent rules are newly introduced in Pega 7.3.

For complete information, see the Help topics and PDN Articles listed in Related Content.

Pega-ProcessEngine: EmailDigestDaily

This agent accumulates all Pega Social conversations and sends them in a preformatted email at the scheduled choice of the user.

The Pega 7 Help describes the notification network in detail under the Index of Case management > Case processing > Notifications.

Two new tables were introduced to store details of all events:

• pc_work_notification
• pr_data_notification_recipient

Pega-ProcessEngine: EmailDigestWeekly

See Pega-ProcessEngine: EmailDigestDaily.

Pega-ProcessEngine: MultichannelCloseExpiredInteractions

After the data is captured by the email daily or weekly digest agents, this agent cleans up all the expired interactions and clipboard pages.

Pega-RuleRefactoring: pxProcessRebase

This agent supports a new DevOps feature used when users are performing continuous integration.

This agent is currently toggled off with the pyCanRebase Rule-Obj-When rule and is being trial-tested by a limited number of users on Pega 7.3.

Pega-RulesEngine: DisableDormantOperators

This new security agent monitors operators for their last sign-on and disables operators who exceed a specified number of days of inactivity.

Pega-RulesEngine: ClusterAndDBCleaner

This agent replaces the Pega-RULES: SystemCleaner agent (first part).

Pega-RulesEngine: NodeCleaner

This agent replaces the Pega-RULES: SystemCleaner agent (second part).

Obsolete rule

Pega-DecisionEngine: UpdateAdaptiveModels is marked as obsolete and withdrawn from Pega 7.3.

Deprecated rules

The following Agent rules are deprecated in Pega 7.3:

• Pega-RULES: StaticContentCleaner
• Pega-RULES: SystemCleaner
  Replaced by ClusterAndDBCleaner and NodeCleaner agents
• Pega-RulesEngine: PersistSystemInfo