SR-D85558 · Issue 548285
Handling added for prolonged Heartbeat Update Queries
Resolved in Pega Version 8.3.3
After restart, the pyFTSIncrementalIndexer queue size had hundreds of thousands of entries even though it was empty prior to the restart.Investigation traced this to a job scheduler that checked all the database connections everyday at 1 EST by using a list that contained some connections which did not exist. Checking those invalid connections caused other update queries to queue and wait, resulting in the update heartbeat query taking longer than its default beat. This caused a Split Brain issue wherein other nodes considered the long-executing node to be dead and triggered a rebalance while the node itself continued to execute partitions thinking that it was healthy. This caused duplicate processing of records. To resolve this, a fail safe has been added: while updating heartbeat in Service Registry, nodes will enter safe mode when the update query is taking longer than the default beat.
SR-D82727 · Issue 547722
Improved management for table pr_log_dataflow_events
Resolved in Pega Version 8.3.3
The Lifecycle event table was sometimes growing too large. This additional strain of database transaction volume caused poor performance on the Dataflow tier and lead to cluster instability and time-consuming cluster restarts. Due to problems in one of the Pulse tasks, the Pulse thread was not processing single case metrics properly and causing the unbounded queue for single case to grow. This has been addressed by switching to a fixed queue size, which is configurable with the DSS: dnode/single_case_queue_size. The default value of the DSS is 4000, and if changed a system restart is required. An error will be logged each 1000 queue misses, and metrics will be dropped if the queue is full. In addition, the Pulse task frequency has been improved and managed to prevent interference with other Pulse tasks and will be triggered only if a run is system-paused for a long interval. Rebalances now have a failsafe if something unexpected happens during the Pausing of the run, and If the cluster becomes unstable, the life cycle events logs may be disabled with dataflow/run/events/persist .
SR-D89643 · Issue 548290
Old Tumbling Time data in event strategies given TTL for cleanup
Resolved in Pega Version 8.3.3
Old Tumbling Time data keys in event strategies were not being cleaned up, causing Cassandra timeouts after the dataflow run had been running for several months. The longer the dataflow was running using standard compaction, the more the data was potentially spread out across SSTables and the slower it became. This has been resolved by adding a 'time to live' value for tumbling time windows, and event strategies has been switched to use leveled compaction by default.
SR-D86469 · Issue 551895
Performance improvements for Test Strategy data flow
Resolved in Pega Version 8.3.3
In the Test Strategy panel under Single case -> "Settings", selecting the "Data flow" option and choosing CustomerData dataflow was taking an excessive amount of time to run on a system with an extremely large database. To improve performance, two areas have been addressed: 1) the default behavior for record key suggestions in the test panel has been modified to collect only the ID as the additional data is not necessary at that time; 2) a DSS has been added that will opt out of reading and collecting the customer IDs in order to minimize data stored on the clipboard.
SR-D82150 · Issue 546530
Properties persist after revision import
Resolved in Pega Version 8.3.3
When properties were added to a form, they sometimes disappeared after revision import even though the properties always remained visible in a strategy. Investigation showed that the Form changes made in a DD rule were not properly updated in the pyEditElement section. To correct this, logic has been added to handle the form changes in the DD in the following activities:Data-Rule-Summary.pyMergePropositions Data-Rule-Summary.pyRemoveDDRules Data-Rule-Summary.pyWithdrawDDChanges Data-Rule-Summary.pyRemoveRestoreDDRule
SR-D94409 · Issue 553302
Strategy Explain message includes Component ID
Resolved in Pega Version 8.3.3
The Component ID has been restored to the Explain mode message for strategy results.
SR-D85095 · Issue 546338
Updated COUNT logic for strategies with ssavm set to true
Resolved in Pega Version 8.3.3
An error was seen when attempting to save a strategy after setting ssavm to true, indicating an issue in the “COUNT” method in Group By shape. Since the source field is not used and does not need to be evaluated here, the system has been updated to ignore the source field if the operation is COUNT.
SR-D74534 · Issue 538943
Decision table and tree obey allow missing properties checkbox
Resolved in Pega Version 8.3.3
Calling the decision table/tree inside another decision table/tree which has a missing property was throwing an error even though the "AllowMissingProperties" checkbox was enabled in the caller activity. Analysis showed that the setting for AllowMissingProperties did not carry into any subsequent DecisionTree/DecisionTable calls from the top DecisionTree/DecisionTable, and this has been resolved by modifying the decision table and decision tree obtain value functions to pass AllowMissingProperties.
SR-D68707 · Issue 529871
Update Handler will not run during migration
Resolved in Pega Version 8.4.1
Rolling restart of DataFlow, ADM ,VBD, and Util Tiers failed with a PENDING_JOINING error after an in-place upgrade. This was traced to the logic for the update timing: when nodes start after an upgrade from 7.x to 8.x they will migrate data flow runs. Migration happens on only one node, and while it's in progress the other nodes will wait until migration finishes before they come up. At this point the state of the data flow services will be 'PENDING JOINING'. The issue is that while migrating runs, the Data Flow Update Handler was triggered to validate whether there were nodes available on the service the run belongs to. This call can cause the corresponding data flow service to be initialized, but the call will be blocked since all services wait for the migration to end. This resulted in a deadlock which prevented all nodes from coming up successfully. To resolve this, the process has been updated to skip the update handler during migration to avoid triggering the initialization of client services that are waiting on the migration lock.
SR-D74117 · Issue 539463
DDS service will not run Hazelcast check if external Cassandra is configured
Resolved in Pega Version 8.4.1
Services were not responding, and thread dumps seen in the logs indicated that a large number of threads were waiting for one to come back from getting the cluster state for a DSM process. Investigation showed that the threads were waiting for a Hazelcast response about the cluster state. However, since a Hazelcast call is not needed when Pega is configured with external Cassandra, the DDS Service code has been changed to not to check for candidate nodes if configured with external Cassandra cluster.