INC-130304 · Issue 567924
Retry logic added for downloading upgraded rules
Resolved in Pega Version 8.1.9
A Rules upgrade failed while downloading applications from the maintenance server due to an SFTP server connection failure. This has been resolved by adding logic to retry if the first connection attempt fails.
INC-130695 · Issue 587659
Enhancements for upgrading in multi-tenant environment
Resolved in Pega Version 8.1.9
Some muti-tenant installations use the same applications or rule instances with the same pzInsKeys for different tenants. This can cause upgrades to time out due to the system fetching all pzInsKeys (which will have duplicates) and working with them in a default batch size of 500 each over 4 threads. This led to the same keys potentially being allocated and processed in different threads, resulting in duplicate processing and timeouts. This has been resolved by updating the select query to fetch the tentantid and pzInskeys in the MT system to avoid duplicate work in multiple threads. In addition, running Generate Declarative indexes fetches the pzinskeys and generates indexes for each record, but before generating, the existing index for the record is deleted and then inserted. Because the delete query to generate the index was not tenant aware, all of the records for the key were deleted for the tenants for that key, but the new index was created only in one tenant. This has been resolved by enhancing the DELETE query to be tenant aware, which will avoid deleting the indexes for all the tenants given an index key.
INC-132218 · Issue 573359
Resolved buffer overflow for Migration loadDatabase
Resolved in Pega Version 8.1.9
A Rules upgrade failed in the Migration step at loadDatabase stage, involving the move of all the table records from old schema to new schema. This was traced to the inability of Migration to load blob of sizes more than 100 MB, and has been resolved by updating Migration to use byte[] to read the blob content with the help of metadata that contains blob length.
INC-133202 · Issue 574701
TableRenameUtil hashing improved
Resolved in Pega Version 8.1.9
During index name generation, the algorithm that was responsible for index name uniqueness was sometimes insufficient and cerated a loop condition. This has been resolved by using a stronger hash algorithm and refactoring the code that could result in a loop.
SR-D84364 · Issue 551402
Check for circular references added to SearchInventoryImpl to prevent recursive call
Resolved in Pega Version 8.1.9
An out of memory error was traced to SearchInventoryImpl infinitely recursing over a clipboard property, where the child property referenced a parent property and resulted in an endless loop. This has been resolved with the addition of a depth check to ensure that the search does not recurse infinitely.
INC-167311 · Issue 646477
Updated upgrade handling for migrating work objects
Resolved in Pega Version 8.3.6
After upgrading from Pega 6.2 to 8.3, the work migrated work objects were missing SLAs due to missed entries in the assignment tables (PC_ASSIGN_WORKLIST/ PC_ASSIGN_WORKBASKET) . The SLA was firing, but the processing failed due to the fact the runtime could not resolve a 'AddHistoryPage' library function. In this case, multiple upgrades of the application dating back to Pega 4 resulted in the runtime context containing older ruleset versions in higher ruleset versions, hiding the underlying Pega 8 version of the rule. For releases prior to Pega 7.3, Rule-Application was stored in pr4_rule and will be migrated to pr4_rule_application during upgrades. However, since Context Upgrade is run before Optimize Newly Exposed Columns, the pyDependsOnName won't always be populated. To resolve this, the system will filter based on the value in the blob rather than the exposed column so there will be a value regardless of the upgrade-from version.
INC-172675 · Issue 649451
Configuration added for extending queue processor timeout
Resolved in Pega Version 8.3.6
Alerts for queue processor (QP) items which took more than 15 minutes to run could result in the system marking the node as 'unhealthy'. In environments with Pega Health Check enabled, this would shut down the node gracefully. It was not possible to change this default as it was hardcoded. In order to support systems that may have custom processes that run beyond 15 minutes, a a new setting has been exposed that allows configuration of the interval after which a node with long-running queue processor is marked as unhealthy and is restarted. By default this remains 900000 milliseconds / 900 seconds / 15 minutes, but it may be adjusted up to 24 hours to avoid premature node shutdown. The stale thread detection mechanism will take that setting into account and use the provided value or default to 15 minutes if the value was not provided. In addition, the threshold's units in the UI have been changed from ms to seconds.
SR-D37317 · Issue 512553
Run Ruleset Cleanup defaults to true
Resolved in Pega Version 8.2.6
After upgrade, the rule categories and rules were not showing correctly in the App view of the Dev Portal. Many warning messages were also logged related to the Decisioning DM Sample application. This was traced to the rules cleanup script not running properly. While there was a workaround of applying the ruleset cleanup scripts manually after removing the queries that reference the pr_engineclasses table, the cleanup will now be set to run by default (run.ruleset.cleanup=true). In addition, the logic to determine which RuleSets to include has been simplified and most of the pr4_rule_vw deletions have been combined.
SR-D38318 · Issue 519710
Data pages explicitly cleared after QP use
Resolved in Pega Version 8.2.6
The Util Node was showing as Offline in the Search Landing Page, and when Jobs were submitted for execution from other Nodes the message "Detected active run with unreachable nodes" was logged. The util node, configured as a backgroundprocessing node, was running QPs; the queue size for custom QPs is 500 messages /queue items per minute, but investigation showed the requestor level and thread level data pages corresponding to the QP activities were not being cleared after use. This led to high heap memory issues that made the node unreachable, and has been resolved by adding code to explicitly remove the data pages when processing has finished.
SR-D48396 · Issue 520423
Hazelcast upgraded to resolve node startup issue
Resolved in Pega Version 8.2.6
Post data upgrade, the ADM tier failed to start and the error "java.lang.IllegalStateException: Node failed to start!" appeared. This was traced to a dormant bug in Hazelcast 3.11 that caused starting nodes to fail when the Hazelcast master node was shutting down, which was exposed by recent Pega changes made to enable parallel restarts of nodes in Cloud environments. Hazelcast delivered a fix for the parallel restart problem and the hotfixed jar has been merged into the platform. In addition, previous logic for loading Admin Studio waited 30 seconds before timing out when fetching information for each node. This caused issues with large clusters and Admin Studio not loading. The logic has been updated in the Admin Studio UI to load the page despite delays/issues waiting for nodes to respond to the gathering of cluster data, and the algorithm to detect remote-call timeout has been updated and is applicable to batch operation.