SR-D96836 · Issue 555749
Refinements made to MarkerNode memory use
Resolved in Pega Version 8.5
Significant memory usage can be observed when data join in strategies are missing the join conditions. E.g., when the primary source in the strategy can have up to 300 or even more propositions, the join without the join condition would perform a cartesian product with possibly up to 400 or even more records returned. This may cause performance degradation. To guard against this, an update has been made that seeks to prevent repetitive markers being accumulated under a MarkerNode while trying not to rely on any implementation of equality and hashCode for individual marker implementations.
SR-D89428 · Issue 550392
Data Flow StartTime uses locale timezone
Resolved in Pega Version 8.5
The start time of the dataflow was displayed in GMT instead of the operator locale timezone. This has been corrected.
SR-D77157 · Issue 544472
DataSet preview will use date instead of datetime
Resolved in Pega Version 8.5
While using a DataSet preview functionality, the date appeared as reduced by one day. This has been resolved by parsing date as 'date' instead of 'datetime' to avoid issues with timezone interactions.
SR-D89012 · Issue 550800
DelegatedRules refresh icon made accessible
Resolved in Pega Version 8.5
When using Accessibility, the refresh icon in pzDelegatedRules was being read as "Link". This has been corrected by adding text for the refresh icon.
INC-218145 · Issue 715678
DSS introduced to control DSM clipboard page serialization
Resolved in Pega Version 8.7.3
When using a Kafka dataset to consume a message from an external topic that had an attribute name with a special character contained in a page list structure, using a JSON data transform for the mapping in a realtime dataflow resulted in the error "Exception in stage: KafkaDS; LegacyModelAspectInvokableRuleContainer.invoke-Exception encountered a :java.lang.UnsupportedOperationException." To resolve this, a new DSS dataset/CLASS_NAME/DATASET_NAME/JSONDataTransform/deserialization/useDSMPage has been introduced. When the value is set to true, the process will follow the previous behavior of DSM clipboard pages being generated when Kafka records are deserialized using JSON data transform. When the value is set to false, the JSON data transform will generate regular clipboard pages and convert them later to DSM clipboard pages. This would avoid errors when a JSON data transform calls methods from the Clipboard API that are not implemented by DSM pages. This DSS is set per data set instance. CLASS_NAME and DATASET_NAME are placeholders which should be replaced by data set's pyClassName and pyPurpose property values. In addition, a similar DSS, dataset/CLASS_NAME/DATASET_NAME/JSONDataTransform/serialization/useDSMPage, has been introduced for serialization.
SR-D85095 · Issue 546341
Updated COUNT logic for strategies with ssavm set to true
Resolved in Pega Version 8.5
An error was seen when attempting to save a strategy after setting ssavm to true, indicating an issue in the “COUNT” method in Group By shape. Since the source field is not used and does not need to be evaluated here, the system has been updated to ignore the source field if the operation is COUNT.
SR-D66397 · Issue 530333
ADM out-of-sync corrected for multi-datacenter Cassandra cluster
Resolved in Pega Version 8.5
After setting up the multi-datacenter configuration for a Cassandra cluster that consisted of six nodes in datacenter 1 and three nodes in datacenter 2, failover testing revealed a mismatch in the number of ADM models stored in each datacenter. The mismatch was observed mostly in the number of records present in the "adm_scoringmodel" and "adm_response_commit_log_date_tiered" tables. When Cassandra nodes are down, the other nodes in the cluster will store hints (records to be written) for the down nodes. When these nodes come back online the hints are replayed to those nodes and the data is written. Hints are written for 3 hours, so if a node come back up within 3 hours data is recovered and repairs are not required. The gc_grace_seconds for the above tables that were getting out of sync across the two datacenters was set to zero seconds. The "gc_grace_seconds" attribute is not just used as the time for removal of tombstones, it's also used to set the TTL for records written to the system.hints table. That meant that when the hints were written for the ADM tables for the nodes that were down, they were immediately expired since it was set to 0 and not played back when the terminated nodes restarted and joined the cluster. This has been resolved with this fix for all customers new to this release. Existing customers already on v7.3 or higher will need to complete the local change detailed below: Connect to the Cassandra cluster using cqlsh in the Pega Cassandra distribution and then run ALTER TABLE adm_commitlog.adm_response_commit_log_date_tiered WITH gc_grace_seconds = 86400; to change the relevant setting from zero to the equivalent of one day - the same length of time that the data in the table lives for. This will mean that any hints written can still be used to replay data to another node while the data itself is alive. It does also mean, however, that, given a constant load, a day's worth of expired ADM event data in the table will always be present on the disk, as the tombstones can now not be cleaned up for a day.
INC-218172 · Issue 716398
Text analytics character limit set to avoid memory issues
Resolved in Pega Version 8.7.3
Utility nodes were unstable related to searching, and email listener threads became stuck during Rule-based Text Annotation (RUTA) and natural language processing (NLP) work on incoming emails. This happened when the system experienced high memory consumption or exceeded memory usage when using text analytics. This has been resolved by setting the default maximum character limit for NLP analysis to 25,000 characters to avoid RUTA memory issues. If text is provided > 25,000 characters, the system will consider only the top 25,000 characters and a flag will appear on NLPOutcome to indicate text has been limited. This character limit is configurable, but if the configuration is set in excess of 25,000 a warning will be shown prior to saving the change.
INC-223376 · Issue 723575
JMX authentication enabled by default for embedded Kafka and Cassandra
Resolved in Pega Version 8.7.3
For on-premises clients, a potential vulnerability for a Remote Code Execution using the JMX interface on Cassandra and Kafka using exposed network ports has been mitigated by enabling JMX authentication by default for embedded Kafka and Cassandra.
SR-D85558 · Issue 548286
Handling added for prolonged Heartbeat Update Queries
Resolved in Pega Version 8.5
After restart, the pyFTSIncrementalIndexer queue size had hundreds of thousands of entries even though it was empty prior to the restart. Investigation traced this to a job scheduler that checked all the database connections everyday at 1 EST by using a list that contained some connections which did not exist. Checking those invalid connections caused other update queries to queue and wait, resulting in the update heartbeat query taking longer than its default beat. This caused a Split Brain issue wherein other nodes considered the long-executing node to be dead and triggered a rebalance while the node itself continued to execute partitions thinking that it was healthy. This caused duplicate processing of records. To resolve this, a fail safe has been added: while updating heartbeat in Service Registry, nodes will enter safe mode when the update query is taking longer than the default beat.