INC-158813 · Issue 629484
Updated report handling for simulations using database output
Resolved in Pega Version 8.4.5
When running a simulation with a database table as the output destination, the pxObjClass property was not populating with a value in the results. This caused sub-reports to not be populated with data. Investigation showed that issue happened when the simulation output destination database table was inferred as an external table due to not having an exposed column for pzinskey. To resolve this, the pxObjClass and pxCreateDateTime properties, which were added to simulation output destination tables for compatibility with Report browser, will not be added to those tables when they are created. Instead, to address compatibility issues of simulation output destination classes with Report browse in the Customer Decision Hub, the pyDefaultSummaryReport has been brought up into the Data-Decision-StrategyExecution-ResultOutput class from the @baseclass.
INC-159105 · Issue 628790
Refactored resultset iteration to resolve idle alert
Resolved in Pega Version 8.4.5
Some data flow pulse tasks were executing other database queries while iterating a resultset, leading to an alert being logged indicating "Database pid [29226] in state [idle in transaction] for 15195 ms, the threshold is 5000 ms" because of the amount of time the database was being held open. This has been resolved by refactoring the tasks to make sure they don't do heavy input/output work while iterating over the resultset.
INC-159216 · Issue 638931
DSS added to create consistent handling of longform datetime
Resolved in Pega Version 8.4.5
After upgrade, a difference in handling related to datetime value was seen. For example, EmailSchedRunEndDate is a date type property holding the value "20201016T000000.000 GMT"; in Pega v7.4, a substring function was used to move the extra characters from the date field ex. EmailSchedRunStartDate = @substring(.EmailSchedRunStartDate,0,8), but in Pega v8.4 and higher the long datetime value ( "20201016T000000.000 GMT") was still being used for the date field. This long value was then truncated to 2020101+ when saving to the database, causing errors in later steps. However, research found that if there is a call @toDate function before this step for any other field, the correct date value was set for EmailSchedRunStartDate. While ClipboardPages separate Dates and DateTimes, internally, in Java, both have a time component. The implementation of DSMClipboardPage made no difference for serialization and appended the time component for pure Date properties. To create consistent handling, an update has been made to optionally set the correct behavior after setting the Dynamic System Setting by way of "Pega-DecisionEngine dsm/clipboard/correctDateFormat -> true". This setting would only take effect after a restart of Pega, and the default is false in order to not disrupt any application inadvertently relying on this behavior.
INC-159332 · Issue 629617
Logic updated for finding 'Last' in Interaction History
Resolved in Pega Version 8.4.5
The 'LastReponseDate' and 'LastInteractionId' in the Interaction History summary data set were null. Investigation showed that reusing the ESM component in the aggregation dataset caused any assumptions made during the optimization of ESM to not be true anymore. To resolve this, the logic of the Last aggregation has been modified to not rely on a Last event which might not be stored in-state (such as optimization), but rather to use the list of events which is always stored.
INC-160277 · Issue 628158
Implementation updated to ensure consistent compareDatesByDays results
Resolved in Pega Version 8.4.5
When using compareDatesByDays in a filter shape within a strategy, the results of the expression were different between the expression test and the strategy test run. This was traced to a difference in handling that became visible after upgrade; giving a negative number for the numberOfDays parameter worked differently in v7.4 than in v8.4. The SSA implementation of compareDatesByDays always worked with the absolute difference in days between dates, whereas the Java function in Pega used in expressions did not, and would compare negative differences as well. To restore the expected behavior, the SSA implementation has been adjusted to not work with absolute differences, and tests have been added to ensure equal results for the Java function in Pega and the SSA implementation.
INC-165513 · Issue 645684
Queue Processors made more robust
Resolved in Pega Version 8.4.5
After upgrade, multiple queue processors were not running as expected. Attempting to restart them generated an error. Investigation showed that the real time data flow runs were not picking up or accepting assignments because the local node was under the impression it was still processing data. In this case, the need to synchronize the state of multiple threads caused the queue processors to become stuck in an initializing state due to a race condition that caused the data flow engine to think this run still had threads running when all threads were already stopped. To resolve this, the callback handling has been simplified and made more robust. In addition, in some cases the data flow leader node would believe the service nodes did not accept assignments even when they did. This occurred if many runs and nodes were involved, and was traced to an implicit limit on the NativeSQL query used to read the data to see which assignments were accepted. To resolve this, the key-value store in the Service Registry has been modified to allow a query of more than 500 entries at once.
INC-165704 · Issue 639503
VBD data flow timeout increased and made configurable
Resolved in Pega Version 8.4.5
Intermittent VBD timeouts were seen when writing records to MSK even though no errors were reported on the MSK side. Analysis showed that while batch data flows retry when a timeout occurs, real time data flows do not retry and the configuration to wait up to 10 seconds for an acknowledgement may not be sufficient depending on the system conditions. This has been resolved by increasing the default timeout to 20 seconds and adding a configurable timeout "vbd/streamPublishTimeoutMillis" to allow a customized setting.
INC-167334 · Issue 639316
GRS support added for Kafka Key password
Resolved in Pega Version 8.4.5
An enhancement has been added to support using GRS to set values for the Kafka Key password dynamically.
INC-169125 · Issue 642399
Nodes resume correctly after DDS restart
Resolved in Pega Version 8.4.5
A corner case issue in VBD's code for handling a DDS session was preventing the nodes from recovering correctly after a system shutdown. As part of the process for an event which fires if all DDS nodes are taken down or as part of a switch from embedded to external Cassandra, VBD's cache is invalidated and then re-initialized once new VBD API calls are received or on the VBD service pulse. In this case, the invalidation of the cache did not complete due to logic in the VBD code that can lead to executing a Cassandra query that will not work in the case of all DDS nodes being down. This has been resolved by modifying the handling of a session change event to eliminate inadvertent Cassandra queries so the invalidation can complete correctly and continue the re-initialization process.
INC-169544 · Issue 649540
Enhancement for MaxEnt modeling data
Resolved in Pega Version 8.4.5
An enhancement has been added to create output for the model coefficients, the term frequency, and the inverse term frequency for use in maximum entropy modeling. For MRM processes, every Maximum Entropy (Maxent) based topic model will contain two additional stats resources. These resources can be used to validate and replicate running of topic model outside of Pega. The resources are: 1) Term Frequency file – A CSV file with all words used for training and their cumulative frequency across training set. File name format – TRAINING_DATA_TERM_FREQUENCY_< RandomNumber >.csv olumns – Word, Count 2) Coefficient file – A CSV file with all features (words, taxonomy matches and category matches) and model learnt weights for each topic across training set. File name format – MAXENT_COEFFICIENT_VALUE.csv Columns – Feature, < TopicName1 >, < TopicName2 > ,…, < TopicNameK >