Loading pxActuals data into Visual Business Director causes high memory utilization
High memory utilization and an out-of-memory error might occur in production environments that run Pega Visual Business Director for business reporting.
This memory issue might occur after a planned node restart or when a query is submitted to Visual Business Director, and attempts to retrieve Interaction History records from the pxActuals data set and load those records into Visual Business Director for business reporting.
Cause
The root cause of this issue is physical memory overload when Pega Platform, as a result of a node restart or a query to Visual Business Director, attempts to retrieve a number of Interaction History records from the pxActuals data set for business reporting, and that number exceeds the memory capability of the node that runs the Visual Business Director service. Memory issues can occur for the following types of Visual Business Director-related system events:
- Queries that cover a specific date or time period and result in the retrieval of a substantially sized partition of Interaction History data.
- Queries without a specified time range requirement for retrieving Interaction History data (all available Interaction History data is loaded).
- Restart of a Pega Platform node that is part of the Visual Business Director service, with the Eager load mode enabled (all available Interaction History data is loaded).
- Rerunning a past Pega Customer Decision Hub campaign (for example, if you rerun a campaign that was first run three years ago, Pega Platform attempts to retrieve all Interaction History records that have been captured since the initial campaign run).
- Visual Business Director caching of a compressed version of Interaction History that contains all columns in the IH Fact table and IH Dimension tables. Application-specific columns that are added to the IH tables result in more data being cached in Visual Business Director.
The processing power that is required to retrieve large amounts of VBD data can strain multiple application resources. For example, assume that you have an IH Fact table that stores 100 million Actuals records and that the VBD server is configured to retrieve that data in the Lazy load mode (the server retrieves the data only from the time period that is specified in each query). In this setup, the VBD loads Actuals by running one database query each day. If the time period for retrieving the Actuals data generates multiple database queries, a substantial amount of data will be loaded into the memory of the VBD server. This might cause an overload of the VBD server memory and CPU, database server CPU, and the network.
Additionally, each time you access the Actuals data source from the Pega Platform, the VBD server loads all the data, provided that no start date is specified for loading the data on the Visual Business Director landing page. If the Actuals data source consists of a large number of records (for example, 100 million) and no start date for loading data is specified, resource consumption might increase and system response times might slow down.
You can limit the number of VBD queries by setting the start date for loading the Actuals data. This way, you can make the VBD data retrieval stay within your acceptable limits.
Solution: Decrease the memory requirements of Visual Business Director
- Apply the Pega Decision Strategy
Manager (DSM) cumulative hotfix that, among other enhancements,
significantly decreases the amount of memory that is required to run
Visual Business Director.For more information, contact Pega Support.
- Limit VBD memory consumption by setting a start date for data retrieval
from the pxActuals data set:You can set a date from the recent past or to a future date to disable all data retrieval. By doing so, no or only some Interaction History data will be loaded into Visual Business Director, regardless of the time range requirement that is specified in Visual Business Director queries, or Eager load mode node restarts. The Start date setting corresponds to the pxOutcomeTime property of Interaction History records in the pxActuals data set.
- In the header of Dev Studio, click .
- On the Data Sources tab of the Visual Business Director landing page, click the Expand icon next to the Actuals data source.
- In the Start Date field, click the Calendar icon, and then select the start date for retrieving Interaction History records from the pxActuals data set.
- Click Save.
- Increase the amount of physical memory that is available to your application.
Solution: Limit the number of resources required for VBD
- If you are not using the VBD Planner in your application, for example,
because you are using only the core decision management functionality as
part of the Pega-provided DMSample
application, limit the number of resources allocated to the VBD
server.For example, you can configure the Visual Business Director service to run on a node that is assigned to the Data Flow service or that processes data as part of any inbound decisioning functionality.
For information on configuring Pega Platform nodes as part of decisioning management, see Enabling decision management services.
Previous topic Troubleshooting Visual Business Director