Configuring a selective data extract
With this approach, the size of the Policy Profiles is reduced by stripping off any unwanted data, such as any item metadata or document references that do not need to be extracted for your reporting or data mining needs. Because the size of the Policy Profiles is reduced, this approach allows you to have less frequent extracts with a large number of Policy Profiles, while keeping under control both the duration of the extraction process, and the size of extracted data.
Before following this approach, it is important to understand that all the data stored in the Policy Profile is required by the system for a range of different purposes (for example, for the initialization of new KYC cases, or for the purposes of audit). Therefore, manipulating the data to reduce the size of the Policy Profile cannot be done directly on the Policy Profile itself, but takes place on an alternate storage area with a trimmed-down version of the Policy Profiles that can be used subsequently for extraction.
You can generate the reduced version of the data by creating the assets described below. Use the steps defined in Configuring a full extract as the basis of the process, but apply the steps that are specific to configuring a selective data extract.
Creating storage for extraction
Create a concrete class with a database table mapped to it. To avoid the creation of properties that may be already present, you should extend this class from the PegaKYC -Data -PolicyProfile class, and you must copy its associated database table from the one used by the Policy Profile class. This ensures maximum reuse of the assets from the PolicyProfile class, as well as a physical separation of extraction and application needs.
Populating the storage for extraction
Once the storage class is ready, the next step is to populate this storage with the copies of the Policy Profiles. This must be done in the background to avoid any performance impact on case processing. Create a declare trigger on the Policy Profile class and configure the trigger to execute on commit and in asynchronous mode (In background on copy).
The trigger invokes to an activity that can copy the policy profile to the new table you created before and, during that process, cleans up the metadata by iterating over the Types and deleting the Items property list. Once the trimmed-down version of the Policy Profile is complete, the activity saves the new instance into the database so it can be used later by the extraction process.
Creating an extract rule and job scheduler
As the last step, you must create an extractor rule in the new class, and this rule must be executed through a job scheduler following the general guidelines.For more information, see Configuring a full extract section.
Previous topic Configuring a full extract