Skip to main content


         This documentation site is for previous versions. Visit our new documentation site for current releases.      
 

Triggering the customer data ingestion process

Updated on August 3, 2022

This content applies only to Pega Cloud environments

For each data flow in your data ingestion process, configure an activity that invokes the pxRunDDFWithProgressPage activity and passes the correct parameters to run the data flow.

Tip: The pxRunDDFWithProgressPage uses the data flow class and name as parameters. To configure a generic structure to support any file ingestion, map the ProcessType field in the manifest file to the data flow. This mapping simplifies the development work when a new file ingestion type needs to be built. By mapping the ProcessType field to the data flow, all you need to do to support the new file ingestion type is to create a data flow and a data set, and update the mapping.
Note: Processing of the ingestion case must wait for the data flow execution to complete before proceeding to the next stage.

For information, see Processing data with data flows and Creating an activity.

Pega Customer Decision Hub Implementation Guide
  1. In the header of Dev Studio, click CreateTechnicalActivity.
  2. Provide a label, class, and ruleset for the activity.
  3. Click Create and open.
  4. On the Pages & Classes tab, add pages with these settings:
    Page nameClass
    RunOptionsPageData-Decision-DDF-RunOptions
    ProgressData-Decision-DDF-Progress

    The Progress page contains details from the data flow run that can be used for reporting and validation.

  5. On the Steps tab, configure the activity steps as in the following example.
    Activity steps configuration example
    Activity configuration in Dev Studio showing steps 1 through 4

    Step 2

    Method: Property-SetMethod parameters:
    PropertiesNamePropertiesValue
    RunOptionsPage.pyAppliesToThe class to which your data flow applies
    RunOptionsPage.pyRuleNameThe name of your data flow

    Step 3

    Method: Call pxRunDDFWithProgressPageStep page:RunOptionsPage

    Step 4

    Method: Property-SetStep page:ProgressMethod parameters:
    PropertiesNamePropertiesValue
    local.dfstatus@(PegaMKT-Engine:PegaMKTUtilities).monitorDFProgress(tools,param.sleepTime)
    Param.DataFlowIDProgress.pyWorkItemID
    Progress.pyStatus
    Progress.pyProcessedRecords
    Progress.pyDuration
    monitorDFProgress
    This function forces the activity to wait till the data flow run is over.
    Progress.pyWorkItemID
    The data flow run instance ID. You can view this ID on the Batch processing data flow landing page.
    Progress.pyStatus
    The status of the data flow run. If the status is Failed, you m want to jump to Error Stage.
    Progress.pyProcessedRecords
    The number of records processed by a data flow run. This number needs to match the number of records in the manifest file (totalRecordCount) to confirm whether the data flow was able to read all the records from the file or not. Depending on your requirements, if these numbers do not match, you may or may not want to jump to Error Stage.
    Progress.pyDuration
    The duration of the data flow run.
    Note: You can copy these properties into your work object for reporting purposes. You can also copy the entire Progress page into your work object for debugging purposes.
  6. Configure additional steps for copying the data from the data flow run into a work page and for the required error handling.
    Additional steps configuration example
    Activity configuration in Dev Studio showing additional steps 5 through 12
  7. Click Save.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us