Skip to main content


         This documentation site is for previous versions. Visit our new documentation site for current releases.      
 

Investigating data flow run failures

Updated on July 1, 2022

From the Data Flow landing page, you can access detailed reports on any errors that occur while a data flow is being processed. By analyzing these the error reports, you can quickly diagnose the root cause of an error.

Understanding error messages

Each error message provides information about the data flow run in which the error occurred and a link to a stack trace. You can review the method calls in the stack trace to understand the point at which your app encounters an error and which exceptions are raised.

  1. In the header of Dev Studio, click ConfigureDecisioningDecisionsData Flows.
  2. Click the appropriate tab for the type of data flow run that you are investigating.
    • Batch processing
    • Real-time processing
    • Single case processing
  3. In the list of runs, open the failed data flow run by clicking its name.
    In the following example, the data flow run failed because the number of records in which failures occurred exceeded the threshold:
    An error message for a failed data flow run
    The message says that run failed because it exceeds the maximum number of failed records, which is currently set to 0.
  4. In the error message for the failed run, click Details.
    Result: The stack trace is displayed in a new window, with additional information about the error, as in the following example:

    com.pega.dsm.dnode.impl.dataflow.resilience.policy.MaxFailedRecordsPerRunPolicy$ErrorThresholdReachedException: The run failed, because it exceeds the maximum number of failed records, which is currently set to 1

  5. Optional: To improve the readability of the stack trace, right-click inside the error window, and then click View page source.

Viewing failure counts across components

You can view the number of failed records for each component in a data flow to detect which components are the source of errors.

  1. On the data flow run tab, click the Component statistics tab.
  2. Review the component statistics to understand which data flow component registered failures.
  3. For a component in which errors occurred, click the number in the Failed records column.
    In the following example, all records failed at the filter component:
    Component statistics
    The number of failed records for the filter component is 350.
    Result: A report is displayed, showing the exceptions that the system encountered while running the data flow and processing records.

Viewing failure counts across nodes

If you encounter an error, you can view on which node the error occurred. You can also view the throughput for each node, which might give an indication as to which node is the slowest.

  1. On the data flow run tab, click the Distribution details tab.
  2. Review the node statistics to understand which node registered failures.
  3. For a node on which errors occurred and which you want to inspect, click the number in the Failures column.
    In the following example, errors occurred on both nodes that processed records in the data flow run:
    Distribution statistics
    The first node has 250 record failures. The second node has 100 failures
    Result: A report is displayed, showing the exceptions that the system encountered while running the data flow and processing records.

Viewing exceptions

You have identified the components and nodes on which failures occurred during the data flow run. You can now examine the exceptions that were thrown when your app processed each record that failed.

Before you begin: Display a report that lists all the exceptions for a data flow component or a node.
  1. In the Show last preserved data flow errors window, review the exceptions.

    For each record, the report provides information about the exception, such as the data flow stage at which the exception was thrown and the node that processed the record:

    Report with exceptions
    A sample report contains all the records in which errors occurred during a data flow run.
  2. Choose a record that you want to inspect, and then click View details.
    Result: The stack trace lists the method calls that were in progress when an exception was thrown. This information can help you understand at which point your application ran the error occurred:
    Exception details
    The stack trace lists the method calls that the application was running when it encountered an exception.
  3. In the Show last preserved data flow errors window, for a record that you want to inspect, click View input record.
    Result: Details about the record that caused the error are displayed:
    Input record
    The input record that caused the error is called test data flow life cycle.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us