Skip to main content

         This documentation site is for previous versions. Visit our new documentation site for current releases.      

Managing data flow runs

Updated on July 5, 2022

Control record processing in your application by starting, stopping, or restarting data flows. Monitor data flow status to achieve a better understanding of data flow performance.

  1. In the header of Dev Studio, click ConfigureDecisioningDecisionsData Flows.
  2. On the Data Flows landing page, select the data flow type that you want to manage:
    • To manage data flows that use a non-stream data set as the main input, click the Batch processing tab.
    • To manage data flows that use a streaming data set (such as Kafka, Kinesis, or Stream) as the main input, click the Real-time processing tab.
    • To manage data flows that are triggered in the single case mode from the DataFlow-Execute method, click the Single case processing tab.
  3. In the Action column, click Manage.
  4. In the Manage list, select whether you want to Start, Stop, Continue or Restart a data flow run.
    The available actions depend on the current data flow run status. For example, if a data flow run status is Completed, the available actions include Restart.
  5. Optional: To display detailed information about the data flow run, click a run ID in the ID column.
    You can view the following details:
    Component statistics
    View the number of successful and failed records per data flow component and the average processing time (in milliseconds). You can also view the percentage of the total processing time that your application took to process each component.
    Distribution details
    View the number of data flow nodes that were assigned to process the data. You can also view the number of partitions that were created to process the data in each decision data node. The statistics display the number of records processed by each node, the number of failed records, and the current status of the node.
    Run details
    The default parameters of the run.
    1. Optional: To display detailed information about partitions, on the Distribution details tab, click the number in the # Partitions field.
      Partitions count in a data flow run
      The number of partitions for the Write Data To Interaction Files is six.

      The partitions report displays the metrics for individual partitions.

      If you changed the number of partitions of a topic associated with a queue processor or a streaming data set (Kafka, Kinesis, or Stream), the related data flow run automatically detects the change and adjusts the metrics accordingly. The Intention column contains messages for added and deleted partitions.

      If you added partitions, the new partitions are added to the table and marked to be picked up:

      Partitions report with partitions marked to be picked up
      The report displays four new partitions. In the Intention column, these partitions are marked as Pick up new.

      If you removed partitions, these partitions are stopped and marked for deletion:

      Partitions report with partitions marked for deletion
      The report displays five deleted partitions. In the Intention column, these partitions are marked as Marked for deletion.

      For previously active partitions that were removed, a separate entry is added to the table. This entry provides aggregated metrics of previous partitions: the number of records that these partitions processed when they were active and the number of errors that these partitions had.

      Partitions report with aggregated metrics of previous partitions
      According to the aggregated metrics of previously active partitions, four records were processed by these partitions.
  6. Optional: To display the data flow configuration, click the name of a data flow rule in the Data flow column.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best. is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us