Limit the number of active data flow runs (8.5)
You can now control how many data flow runs can be active at the same time by using the max_active_runs dynamic system settings. Configure a maximum number of concurrent runs that are allowed for a node type to ensure the optimal use of system resources. For example, you can minimize the impact of multiple batch data flow runs on your system capacity.
If the limit is reached, the system queues subsequent runs and waits for active runs to stop or finish before queued runs can start. When the Data Flow service has enough capacity, which means that there are fewer active runs for the node type than the configured limit, the oldest queued runs start automatically.
Set the value to a number greater than zero to define a limit, for example, 10. You can use the special value –1 to indicate that the system can schedule an unlimited number of runs in the service.
For more information, see Limiting active data flow runs.
Previous topic Make your strategies compatible with the optimized strategy execution engine by using a check utility (8.5) Next topic Enhance your data sets with Apache HBase 2.1 and Hadoop 3.0 (8.5)