Data flow run limits
A large number of data flow runs that are active at the same time can deplete your system resources. To ensure the efficient processing of data flows, you can configure dynamic system settings to limit the number of concurrent active data flow runs for a node type.
If the number of active runs reaches the limit, the system adds subsequent runs to a queue. When the Data Flow service has enough capacity, which means that there are fewer active runs for the node type than the configured limit, the oldest queued runs start automatically.
You can limit the number of active data flow runs by using the following dynamic system settings:
- dataflow/max_active_runs
- This setting defines the maximum number of active runs for node types that do not have their own limits configured.
- dataflow/<node_type>/max_active_runs
- A dynamic system setting in this format defines the maximum number of active runs for a particular node type. The <node_type> value is case-sensitive, for example, BackgroundProcessing.
The following table shows the max_active_runs dynamic system settings that are included with the system, with their default values. You can modify these settings or create additional settings for other node types. When the value is set to a number greater than zero, the specified limit is used. When the value is set to –1, an unlimited number of active runs is allowed.
max_active_runs dynamic system settings
| Dynamic system setting | Default limit |
| dataflow/max_active_runs | 10 |
| dataflow/Batch/max_active_runs | 10 |
| dataflow/BackgroundProcessing/max_active_runs | –1 |
| dataflow/RealTime/max_active_runs | –1 |
| dataflow/Search/max_active_runs | –1 |
- Limiting active data flow runs
Ensure the optimal use of system resources by limiting the number of concurrent active data flow runs that are allowed for a node type, such as Batch or Search. If the limit is reached, the system queues subsequent runs and waits for active runs to stop or finish before queued runs can start.
Previous topic Managing data flow runs Next topic Limiting active data flow runs