To achieve high data consumption throughput and concurrency, the Stream service uses the Kafka concept of partitions. Starting in Pega Platform version 8.6.4, the default number of partitions per topic is six, which means that up to six clients can simultaneously consume data from a Stream data set.

In most of the Pega Platform versions it is possible to customize the default number of partitions. For more information, see The support for default partition number customization.

In Pega Platform versions earlier than 8.6.4, the default number of partitions was set to 20. After a system update to version 8.7, the number of partitions of existing Stream data sets does not change. However, if you truncate the pr_data_stream_nodes table, the number of partitions reverts to the new default of six.

In most cases, six partitions is enough for effective processing. However, if your application needs a higher consumption throughput and thus requires more than six processing nodes, you can increase the number of stream partitions. Ensure that the number of partitions per broker does not exceed the limit.

To change the global default number of partitions in the cluster, create and configure the following dynamic system setting: