To achieve high data consumption throughput and concurrency, the Stream service
uses the Kafka concept of partitions. Starting in Pega Platform
version 8.6.4, the default number of partitions per topic is six, which means that up to
six clients can simultaneously consume data from a Stream data set.
In most of the Pega Platform versions it is possible to customize the
default number of partitions. For more information, see The support for default partition number customization.
In Pega Platform versions earlier than 8.6.4, the default number of
partitions was set to 20. After a system update to version 8.7, the number of partitions
of existing Stream data sets does not change. However, if you truncate the pr_data_stream_nodes table, the number of
partitions reverts to the new default of six.
In most cases, six partitions is enough for effective processing. However, if your
application needs a higher consumption throughput and thus requires more than six
processing nodes, you can increase the number of stream partitions. Ensure that the
number of partitions per broker does not exceed the limit.
To change the global default number of partitions in the cluster, create and configure
the following dynamic system setting:
- In the header of Dev Studio, click .
- On the New tab, configure the following
- In the Dynamic system settings short description
field, enter the purpose of this setting.
- In the Owning Ruleset field, enter:
- In the Setting Purpose field, enter:
- Click Create and open.
- In the Value field, enter the new global default
partitions count per topic.
- Click Save.
- Restart the server.