Operating the Stream service
Learn how to do a rolling and full restart of your Stream service, to manually scale up and scale down the service, and to recover the service after a failure.
Starting in version 8.7, the use of an internal Kafka is deprecated. On-premises systems that have been updated from earlier versions of Pega Platform can continue to use Kafka in embedded mode. However, to ensure future compatibility, do not create any new environments using embedded Kafka. When configuring the Stream service in a new environment, use external Kafka.
When the Stream service is running in External Mode, there are no Stream Tier instances and none of the procedures mentioned below are required.
Rolling restart Stream Tier
The Stream tier can be "rehydrated" by restarting one instance at a time. Terminate each instance in a graceful fashion. It may take up to 10 minutes to shut down a Stream instance as the data ownership has to be handed over to the remaining nodes. Do not forcefully terminate instances during this period.
When an instance comes back online, wait for a grace period of 5 minutes before you restart the next instance. During the grace period, the Stream node replicates the data which might have been missed during the period that the instance was unreachable.
Full restart Stream Tier
In case a rolling restart is not possible for some reason, you can do a full restart.
Take the nodes out of service one instance at a time. Each instance has to be terminated in a graceful fashion. It make take up to 10 minutes to shut down a Stream instance as the data ownership has to be handed over to the remaining nodes. Do not forcefully terminate instances during this period.
Bring instances back up in reverse order.
Scaling up Stream Tier
Stream service does not support automatic scaling, however, a manual scale-up is possible.
- In the header of Dev Studio, click .
- Ensure that all Stream instances are up and have the status NORMAL.
- Ensure that under-replicated and offline partitions are 0 for all instances in the list.
- If under-replicated and offline partitions are greater than 0, wait until they
get to 0.
Scaling up the Stream service - Start an additional Stream instance.
Note: You can add only one instance at a time. Repeat the procedure if you need to add more instances.
Scaling down Stream Tier
Stream service does not support automatic scaling, however a manual scale-down is possible:
- In the header of Dev Studio, click .
- Select a node you would like to take out of the service, and click .
- Wait until the instance is removed from the list.
- Terminate the node.
Full recovery Stream Tier
In case of a catastrophic failure, the following procedure can be followed to recover
the Stream tier:
- Bring all Stream service instances down.
- In the header of Dev Studio, click .
- Decommission all Stream service instances.
- Truncate the Stream service tables in the Pega Platform data
schema:
- truncate table pr_data_stream_nodes
- truncate table pr_data_stream_sessions
- truncate table pr_data_stream_node_updates
- Delete Kafka-data directory from the Stream instances.
Note: This directory, by default is created under Tomcat folder. - Bring the Stream instances back up one instance at a time.
Previous topic Monitoring the Stream service Next topic Understanding pr_data_stream_* tables