You can create an instance of a Kinesis data set in Pega Platform to connect to an instance of Amazon Kinesis Data Streams. The Amazon Kinesis Data Streams
service ingests a large amount of data in real time, durably stores it, and makes it available
for lightweight processing.
For Pega Cloud applications, you can use a Pega-provided Kinesis data stream or connect to your own Kinesis data
Before you begin: Make sure that the Identity and Access Management (IAM) policies
in Amazon Web Services (AWS) are set to allow access to Kinesis data streams. For more
information, see the AWS documentation about IAM policies. To use your own Kinesis account
with data streams, change the value of the useExternalKinesisAccount
dynamic system setting to true.
In the header of Dev Studio, click CreateData ModelData Set.
On the Create Data Set tab, in the Data Set Record
Configuration section, define the following settings to identify your data
In the Label field, enter the data set label.
Result: The identifier is automatically created based on the data set
Optional: To change the automatically created identifier, click Edit,
enter an identifier name, and then click OK.
In the Type list, select
In the Context section, specify the application context,
applicable class, ruleset, and ruleset version of the data set.
Click Create and open.
In the Connection section, configure the connection to your AWS
Note: This step is not available if your Pega Platform application is hosted in a Pega Cloud
environment (the onPegaCloud dynamic system setting is set to
true) and you use Pega-provided
Kinesis data stream.
For more information about the available regions, see the AWS
In the Stream section, select a stream that is available in your
Kinesis configuration instance.
Important: If you use a Kinesis data stream with
Pega Platform on premises or a Kinesis data stream with Pega Platform in the cloud that are in different regions, you might
experience performance issues during data set operations. For optimal performance, use a
Kinesis data stream with Pega Platform in the cloud that are in the
Optional: In the Partition key(s) section, define the data set
By configuring partitioning, you ensure that related records are sent to the same
partition. If you do not define partition keys, the Kinesis data set randomly assigns
records to partitions, which can hinder its performance.
Click Add key.
In the Key field, press the Down Arrow key to select the
property that you want the Kinesis data set to use as a partitioning key.
Note: By default, the available properties to be used as keys
correspond to the properties of the Applies To class of the Kinesis data set.