Skip to main content

         This documentation site is for previous versions. Visit our new documentation site for current releases.      

This content has been archived and is no longer being updated.

Links may not function; however, this content may be relevant to outdated versions of the product.

Creating a Kinesis data set

Updated on March 11, 2021

You can create an instance of a Kinesis data set in Pega Platform to connect to an instance of Amazon Kinesis Data Streams. Amazon Kinesis Data Streams ingests a large amount of data in real time, durably stores it, and makes it available for lightweight processing. For Pega Cloud applications, you can use a Pega-provided Kinesis data stream or connect to your own Kinesis data stream.

Before you begin: Make sure that the Identity and Access Management (IAM) policies in Amazon Web Services (AWS) are set to allow access to Kinesis data streams. For more information, see the Amazon Web Services (AWS) documentation about IAM policies. To use your own Kinesis account with data streams, change the value of the useExternalKinesisAccount dynamic system setting to true
  1. Create an instance of the Kinesis data set rule.
  2. In the Connection section, select a Kinesis configuration instance and a region. For more information about the available regions, see the Amazon Web Services (AWS) documentation.
    Note: This step is not available if you are running Pega Platform in a cloud environment (the onPegaCloud dynamic system setting is set to true) and you are using a Pega-provided Kinesis data stream.
  3. In the Stream section, select a stream that is available in your Kinesis configuration instance.
    Important: If you use a Kinesis data stream with Pega Platform on premises or a Kinesis data stream with Pega Platform in the cloud that are in different regions, you might experience performance issues during data set operations. For optimal performance, use a Kinesis data stream with Pega Platform in the cloud that are in the same region.
  4. Optional: In the Partition key(s) section, define the data set partitioning.
    By configuring partitioning, you ensure that related records are sent to the same partition. If you do not define partition keys, the Kinesis data set randomly assigns records to partitions, which can hinder its performance.
    1. Click Add key.
    2. In the Key field, press the Down Arrow key to select the property that you want the Kinesis data set to use as a partitioning key.

      By default, the available properties to be used as keys correspond to the properties of the Applies To class of the Kinesis data set.

  5. Click Save.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best. is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us