Skip to main content


         This documentation site is for previous versions. Visit our new documentation site for current releases.      
 

Creating a Stream data set

Updated on July 5, 2022

Process a continuous data stream of events (records) by creating a Stream data set.

You can test how data flow processing is distributed across Data Flow service nodes in a multinode decision management environment by specifying the partition keys for Stream data set and by using the load balancer provided by Pega. For example, you can test whether the intended number and type of partitions negatively affect the processing of a Data Flow rule that references an event strategy.
  1. In the header of Dev Studio, click CreateData ModelData Set.
  2. On the Create Data Set tab, in the Data Set Record Configuration section, define the following settings to identify your data set:
    1. In the Label field, enter the data set label.
      Result: The identifier is automatically created based on the data set label.
    2. Optional: To change the automatically created identifier, click Edit, enter an identifier name, and then click OK.
    3. In the Type list, select Stream.
  3. In the Context section, specify the application context, applicable class, ruleset, and ruleset version of the data set.
  4. Click Create and open.
  5. Optional: To create partition keys for testing purposes, in the Stream tab, in the Partition key(s) section, perform the following actions:
    Note: Create partition keys for Stream data sets only in application environments where the production level is set to 1 - Sandbox, 2 - Development, or 3 - Quality assurance. For more information, see Specifying the production level.
    1. Click Add key.
    2. In the Key field, press the Down arrow key, and then select a property to use as a partition key.
      The available properties are based on the applicable class of the data set which you defined in step 3.
    3. To add more partition keys, repeat steps 5.a through 5.b.
    For more information on when and how to use partition keys in a Stream data set, see Partition keys for Stream data sets.
  6. Optional: To disable basic authentication for your Stream data set, perform the following actions: in the Settings tab, perform the following actions:
    1. Click the Settings tab.
    2. Clear the Require basic authentication check box.
      The REST and WebSocket endpoints are secured by using the Pega Platform common authentication scheme. Each post to the stream requires authenticating with your user name and password. By default, the Enable basic authentication check box is selected.
  7. Confirm your settings by clicking Save.
  8. Optional: To populate the Stream data set with external data, perform the following actions:
    1. In the navigation panel of Dev Studio, click RecordsIntegration-ConnectorsConnect REST.
    2. Select an existing Pega REST service or create a new Connect REST rule.
    3. Configure the settings in the Methods tab.
  • Partition keys for Stream data sets

    You can define a set of partition keys in a Stream data set to test how data flow processing is distributed across Data Flow service nodes in a multinode decision management environment by using the default load balancer. For example, you can test whether the intended number and type of partitions negatively affect the processing of a Data Flow rule that references an event strategy.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us