Creating a Kafka configuration instance

A Kafka configuration instance represents an external Apache Kafka server or cluster of servers that is the source of stream data that is processed in real time by Event Strategy rules in your application. You must create a Kafka configuration instance before you can create Kafka data sets for connecting to specific topics that are part of the cluster. You can create an instance of a Kafka cluster in the Data-Admin-Kafka class of Pega Platform.

Before you begin: 
Note: You can connect to Apache Kafka cluster version 0.10.0.1 or later.
  • You can use the SASL authentication for communication between Pega Platform and the Kafka cluster by performing the following actions:
    1. In the Kafka cluster, configure the KafkaClient login credentials in the JAAS configuration file to enable either simple (based on password and login) or Kerberos authentication.
    2. Pass the JAAS file location as a JVM parameter in the Kafka cluster, for example, - Djava.security.auth.login.config = <path_to_JAAS_file>
    Note: When you configure the SASL authentication settings through the JAAS configuration file, you can enter the corresponding configuration credentials in the Authentication section of a Kafka configuration instance. Otherwise, the No JAAS configuration file set message is displayed. For more information about configuring the JAAS file, see the Apache Kafka documentation.

Perform the following steps to create a Kafka configuration instance that represents a Kafka cluster in Pega Platform:

  1. In the header of Dev Studio, click Create > SysAdmin > Kafka.
  2. Enter a short description for the rule.
  3. In the Kafka field, enter the rule ID, for example, MyKafkaInstance.
  4. Click Create and open.
  5. In the Details section, perform the following actions:
    1. Click Add host.
    2. In the Host field, enter the address of the Kafka cluster.
    3. In the Port field, enter the port number.
    4. Optional: Click Add host to configure additional host and port pairs to connect to.
    Note: Pega Platform discovers all the nodes in the cluster during the first connection. This means that you can enter a single host and port combination to connect to a Kafka cluster. As a best practice, enter at least two host and port combinations to ensure a successful connection when a node is unavailable during a Pega Platform restart.
  6. Optional: Configure the SSL authentication settings for the communication between Pega Platform and the Kafka cluster:
    1. In the Security settings section, select the Use SSL configuration check box.
    2. In the Truststore field, press the Down Arrow key and select a truststore file that contains a Kafka certificate or create a truststore record by clicking the Open icon.
    3. Configure the client authentication by selecting the Use client certificate check box and providing the Pega Platform private key and private key password credentials in the Keystore and Key password fields respectively.
  7. Optional: If the SASL authentication method is enabled in the Kafka cluster, configure the SASL authentication settings for the communication between Pega Platform and the Kafka cluster. In the Authentication section, depending on the SASL authentication method that you configured in the Kafka cluster, perform one of the following actions:
    • Enter the login and password credentials.
    • Enter the Kerberos authentication key.
  8. Click Save.