Configuring settings for HDFS connection

Use the HDFS settings in the Hadoop data instance to configure connection details for the HDFS data sets.

  1. In the Connection tab of a Hadoop data instance, select the Use HDFS configuration check box.
  2. In the User name field, enter the user name to authenticate in HDFS.
  3. In the Port field, enter the port of the HDFS NameNode. The default port is 8020.
  4. Optional: To specify a custom HFDS NameNode, select the Advanced configuration check box.

    • In the Namenode field, specify a custom HDFS NameNode that is different from the one defined in the common configuration.

    • In the Response timeout field, enter the number of milliseconds to wait for the server response. Enter zero or leave it empty to wait indefinitely. The default timeout is 3000.

    • In the KMS URI field, specify an instance of Hadoop Key Management Server to access encrypted files from the Hadoop server. For example, for a KMS server running on http://localhost:16000/kms, the KMS URI is kms://http@localhost:16000/kms.
  5. Optional: To enable secure connections, select the Use authentication check box.
    Note: To authenticate with Kerberos, you must configure your environment. For more details, see the Kerberos Network Authentication Protocol documentation.

    • In the Master kerberos principal field, enter the Kerberos principal name of the HDFS NameNode as defined and authenticated in the Kerberos Key Distribution Center, typically following the nn/<hostname>@<REALM> pattern.

    • In the Client kerberos principal field, enter the Kerberos principal name of a user as defined in Kerberos, typically in the following format: <username>/<hostname>@<REALM>.

    • In the Keystore field, enter the name of a keystore that contains a keytab file with the keys for the user who is defined in the Client Kerberos principal setting.

      Note: The keytab file is in a readable location on the Pega Platform server, for example: /etc/hdfs/conf/thisUser.keytab or c:\authentication\hdfs\conf\thisUser.keytab.
  6. Test the connection to the HDFS NameNode, by clicking Test connectivity.