Skip to main content

         This documentation site is for previous versions. Visit our new documentation site for current releases.      

This content has been archived and is no longer being updated.

Links may not function; however, this content may be relevant to outdated versions of the product.

Configuring Hadoop settings for an HDFS connection

Updated on March 11, 2021

Use the HDFS settings in the Hadoop data instance to configure connection details for the HDFS data sets.

  1. In the Connection tab of a Hadoop data instance, select the Use HDFS configuration check box.
  2. In the User name field, enter the user name to authenticate in HDFS.
  3. In the Port field, enter the port of the HDFS NameNode. The default port is 8020.
  4. Optional: To specify a custom HFDS NameNode, select the Advanced configuration check box.
    • In the Namenode field, specify a custom HDFS NameNode that is different from the one defined in the common configuration.

    • In the Response timeout field, enter the number of milliseconds to wait for the server response. Enter zero or leave it empty to wait indefinitely. The default timeout is 3000.

    • In the KMS URI field, specify an instance of Hadoop Key Management Server to access encrypted files from the Hadoop server. For example, for a KMS server running on http://localhost:16000/kms, the KMS URI is kms://http@localhost:16000/kms.
  5. Optional: To enable secure connections, select the Use authentication check box.
    Note: To authenticate with Kerberos, you must configure your environment. For more details, see the Kerberos Network Authentication Protocol documentation.
    • In the Master kerberos principal field, enter the Kerberos principal name of the HDFS NameNode as defined and authenticated in the Kerberos Key Distribution Center, typically following the nn/<hostname>@<REALM> pattern.

    • In the Client kerberos principal field, enter the Kerberos principal name of a user as defined in Kerberos, typically in the following format: <username>/<hostname>@<REALM>.

    • In the Keystore field, enter the name of a keystore that contains a keytab file with the keys for the user who is defined in the Client Kerberos principal setting.

      Note: The keytab file is in a readable location on the Pega Platform server, for example: /etc/hdfs/conf/thisUser.keytab or c:\authentication\hdfs\conf\thisUser.keytab.
  6. Test the connection to the HDFS NameNode, by clicking Test connectivity.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best. is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us