Configure local and remote storages to use them as data sources for your decision strategies.

• Creating an HDFS data set record

  You must configure each instance of the HDFS data set rule before it can read data from and save it to an external Apache Hadoop Distributed File System (HDFS).

• Configuring Hadoop settings for an HDFS connection

  Use the HDFS settings in the Hadoop data instance to configure connection details for the HDFS data sets.

• Creating a File data set record for embedded files

  To read data from an uploaded file in CSV or JSON format, you must configure an instance of the File date set rule.

• Creating a File data set record for files on repositories

  To process data in parallel by using multiple threads, the File Data Set operates on a collection of files instead of a single file. The file path pattern includes tokens that match existing files to the pattern or generate new files.

• Requirements for custom stream processing in File data sets

  Standard File data sets support reading or writing compressed .zip and .gzip files. To extend these capabilities to support encryption, decryption, and other compression methods for files in repositories, implement custom stream processing as Java classes on the Pega Platform server classpath.