Storing customer and analytical dataConfiguring data sources

Parent topic: Configuring data sources

File storage

Configure local and remote storages to use them as data sources for your decision strategies.

To read, write, and apply data stored in files, create HDFS and File data sets.

Creating an HDFS data set record
You must configure each instance of the HDFS data set rule before it can read data from and save it to an external Apache Hadoop Distributed File System (HDFS).
Creating a File data set record for embedded files
To read data from an uploaded file in CSV or JSON format, you must configure an instance of the File date set rule.
Creating a File data set record for files on repositories
To enable a parallel load from multiple CSV or JSON files located in remote repositories or on the local file system, create a File data set that references a repository. This feature enables remote files to function as data sources for Pega Platform data sets.
Requirements for custom stream processing in File data sets
Standard File data sets support reading or writing compressed .zip and .gzip files. To extend these capabilities to support encryption, decryption, and other compression methods for files in repositories, implement custom stream processing as Java classes on the Pega Platform server classpath.

Parent topic: Configuring data sources