Creating a manifest file to ingest customer data
This content applies only to Pega Cloud environments
Provide metadata about the files that you want to transfer by creating a manifest file and uploading it to the Pega Cloud SFTP server.
The manifest file can include such information as files names and sizes, the number of records in each file, and the process type of the case that manages the ingestion process. In a system configured for ingestion, a file listener listens for the arrival of a manifest file, and then runs a preprocessing check that validates the uploaded files against the manifest file. If the preprocessing check is successful, the system starts the ingestion process.
- Create the manifest file in XML format.
For example: <?xml version='1.0' encoding='UTF-8'?> <manifest> <processType>AccountIngestion</processType> <totalRecordCount>1300</totalRecordCount> <files> <file> <name>account_data_pega_20210531.csv.gz</name> <size>1437966</size> <recordCount>700</recordCount> </file> ... </files> </manifest>Field Description Example processType Type of data being loaded. This field also identifies the data flow to run. AccountIngestion totalRecordCount Total number of records across all data files. 1300 recordCount Record count for one file. 700 name Name of the data file that needs to be agreed with the client and configured in the file data set. The name can have suffix substitution. Note: The file path in the manifest file name needs to be relative to the repository configured in the data set. For example, if your file is in the root folder of your repository, you can directly access the file name, if not, you will need to include the folder structure as a part of the name. account_data_pega_20210531.csv.gz size Size for one file in bytes. This field is optional and used if there is a need to do file size validation. Additional process work is required if size validation must be performed. 1437966 - Upload the manifest file to the Pega Cloud SFTP server.
Previous topic Configuring the data ingestion process Next topic Configuring a repository to ingest customer data