Configure a file listener associated with a storage destination
This article applies to both Pega Cloud and on-premises customers and provides an example that is specific to Pega Cloud. This article is specific to Pega PlatformTM versions 7.2.2 through 7.3.1.
This article describes how Pega Cloud users can configure a File Listener to process files from an S3 bucket; however, Pega Cloud and on-premises users can create a new or update an existing File Listener rule to process files from a target in any storage destination. After a system upgrade to Pega Platform 7.4 or later, it is a best practice to update older file listeners to target files in repositories. To update a file listener in Pega Platform 7.4 or later, see Configure a file listener associated with a storage repository.
For on-premises systems, Pega also supports configuring a file listener on a local file system. In this case, set the Source Location to the file path on the local file system. This is not supported on Pega Cloud systems.
This article assumes that you are familiar with Amazon Web Services (AWS) and appropriate terminology. It assumes that you are familiar with creating, configuring, and troubleshooting the File Listener rule, and all related rules.
Before you begin
Ensure that you have the following available:
- A file service configured in Pega that defines how to process file content (the process rules) and invokes subsequent downstream content activities. For details, see Configuring a file service and file listener to process data in files.
- A target S3 Bucket
- An AWS IAM User ID configured with:
- The ability to read and write to the target S3 bucket
- An Access Key ID and Secret Access Key
- If you are using encryption via an AWS Key Management Service (KMS) key, the Amazon Resource Name (ARN) of your KMS Encryption Key
This task includes the following steps:
Creating an S3 storage destination
To have a file listener process files from an S3 bucket location, you must configure several dynamic system settings (DSS), each using a storage destination alias that is specific to your environment to indicate an S3 bucket in the required Setting Purpose field. For example, replace <storageDestinationAlias> in each DSS listed in the following table with an alias appropriate to your site, such as "MycorpS3.” You can create subsequent Pega File listeners to reference this storage destination alias.
- In Designer Studio, click Create > SysAdmin > Dynamic System Settings.
- Create a new record for each required DSS:
DSS record | Short description | Setting purpose | Value | Owning ruleset |
---|---|---|---|---|
Configure a storage destination type | Storage Destination Type | storage/class/<storageDestinationAlias>:/type | aws-s3 | Pega-Engine |
Configure the S3 bucket name | Bucket Name | storage/class/<storageDestinationAlias>:/bucket | <your S3 bucket name> | Pega-Engine |
Specify an AWS User Access Key ID that is used to authenticate interactions with this storage destination | IAM User Access Key ID | storage/class/<storageDestinationAlias>:/accesskeyid | <your IAM user Access Key ID> | Pega-Engine |
Specify an AWS User Secret Access Key ID that is used to authenticate interactions with this storage destination | IAM User Secret Access Key ID | storage/class/<storageDestinationAlias>:/secretaccesskey | <your IAM user Secret Access Key> | Pega-Engine |
If your S3 buckets are encrypted with AWS KMS, specify the KMS Encryption Key ID | KMS Encryption Key ID | storage/class/<storageDestinationAlias>:/kmsencryptionkeyid | <ARN of your KMS Encryption Key> | Pega-Engine |
Specify a root path in the S3 bucket | Root path | storage/class/<storageDestinationAlias>:/rootpath | <existing folder or prefix within your S3 bucket> | Pega-Engine |
- Save these dynamic system settings.
- From the My Pega Cloud self-service portal, restart your App and Util tier nodes. For detailed instructions, to Restarting an environment from My Pega Cloud.
Creating the file listener
Create a file listener that is associated with the storage destination that you previously configured. For descriptions of configurable file listener property fields, see File Listener form - Completing the Properties tab.
The file service must exist before you can create the file listener. You can also update an existing File Listener so that it processes files in the target S3 location; in this case, update the Source Location field and Source Name Mask appropriately.
The file service must exist before you can create the file listener.
- In the navigation pane of Designer Studio, click Records > Integration-Resources > File Listener.
- Click .
- Complete the fields and click .
- On the Properties tab, enter the following required file listener properties:
- Source Location: file://<storageDestinationAlias>/targetfolder
where storageDestinationAlias matches the alias you set in the DSS records for the storage destination.
In this example, if<storageDestinationAlias>=MycorpS3
androotpath=root/main/files
, then the Source Location configured is equal tofile://MycorpS3:/root/main/files/targetfolder
. - Source Name Mask: *.cvs, *,pdf, and so on.
Enter the type to match the type of files you want to listener to read and process.
- Source Location: file://<storageDestinationAlias>/targetfolder
- Click .
- In the navigation pane of Designer Studio, click System Operations Listener Management.
- In the list on the bottom of the screen, select the appropriate File Listener, and click Restart.
Starting the file listener
- In Admin Studio, click Resources > Listeners.
- Select a node on which to start the listener.
- Select Start listener.
- In the empty field below the Start listener option, press the Down Arrow key and select the name of the listener you want to start.
- Click Apply.
For more information, see Starting a listener.
Testing the file service
When you finish creating the file service and you have started the file listener, test the configuration.
- In the navigation panel, click Records > Integration-Services > Service File.
- Click in the row of the file service rule to open it.
- Select Actions > Trace.
- Put a file in the input directory and wait for the listener to wake up and check the directory.
- Watch the Tracer as it adds new rows of information.
- Examine the results and verify that the file service worked the way you expected it to.