When running archival jobs and purge jobs, use the following settings:

• Configure dataarchival/LimitPerPolicy to change the rate at which you archive eligible cases. For more information about archive settings, see Creating and configuring case archive settings.
• Configure dataarchival/purgeQueryLimit to change the rate at which you delete records after they have been archived to Pega Cloud File Storage. For more information about archive and purge settings, see Configuring purge cycle settings.

Plan to run your archive and purge jobs at an initially fast rate to clear your backlog.

1. Identify the number of cases in your initial backlog and the rate at which cases become eligible for archiving, then create an archival policy for those cases.
   For example, your system contains a backlog of 10,000 cases and adds 500 cases a day.

   For more information about creating an archival policy, see Creating an archival policy.

2. Schedule an archive and purge process that finishes during a time period of low-system load.
   For example, your system might experience low system load every day for five hours or every weekend for 12 hours.

   For more information about scheduling an archival policy, see Scheduling a case archival policy.

3. Determine the upper range for archiving and purge processing by adjusting the dataarchival/LimitPerPolicy and dataarchival/purgeQueryLimitsettings. You can run an experiment to determine the necessary values by performing the following steps:
   1. Set a low rate in the dataarchival/LimitPerPolicy and dataarchival/purgeQueryLimit dynamic system settings, respectively.

      A reasonable low value for these settings is 100. For more information about archive and purge settings, see Creating and configuring case archive settings and Configuring purge cycle settings.

   2. Run a process with the following jobs, using the low value for the number of cases:
      • pyPegaArchiver
      • pyPegaIndexer
      • pyPegaPurger

        For more information, see Case archiving and purging overview.

   3. Monitor the progress of your case archiving process.  For more information, see Monitoring the progress of your case archival process.
   4. Evaluate the performance impact and time that the low rate of archive and purge process has on your system.
   5. Increase the settings for archiving and purging values and run the archiving process again with increased values.

      A faster rate of archiving impacts system resources more, and takes longer to complete.

      Note: The pyPegaPurger job can take a significant amount of time, and even a small value, such as 5000, for the dataarchival/purgeQueryLimit setting can cause timeout errors. If you encounter an error, use a smaller value for dataarchival/purgeQueryLimit and then schedule multiple runs of only the pyPegaPurger job until pr_metadata is empty. This indicates that all cases processed by pyPegaCopier and pyPegaIndexer have been purged by pyPegaPurger.
   6. Continue to increase the rate to determine that maximum value with which you can complete the process within a low system load duration and with an acceptable system impact.
4. Clear your backlog by using the maximum rate found above. The maximum rate of archiving and purging determines the time frame for clearing your backlog.
5. Adjust the archiving and purging process after clearing your backlog.
   • If your current rate of archiving and purging meets or exceeds the rate at which cases become eligible for archiving, keep that archiving rate.
   • If your current rate of archiving is slower than the rate at which cases become eligible for archiving, plan to run your archival and purge jobs more frequently to archive faster.
   For example: If your rate does not exceed the rate at which cases become eligible for archiving, then run your archive and purge jobs twice a day instead of once.