Extract rules - Completing the File Specification tab

Complete the File Specification tab to control where extract output is written. The fields that display depend on the Output Format that you select in the Definition tab.

Note: Use the -P command-line option to override the output directories specified on this tab, instead directing all files to a specified location.
Output Format Fields
XML These substitutions are also automatically applied to the BIX schema file for XML (XSD file), which is automatically generated for each run of an Extract rule using XML output for its data.
CSV CSV Output Directory — Identify a directory to receive the CSV files.
Database Select the name of the database (an instance of Data-Admin-DB-Name ) to receive the extracted data.
Note: Because the Pega RULES database is the source of the data, do not select PegaRULES as the target in this field.

Before extracting data to the target database, you must first create a schema to match the extracted data. You can use BIX to generate an SQL file that creates the appropriate schema. For more information, see Running a BIX extract from the command line.

After you complete this tab, save the rule form, then click Test Connectivity to verify that you have a valid path to the database or directories specified in this tab. Pega Platform reports the results of the test in a pop-up window.

Create a manifest

Select the Include manifest check box to create a manifest, or summary statement, of the records extracted. You can create the manifest in the file format of your choice, independent of the output format that you selected for the actual extract.

Depending on your choice, additional fields display:

Output Format Fields
XML These substitutions are also automatically applied to the BIX schema file for XML (XSD file) that is automatically generated for each run of an Extract rule that outputs data in XML output.
Comma Separated File (CSV) Manifest CSV file output directory — Identify a directory to receive the CSV files.
Database Schema Manifest output database name — Specify the name of the database where the manifest is to be written.

If you want to output the manifest to a database, you must prepare the database by adding two tables, pr_extract_summary and pr_extract_details. When you run the extract for the first time (or after editing the rule):

  1. In the form where you enter run parameters, select the Generate manifest schema DDL check box to create the schema DDL that is used to create the tables.
  2. In the next field, provide the location on your computer to create the schema.
  3. Provide the created schema to your database administrator, who can then update the database so it can receive the manifest.
  4. The manifest DDL scripts are generated with the default schema name prefixed to the table name.

Click Test Connectivity to verify that you have a valid path to the database or directory that you specified. Pega Platform reports the results of the test in a pop-up window.

In the Include check total property field, you can select an extract property to use as a "checksum" for the extract.

Contents of the manifest

The manifest that you generate includes a summary of the extract run, including:

  • The unique identifier for the run, consisting of:
    • pxExtractIdentifier, which is the pzInsKey of the extract or a command-line parameter passed in using the -k option, with the sequence number appended.
    • pxExtractDateTime, which holds the date and time of the start of the run.
  • The application that the extract is associated with (if it is associated with a single application).
  • The number of class instances that the extract retrieved from the class table during the run.
  • The DateTime for the start and end of the run and the elapsed time of the run in minutes, seconds, and milliseconds.
  • The number of retrieved class instances that failed (for which the run inserted no records into the destination file or database table).
  • The total of all values of the extracted numeric property, if the extract specifies a numeric property in the Include check total property field.

If you save the manifest to CSV or to a database, the manifest also includes for each destination CSV file or table:

  • The unique identifier for the run.
  • The name of the destination table.
  • The number of INSERTs committed to the table.

The extract writes the manifest records as a single commit at the end of the extract. If there is an error during this commit, the extract writes the manifest information to the log file.

Notes about output

  • When scalar properties in an embedded value list, page list, value group, or page group have been selected for extraction to a CSV file or to a database, BIX generates a separate CSV extract file or a separate database table for the embedded properties. BIX promotes scalar properties of embedded pages (not page lists or page groups) to the parent node.
  • When you extract data for insertion into a database, text values longer than the defined length of the destination database column are automatically truncated. The “+” character is appended to the end of the extracted text value to indicate that it has been truncated.
  • The entry in the PegaBIX log file for each run contains the BIX parameters used for that run, except for any password provided. For more information, see Command line BIX extracts.
  • When extracting Boolean properties to a database, top-level properties are extracted as 0 and 1, and value list and value group properties are extracted as true and false.