Database tables for monitoring models
Database tables contain monitoring information for adaptive and predictive models. This data is used to populate the charts in the Monitor tab of every model rule in Dev Studio and additional reports in the section of Prediction Studio.
The tables are transparently designed to contain monitoring data that you can easily use to create your custom reports in Pega Platform or any other reporting tool. Before you customize model reports to serve your business needs, learn what properties you can find in these tables.
For more information about how the Adaptive Decision Manager (ADM) service captures and populates that monitoring data, see Configuring the Adaptive Decision Manager service.
Properties in the PR_DATA_DM_ADMMART_MDL_FACT table
Properties with the py prefix are defined on the Data-Decision-ADM-ModelSnapshot class. The out-of-the-box data set which applies to this class is pyModelSnapshots.
| Property | Description |
| pxApplication | The name of the application. |
| pyAppliesToClass | The Applies To class of the Adaptive Model rule with which the model is associated. |
| pyModelID | A unique identifier for each model. |
| pyConfigurationName | The name of the Adaptive Model rule with which the model is associated. |
| pySnapshotTime | The time when the snapshot was taken. |
| pyIssue | The default model identifier that defines the model context in your application. For more information, see Model context. |
| pyGroup | The default model identifier that defines the model context in your application. |
| pyName | The default model identifier. When the context of the models is different from the default hierarchy, this property contains the context values in the JSON format. |
| pyChannel | The default model identifier that defines the model context in your application. |
| pyDirection | The default model identifier that defines the model context in your application. |
| pyTreatment | The default model identifier that defines the model context in your application. |
| pyPerformance | The performance of each ADM model. |
| pySuccessRate | The measure of the fraction or percentage of success among the total requests. |
| pyResponseCount | The number of all responses for each model. |
| pxObjClass | The name of the class object. |
| pzInsKey | The internal key to represent the object. |
| pxInsName | The visible key of an instance. |
| pxSaveDateTime | The time and date of the last save. |
| pxCommitDateTime | The date and time when the commit occurred. |
| pyActivePredictors | The number of active predictors for each model. |
| pyTotalPredictors | The number of all predictors for each model. |
| pyNegatives | The number of negative responses for each model. |
| pyPositives | The number of positive responses for each model. |
| pyModelData | The compressed and encoded scoring model. |
| pyModelVersion | A unique identifier for the model version. When a model is updated, it receives a new version identifier. Every decision also contains this model version, so if Full Auditability is enabled, you can use the model version to find the exact model used. |
| pyFactoryUpdatetime | The time when the model factory was last updated. |
Properties in the PR_DATA_DM_ADMMART_PRED table
Properties with the py prefix are defined on the Data-Decision-ADM-PredictorBinningSnapshot class. The out-of-the-box data set which applies to this class is pyADMPredictorSnapshots.
| Property | Description |
| pxCommitDateTime | The date and time when the commit occurred. |
| pxSaveDateTime | The time and date of the last save. |
| pyModelID | A unique identifier for each model. |
| pxObjClass | The name of the class object. |
| pzInsKey | The internal key to represent the object. |
| pxInsName | The visible key of an instance. |
| pyPredictorName | The name of the predictor. |
| pyContents | The overall range of the predictor. |
| pyPerformance | The performance of the predictor. |
| pyPositives | The number of positive responses for the predictor per model. |
| pyNegatives | The number of negative responses for the predictor per model |
| pyType | The type of predictor—numeric or symbolic. |
| pyTotalBins | The number of bins for the predictor per model. |
| pyResponseCount | The number of responses for the predictor per model. |
| pyBinNegatives | The number of negative responses for the predictor per model in the bin. |
| pyBinPositives | The number of positive responses for the predictor per model in the bin. |
| pyBinType | The type of bin:
|
| pyBinNegativesPercentage | The percentage of negative responses in the bin out of all responses for the predictor. |
| pyBinPositivesPercentage | The percentage of positive responses in the bin out of all responses for the predictor. |
| pyBinSymbol | The actual range of the bin. |
| pyBinLowerBound | The lower bound of the bin. This property applies to numeric predictors. |
| pyBinUpperBound | The upper bound of the bin. This property applies to numeric predictors. |
| pyRelativeBinPositives | The difference between the number of positive responses for the predictor in the bin that is measured per model and the number of positive responses for the predictor in the bin from the last snapshot. |
| pyRelativeBinNegatives | The difference between the number of negative responses for the predictor in the bin that is measured per model and the number of negative responses for the predictor in the bin from the last snapshot. |
| pyBinResponseCount | The total number of responses for the predictor in the bin that is measured per model. |
| pyRelativeBinResponseCount | The difference between the number of all responses for the predictor in the bin that is measured per model and the number of all responses for the predictor in the bin from the last snapshot. |
| pyBinResponseCountPercentage | The percentage of responses falling under a particular bin for a predictor that is measured per model. |
| pySnapShotTime | The time when the snapshot was taken. |
| pyBinIndex | An index assigned to a predictor bin entry. For example, when a model contains the AGE predictor that has 10 bins, then we have 10 entries in the table and the pyBinIndex for each entry is: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10. |
| pyLift | Behavior in the predictor bin divided by the overall behavior. |
| pyZRatio | The number of standard deviations in the behavior of the predictor bin differs from the overall behavior. |
| pyEntryType | An indication whether it is an active or inactive predictor. Active predictors are the ones that are used by the model. |
| pyFeatureImportance | The importance of the predictor to the model. |
Properties in the PR_DATA_DM_BINARY_DISTRIBUTION table (binary outcome predictive models)
For binary outcome models, the count of the positives and negatives is stored in a granular set of bins which are used to calculate the AUC and the ROC curves. The score distribution and the observed responses overlay occur during training.
| Property | Description |
| pxApplication | The name of the application. |
| pyRulesetName | The name of the ruleset to which the Predictive Model rule belongs. |
| pyRulesetVersion | The version of the ruleset to which the Predictive Model rule belongs. |
| pyModelID | A unique identifier for each model. |
| pyFactoryKey | A key that corresponds to the version information in the ADM factory data set (internal audit usage). |
| pySnapshotTime | The time when the snapshot was taken. |
| pySnapshotID | An identifier for the snapshot. |
| pyBinID | A unique identifier for each bin that is created. |
| pyBinLabel | A label for each bin with ranges. |
| pyBinUpper | The upper boundary of the bin. |
| pyBinLower | The lower boundary of the bin. |
| pyPredictorName | The name of the predictor for which the statistic is stored. |
| pyCount | The total number of responses for the bin. |
| pyPositiveCount | The number of positive responses. |
| pyNegativeCount | The number of negative responses. |
| pyDataUsage | The stage of the model building process at which the data was derived—training, predicted, or observed. |
| pyMetricType | The metric type for the bin—binary distribution or side-by-side distribution. |
| pyBehavior | The behavior of the bin, usually the percentage of positive responses. |
| pyPercentage | The percentage of responses in the bin. |
| pxObjClass | The name of the class object. |
| pzInsKey | The internal key to represent the object. |
| pxInsName | The visible key of an instance. |
| pxSaveDateTime | The time and date of the last save. |
| pxCreateDateTime | The time and date when the record was created. |
| pxUpdateDateTime | The time and date when the record was updated. |
| pxCommitDateTime | The date and time when the commit occurred. |
Properties in the PR_DATA_DM_CONTINGENCYTABLE table (categorical outcome predictive models)
For categorical outcome models, the confusion matrix of responses is the main statistic. Each cell in the confusion matrix is stored as a record in the database table.
The confusion matrix is used to calculate the performance values, such as the F-statistic for the model or the accuracy for the classes.
| Property | Description |
| pxApplication | The name of the application. |
| pyRulesetName | The name of the ruleset to which the Predictive Model rule belongs. |
| pyRulesetVersion | The version of the ruleset to which the Predictive Model rule belongs. |
| pyModelID | A unique identifier for each model. |
| pyFactoryKey | A key that corresponds to the version information in the ADM factory data set (internal audit usage). |
| pySnapshotTime | The time when the snapshot was taken. |
| pySnapshotID | An identifier for the snapshot. |
| pyIdentifier | An identifier for the category combination. |
| pyBinLabel | A label for each bin with ranges. |
| pyPredictedCategory | The category at the make-decision time. |
| pyActualCategory | A category that was retrieved from the response. |
| pyDataUsage | The stage of the model building process at which the data was derived—training, predicted, or observed. |
| pyDistributionType | The type of distribution that is stored—residual values or actual values. |
| pyCount | The number of values that are observed in each bin. |
| pxObjClass | The name of the class object. |
| pzInsKey | The internal key to represent the object. |
| pxInsName | The visible key of an instance. |
| pxSaveDateTime | The time and date of the last save. |
| pxCreateDateTime | The time and date when the record was created. |
| pxUpdateDateTime | The time and date when the record was updated. |
| pxCommitDateTime | The date and time when the commit occurred. |
Properties in the PR_DATA_DM_HISTOGRAM table (continuous outcome models)
For continuous outcome models, the difference between the predicted outcome and the actual outcome is used to measure the performance. The distribution of these residual values is stored in bins of equal interval size. The Information that is gathered in the bins is used to calculate the root-mean-square error (RMSE) and mean absolute error (MAE) performance statistics for the model.
| Property | Description |
| pxApplication | The name of the application. |
| pyRulesetName | The name of the ruleset to which the Predictive Model rule belongs. |
| pyRulesetVersion | The version of the ruleset to which the Predictive Model rule belongs. |
| pyModelID | A unique identifier for each model. |
| pyFactoryKey | A key that corresponds to the version information in the ADM factory data set (internal audit usage). |
| pySnapshotTime | The time when the snapshot was taken. |
| pySnapshotID | An identifier for the snapshot. |
| pyBinID | A unique identifier for each bin that is created. |
| pyBinLabel | A label for each bin with ranges. |
| pyBinUpper | The upper boundary of the bin. |
| pyBinLower | The lower boundary of the bin. |
| pyDataUsage | The stage of the model building process at which the data was derived—training, predicted, or observed. |
| pyDistributionType | The type of distribution that is stored—residual values or actual values. |
| pyCount | The total number of responses for the bin. |
| pyMinimum | The minimum value in the bin. |
| pyMaximum | The maximum value in the bin. |
| pyAverage | The average value in the bin. The minimum, maximum, and average values help describe the distribution of data in the bin. |
| pxObjClass | The name of the class object. |
| pzInsKey | The internal key to represent the object. |
| pxInsName | The visible key of an instance. |
| pxSaveDateTime | The time and date of the last save. |
| pxCreateDateTime | The time and date when the record was created. |
| pxUpdateDateTime | The time and date when the record was updated. |
| pxCommitDateTime | The date and time when the commit occurred. |
Properties in the PR_DATA_DM_SNAPSHOTS table (snapshot summary)
The monitoring information that is stored in the monitoring data mart contains data that is related to the same point in time; that collection of monitoring data is called a snapshot. This table contains one record per snapshot per model.
This information can be linked to data in other tables that contain more detailed, binned information that is used to calculate the performance statistics.
One-time model data may consist of the following details:
- The number of responses in the snapshot
- The performance of the model during the snapshot
- The user that triggered the snapshot
| Property | Description |
| pxApplication | The name of the application. |
| pyRulesetName | The name of the ruleset to which the Predictive Model rule belongs. |
| pyRulesetVersion | The version of the ruleset to which the Predictive Model rule belongs. |
| pyModelID | A unique identifier for each model. |
| pyFactoryKey | A key that corresponds to the version information in the ADM factory data set (internal audit usage). |
| pySnapshotTime | The time when the snapshot is taken. |
| pySnapshotID | An identifier for the snapshot. |
| pySnapshotDay | The day when the snapshot is taken. This property is only applicable for the aggregated entries. For individual snapshots, the value for pySnapshotDay is empty. Individual snapshots are captured when the agent runs the activity daily or when you click Refresh Data. |
| pyValue | The value of the (meta) property. |
| pyLabel | The label of the category. |
| pySnapshotType | The frequency of taking the snapshot—daily, weekly, monthly, and
so on. This property is only applicable for the aggregated entries. For individual snapshots, the value for pySnapshotType is empty. |
| pyDataUsage |
|
| pyIdentifier | The identifier of the category for the snapshot. |
| pyName | The name of the (meta) property. |
| pyDistributionType | The type of distribution that is stored—residual values or actual values. |
| pyCount | The total number of responses for the bin. |
| pxObjClass | The name of the class object. |
| pzInsKey | The internal key to represent the object. |
| pxInsName | The visible key of an instance. |
| pxSaveDateTime | The time and date of the last save. |
| pxCreateDateTime | The time and date when the record was created. |
| pxUpdateDateTime | The time and date when the record was updated. |
| pxCommitDateTime | The date and time when the commit occurred. |
| pyPurpose | The model name for which the metric is being calculated, for example, PREDICTCHURN. |
| pyClassName | The model class name for which the metric is being calculated, for example, DMORG-DMSAMPLE-DATA-CUSTOMER. |
| pyFieldName | The predictor/output name for which the metric is being calculated, for example, propensity/Age. |
| pyFieldUsage | Signifies whether the field is an input/output. |
Properties in the PR_DATA_DM_DECISION_DISTRIBUTION
| Property | Description |
| pyPurpose | The model name for which the metric is being calculated. |
| pyClassName | The model class name for which the metric is being calculated. |
| pyFIeldName | The field for which the metric is being calculated. |
| pySnapshotTime | The time when the aggregated metric was generated. |
| pySnapshotType | The aggregation type: daily, weekly or monthly. |
| pySnapshotDay | The day on which the snapshot was taken. |
| pyModelType | Model type. |
| pyIdentifier | The hash of purpose, classname, label, and fieldname. |
| pyDistributionType | Field is numeric/symbolic. |
| pyLabel | Bin label or the class label of the bin. |
| pyCount | Total count of values for daily/weekly/monthly. |
| pyRelativeCount | Count of values for daily/weekly/monthly. |
| pyFieldUsage | Signifies whether the field is an input/output. |
Properties in the PR_DATA_DM_SUMMARIES table
| Property | Description |
| pyModelID | The ID of the model |
| pySnapshotDay | The day on which the snapshot was taken. |
| pySnapshotTime | The time when the snapshot was taken. |
| pySnapshotType | The aggregation type: daily, weekly or monthly. |
| pyData | The serialized object data. |
Properties in the PR_DATA_DM_NOTIFICATION table
This table contains Prediction Studio notifications that inform about sudden drops in predictive performance, models with low performance, or other issues with models.
| Property | Description |
| pzInsKey | The internal key to represent the object. |
| pxObjClass | The name of the class object. |
| pxInsName | The visible key of an instance. |
| pxSaveDateTime | The time and date of the last save. |
| pxCommitDateTime | The date and time when the commit occurred. |
| pyLabel | The name of the model for which the notification is generated. |
| pyRulesetName | The name of the ruleset to which the model rule belongs. |
| pyRulesetvVrsion | The version of the ruleset to which the model rule belongs. |
| pyNotificationType | The objective on which the notification is generated. For example, notifications can be based on Performance/Response count. |
| pxCreateDateTime | The time and date when the record was created. |
| pxUpdateDateTime | The time and date when the record was updated. |
| pyTriggerTime | The time when the notification is generated. |
| pxCreateOperator | Operator ID. |
| pyConfigID | An identifier that indicates the model insName or model ID for adaptive models. |
| pyContext | The values of model identifiers that define the model context for an Adaptive Model rule. |
| pyDescription | The notification message that is visible in the user interface. |
| pyModelID | A unique identifier of the model rule. |
| pyModelReferenceKey | A reference to the models of a single Adaptive Model rule. |
| pyModelType | The type of model for which the notification is generated (predictive or adaptive). |
| pyNotificationTypeID | A unique ID for each notification type. |
| pyPriority | The priority for each notification type. |
| pyStatus | The status that indicates whether the user has read the notification. |
Previous topic Adaptive model details Next topic Best practices for choosing predictors