Skip to main content


         This documentation site is for previous versions. Visit our new documentation site for current releases.      
 

Monitoring predictions

Updated on July 5, 2022

Analyze how successful your predictions are in predicting the outcomes that bring value to your business. Gain insights by reviewing performance charts for predictions and the models that drive them.

Note: This procedure applies to predictions that are used in customer engagement and case management. For information about analyzing text predictions, see Monitoring text predictions.
If a prediction uses more than one model, you can select the outcome for which you want to see performance statistics, for example, clicks, conversion, or clicks and conversion combined.
  1. In the navigation pane of Prediction Studio, click Predictions.
  2. From the list of predictions, open a prediction that you want to analyze.
  3. On the Analysis tab, click Prediction.
    1. In the Outcomes field, select the outcome to analyze.
      If a prediction predicts a single outcome (single-stage prediction), the outcome is already selected. If a prediction predicts outcomes that occur in a sequence (multistage prediction), you can select each individual outcome or the two outcomes combined.
      For example: For a multistage prediction that predicts whether a customer who is likely to click a web banner is also likely to accept the corresponding offer and convert, you can select from the following outcomes:
      • Clicks
      • Conversion
      • Clicks + Conversion
    2. In the Time frame field, select the period that you want to analyze.

      The charts show the performance measures that are relevant to the selected outcome. For example, a performance analysis with regard to churn covers the churn rate, lift, AUC, and total cases, as in the following figure:

      Prediction analysis
      The Analysis tab in a prediction showing the performance charts for churn.
  4. Click the Models tab.
  5. If more that one model drives the prediction, in the Models field, select the model the performance of which you want to analyze.

    The performance chart shows how effective the model is over time. If a shadow model runs alongside an active model, the chart displays the performance curves for each model in the active-shadow pair. The comparison can help you determine which model is more effective and decide whether you want to promote the shadow model to the active model position. The analysis is available for both outcome-based and supporting models.

    The following figure shows the performance chart for the active model, Predict Churn DRF. This model is paired with a shadow model, Churn GBM. The chart shows two distinct lines that illustrate the performance of each model over time. Hover over the lines with your cursor to see the exact scores for each model at different times. In this example, the two lines converge around January 27, suggesting that the models performed equally well. The detailed information on mouse hover shows that the active model had a slightly higher score.

    A model performance chart in a prediction
    The Analysis tab in a prediction showing the performance of an active model and its shadow as two distinct lines.

Have a question? Get answers now.

Visit the Support Center to ask questions, engage in discussions, share ideas, and help others.

Did you find this content helpful?

Want to help us improve this content?

We'd prefer it if you saw us at our best.

Pega.com is not optimized for Internet Explorer. For the optimal experience, please use:

Close Deprecation Notice
Contact us