Monitoring a predictive model

Verify the accuracy of your predictive models by analyzing the data gathered in the Monitor tab.

Before you begin: To monitor a predictive model, ensure that a system architect creates a response strategy that references the model and defines the values for the .pyOutcome and .pyPrediction properties, where:
  • The .pyPrediction value is the same as the model objective that is visible in the Model tab for that predictive model (applies to all model types).
  • For binary models, the .pyOutcome value is the same as one of the outcome labels that is visible in the Model tab for that predictive model. For continuous and categorical models, this parameter value does not need to correspond to the model settings.
For more information, see Headless decisioning.
  1. In the navigation pane of Prediction Studio, click Models.
  2. Click the predictive model that you want to monitor.
  3. To load the latest monitoring data, on the Actions menu of the model page, click Refresh.
  4. On the Monitor tab, in the Time range and Time frame sections, specify the time for which you want to analyze the data.
  5. Review the predictive model performance data:
    1. In the Performance area, verify how accurately your model predicted the outcomes in the specified time, compared to the expected value.
    2. In the Total responses area, analyze the number of responses that were gathered in the specified time.
    3. In the Score distribution area, analyze how a predictive model segmented cases in the population.
    4. In the Success rate area, analyze the number of successful outcomes as a percentage of all propositions.

      Successful outcome is the outcome that the predictive model predicts. You can find this setting in the Model tab for that model.

    For more information on how to interpret different performance charts, see Metrics for measuring predictive performance.