Configuring the monitoring of model input and output
Monitor input, that is, predictors, and output of your models and predictions to observe whether they behave as you expect.
Models use predictors, such as age, income, location, or current product subscriptions, to predict customer responses or case outcomes. The output is the data that model generate, including propensities, numeric output for continuous models, and symbolic output (labels) for binary and categorical models.
Monitoring predictors and output can provide useful insights about the performance of your predictions and models. It helps in the early stages of model development, before you capture any responses. It also helps for certain model types that do not receive responses regularly or at all, such as predictive models.
Review the information about the estimated payload for model monitoring, which determines the amount of model executions that you can monitor without a negative effect on system performance.
Configure the analytics repository:
- Ensure that the repository has 15 GB of disk space.
- Do not use the
defaultstoretemporary repository, but a more stable one.
Ensure that the disk space allocated to the Stream service is 250 GB.
- In the navigation pane of Prediction Studio, click .
- In the Monitor model input and output section, ensure that the Monitor model input and output data checkbox is selected.
- In the Monitor percentage field, define the amount of
data that you want to monitor.
Note: The default monitor percentage is 5%, which means that the system monitors 5% of all model executions. Higher values might affect performance. Choose a value that meets your monitoring requirements and ensures good system performance. For more information, see Estimating the model monitoring payload.
- In the Monitoring coverage list, select which data you
want to monitor:
You can find the metrics on the Analysis tab in predictions. For more information, see Monitoring predictions.
- Monitor output (propensities, numeric values, labels) and input (predictors) by selecting Monitor output and input.
- Monitor output only (propensities, numeric values, labels) by selecting Monitor output only.
- Optional: To record and save the monitoring data in the analytics repository, select Record monitoring data.
- Click Save.
Previous topic Optimizing properties to speed up segment creation Next topic Configuring the prospect class