Model update statuses and notifications
As a candidate model passes through the various stages of the model update process, such as validation, review, or shadow mode, its status changes in Prediction Studio. Data scientists and other stakeholders then receive email notifications about the key events in the life cycle of a model update.
Operations related to updating models are part of the Machine Learning Operations (MLOps) feature of Pega Platform.
Learn about these statuses and notifications to better understand the model update process.
Candidate model statuses in Prediction Studio
- CONFIGURATION IN PROGRESS
- The creation of artifacts is in progress. This status appears after you trigger a model update request.
- CONFIGURATION FAILED
- The creation of artifacts failed. The mapping of predictors was not successful. Open the model by clicking its name, and read the error message in the Model comparison window. Resolve the error (for example, by fixing the mapping of predictors) and resume the analysis by saving your changes. If necessary, you can reject the model.
- VALIDATION IN PROGRESS
- The creation of artifacts is complete. The comparison of the current model with the candidate model is in progress.
- VALIDATION FAILED
- The creation of artifacts failed. This status appears when an error occurs during the creation of the required artifacts, or during analysis. Open the model by clicking its name, and read the error message in the Model comparison window. Resolve the error and resume the analysis by saving your changes or reject the model.
- READY FOR REVIEW
- Validation is complete and a data scientist can review the model update.
- ACCEPTED
- A data scientist approved the model update with a comment on the approval. The candidate model was added to the prediction in shadow mode or as an active model.
- REJECTED
- A data scientist rejected the candidate model with a comment on the rejection. For example, by reviewing the model comparison charts, the data scientist can evaluate the performance of the candidate model as insufficient.
- SHADOW
- The new model shadows the old model, that is, the new model receives production data and generates outcomes. The system tracks the outcomes of the shadow model, but does not use them to make business decisions. You can monitor the performance of the shadow model, and decide whether to promote it as the new active model or reject it.
Email notifications
Prediction Studio sends email notifications to data scientists and other stakeholders to inform them about the following key events in a model update:
- A model update is ready for review and pending approval.
- A model update failed during analysis (configuration and validation stages). The email contains the reason for the failure. For example, a model update can fail because the predictors are not mapped correctly or because a shadow model is already deployed for the active model.
In environments with Pega Customer Decision Hub, the system creates a Change prediction request in Pega 1:1 Operations Manager to process the model update that a data scientist triggers from Prediction Studio. Data scientists and other stakeholders receive email notifications to inform them about the status of the change request in the following situations:
- The change request is successfully resolved. The email contains the expected deployment date on production.
- A revision manager locked the revision that is associated with the change request. Deployment of the model is on hold.
- A revision manager withdrew the revision that is associated with the change request. As a result, the change request is also withdrawn. The branch (for example, M-7012) that contains the model update is deleted in Prediction Studio.
Previous topic Active and candidate model type combinations Next topic Active and candidate model comparison charts