Evaluating candidate models with MLOps
After you add a candidate model to a prediction, Prediction Studio configures and validates the new model, and provides comparison data to help you evaluate the new model. Decide whether you want to approve the new model for deployment to production or reject it.
Model evaluation is part of the Machine Learning Operations (MLOps) feature of Pega Platform.
Pega Customer Decision Hub
- In the navigation pane of Prediction Studio, click Predictions.
- From the list of predictions, open a prediction that contains a candidate model, and then click the Models tab.
- Display the candidate model by expanding the twist arrow to the left of the current model.
- Ensure that the status of the candidate model is READY FOR REVIEW. If the configuration or validation of the model is still in progress, wait several seconds, and then refresh the page by clicking .
- Open the candidate model by clicking its name.
- Optional: If you want to add a data set for the comparison or use a different data set,
click Configure data set.
- In the Validation data set window, select the data
set and the outcome column to compare the two models.The outcome column contains the response labels for the outcome.
- Review the properties in the data set and their values.In the Records field, you can select the number of records for the data set preview. The values are recalculated after each change.
- Click Submit.
- In the top-right corner of the Model comparison window, click Save.
- In the Confirm save dialog box, click Yes.
- Wait several seconds for the validation to complete, and then refresh the page by clicking .
- When the status of the model changes to READY FOR REVIEW, open the model by clicking its name, and review the new comparison charts.
- In the Validation data set window, select the data
set and the outcome column to compare the two models.
- Review the comparison charts to decide which model is better for your use case.
- Optional: Export the analysis data to a CSV file by clicking Download analysis data.
- In the top-right corner, click Evaluate.
- In the evaluation window, approve or reject the model:
- To let the new model shadow the current model, click Approve new candidate model and start shadowing (recommended).
- To replace the current model, click Approve candidate model and replace current active model.
- To reject the new model, select Reject candidate model.
- In the Reason field, provide a comment on the approval or rejection.
- Click Save.
Previous topic Replacing models in predictions with MLOps Next topic Promoting shadow models with MLOps