After servicing a call and addressing the customer needs, the actual call context becomes apparent. To compare the actual context with the predicted call context, a response strategy captures the outcomes of every call. The difference between actual and predicted outcomes determines the accuracy or predictive power of the model. The smaller the difference, the better the model is at predicting customer behavior.

To gather responses, create a response strategy that captures outcomes.

Use case

uPlusTelco wants to improve the experience of their customer support by predicting the reason for each customer call. To achieve that goal, the data analytics team built a predictive model and uploaded it to Prediction Studio, while a system architect created a decision strategy with that model, and deployed that strategy in a decision data flow.

Your role as a system architect is to create a response strategy that references the Predict Call Context model, to gather responses and monitor the predictive power of the model.

Before you begin

Create a decision strategy for the predictive model. See Creating a decision strategy.

Creating a response strategy for monitoring predictive models

1. In Dev Studio, click Create > Decision > Strategy.
2. On the New tab, configure the basic strategy settings:
   1. In the Label field, enter a name, for example:

      Set My Responses to Monitor

   2. In the Context section, enter the same class and open ruleset version as for the model that you imported, for example:

      Apply to: DMOrg-DMSample-SR-PMM

      Development branch: No branch

      Add to ruleset: DMSample

      Ruleset version: 08-01-99

      

3. Verify the initial settings, and then click Create and open.
4. On the strategy canvas, add the external input shape for ingesting data by right-clicking the canvas and selecting Enable external input.
5. Define the model outcome that you want to monitor:
   1. Add the Set property shape by right-clicking the canvas and selecting Enrichment > Set property.
   2. Connect the Set property shape to the External Input and the Results shape.
   3. Double-click the Set property shape.
   4. In the Set property properties dialog box, enter a name for the shape, for example:

      Set My Call Context Outcome

   5. In the Define action, target, and source section, add a new action for setting the model objective by clicking Add item.
   6. In the new row, in the Target column, enter: .pyPrediction
   7. In the Source column, enter the model objective label that you set during the PMML import operation, for example:

      "CallContext"

      To enable response capture, the model objective label must be the same as the .pyPrediction parameter value in the response strategy, for example:

      
   8. Add a new action for setting the model outcome by clicking Add item.
   9. In the new row, in the Target column, enter: .pyOutcome
   10. In the Source column, enter the model outcome that you want to monitor, for example:

       @if((.pxSegment=="Customer Service"&&@random()<0.20),"Complaint",.pxSegment)

       

   11. Verify the settings, and then click Submit.
6. On the strategy canvas, verify that you have all the following shapes, and then click Save:

   

Deploying a strategy in a response data flow

Reference your strategy in a response data flow to gather customer responses for analysis.

For the purpose of this tutorial, use an existing data flow - Set Response To Monitor - as the baseline. This model provides preconfigured elements that you can use to effectively run your strategy:

• The source data set with decision results that are collected in the decision data flow.
• The target data flow for analyzing the outcomes, based on their Interaction ID.

After running this response data flow with your own strategy, the Analytics data flow runs the outcome analysis for predictive model monitoring.

1. In Dev Studio, click Records > Data Model > Data Flow.
2. On the list of the Data Flow rule instances, locate and click SetResponsesToMonitorModels.
3. On the data flow canvas, click Save as.
4. In the New tab, configure the basic data flow settings:
   1. In the Label field, enter a name, for example:

      Set Responses to Monitor My Models

   2. In the Context section, enter the same class and open ruleset version as for the strategy that you created, for example:

      Apply to: DMOrg-DMSample-SR-PMM

      Development branch: No branch

      Add to ruleset: DMSample

      Ruleset version: 08-01-99

5. Verify the initial settings and click Create and open.

6. On the data flow canvas, reference your response strategy:

   1. Double-click the strategy shape.

   2. In the Edit: Decision strategy configurations dialog box, in the Strategy field, delete the current entry and enter the name of your decision strategy, for example:

      SetMyResponsestoMonitor

   3. In the Mode field, select Capture response for previous decision by interaction ID.

      By turning this setting on, you configure the strategy to retrieve the adaptive inputs and strategy results for the interaction ID.

   4. In the Interaction ID field, enter: .pxInteractionID

   5. Leave all other settings unchanged. Click Submit.

      

       

7. On the strategy canvas, verify that you have all the following shapes and click Save:

Conclusions

You have defined the objective and outcome of your Predict Call Context model and applied your strategy in a response data flow, which enables capturing responses to monitor the predictive performance of your model.

What do to next

Gather customer responses to produce statistics and analyze the response data by using varied predictive power metrics. See Monitoring predictive performance.

For more information on response strategies, see Headless decisioning.

To view the main process outline for this tutorial, see Monitoring predictive models.