Test mode implementation
Activate test mode to facilitate the testing and debugging of you next-best-action framework. When test mode is active, actions that would otherwise be removed from the strategy results (SR), are instead flagged as being selected for removal.
Each different type of filtering has its own Boolean property which is set to true if an action would have been removed. Whenever any of these properties are set true, the IsTestModeFiltered property is also set true to indicate that the action has been filtered.
Test mode is active when the ExecuteInTestMode property in the Data-Decision-Request-Customer class is set to true. During the framework execution, a Switch rule is used to test ExecuteInTestMode, and the test mode logic will only be executed if it is true, otherwise action filtering will occur as normal. Thus, if test mode is not active, none of the test mode Boolean properties will be set.
ExecuteInTestMode will be set in the data flow that calls the next-best-action strategy framework when using the Customer Profile Viewer, or it may be set in a data transform when using the Test panel in a strategy.
Setting the flags
The filtered flags are set in various ways depending on how actions are being removed from the strategy results.
The following is a list of the flags currently being set, showing the strategies in which they are set, and the method of deployment. They are shown in the order that they are evaluated, although some will depend on whether action or treatment level models are being used.
Property name | Set in strategy | Deployment method | Notes |
IsTestModeFiltered | All of the below | Not applicable | When the test mode is active, this property is set to true when an action would have been filtered by any of the properties below. |
IsSuppressed | Constraints | Data join | The action is excluded by a suppression constraint. |
IsJourneyActionExcluded | Customer Journeys | Filter | The action is excluded because the ActionRequiresJourney is set to true. |
IsNoChannelForAction | Create Eligible Channels | Exclusion | There is no enabled channel for the action's treatments. |
IsOutboundLimitExceeded | Outbound Channel Limits Bundling Settings & Yield Actions Contact Policy Limits | Filter | The action would have exceeded an Outbound channel limit. |
IsMaturityCapExceeded | Handle Model Maturity | Filter | The action model is not mature, and the maturity cap is exceeded for the action and customer combination. |
IsPropensityThresholdExceeded | Apply Propensity Thresholds | Filter | The action exceeded a propensity threshold. |
IsNoTreatmentForChannel | Treatments & Channels | Exclusion | There is no active treatment for an enabled channel. |
IsPriorityThresholdExceeded | Apply Priority Thresholds | Filter | The action exceeded a priority threshold. |
IsInvalidBundleExcluded | Bundling Settings & Yield Actions | Data join | The action is part of an invalid bundle (parent with no children or child with no parent). |
IsYieldToPrimaryExcluded | Bundling Settings & Yield Actions | Filter | The action was redirected to a primary contact and the original is suppressed. |
IsAlternatePrimaryExcluded | Bundling Settings & Yield Actions | Exclusion | The action is valid for multiple primary contacts but only delivered to a single primary contact. |
IsActionLimitExceeded | Apply Action Limits Common | Filter | The action would have exceeded an action limit. |
IsTreatmentExcluded | Final Action Limits And Bundling | Filter | The action or treatment is excluded by channel processing such as treatment placement allocation |
Converting strategies to use test mode
Each of the methods used to deploy test mode to set the filter properties in existing strategies is described below.
Filter deployment method
This method is used where actions are removed from the strategy results using a Filter shape.
The basic idea is to insert a new Set Property shape immediately prior to the filter, and set the Eligible property to the condition currently in the filter. All input connectors to the filter are reconnected to the new Set Property shape, and the Set Property shape is connected to the flter. The filter condition is replaced by .Eligible.
Create an additional Set Property shape with the name Test Mode
Filtered and connect it to the Set Property shape created in the prior step.
This new shape sets the following properties:<Filter Property> = ! .Eligible
.IsTestModeFiltered = @or(.IsTestModeFiltered, <Filter
Property>)
The value of IsTestModeFiltered will be true if any prior filters were true or if the current filter is true – this property is needed to exclude filtered actions from any logic where arbitration is required, since the filtered actions would adversely affect the ranking of the valid actions.
Finally, a Switch rule is used to select the Test Mode Filtered shape if Primary.ExecuteInTestMode is true, otherwise it selects the original Filter shape.
If not executing in test mode, the logic flows through the Filter and the actions are removed without setting any addition properties (other than .Eligible), but when in test mode, the actions are not removed, the new <Filter Property> is set true if the action would have been removed, and IsTestModeFiltered is also maintained.
The following is an example of this method from the Apply Priority Thresholds strategy, where the four shapes shown replace the original single Apply Thresholds Filter shape.
The original Filter condition was:
.Priority > .PriorityThreshold
The new shapes are as follows:
- Exceeds Threshold Check
.Eligible = .Priority > .PriorityThreshold
- Apply Thresholds
.Eligible
- Test Mode Filtered
.IsPriorityThresholdExceeded = ! .Eligible
.IsTestModeFiltered = @or(.IsTestModeFiltered, .IsPriorityThresholdExceeded)
- Test Mode Check
Select Test Mode Filtered if Primary.ExecuteInTestMode
Otherwise Apply Thresholds
Exclusion deployment method
This method is used where actions are removed from the strategy results by a process involving various different sub-strategies, and it is simpler to flag them as a group rather than alter each of the individual sub-strategies.
As with the Filter method, the basic concept is to switch between two paths based on the ExecuteInTestMode property, but in this case the filtered actions are determine by excluding the unfiltered actions that emerge after the filtering process from the actions that were present before the process.
The following is a section from the Final Action Limits And Bundling strategy, where after Apply Final Action Limits has been executed, inbound and outbound channel processing occurs, which may result in actions and treatments being removed, especially for Web and Mobile channels where the top treatments are allocated to placements, and the remainder are dropped.
The following figure shows the logic before the test mode check is added:
The following figure shows the logic after the test mode check is added:
The Unfiltered Treatments Set Property shape is connected from the output of the filtering process (Inbound Channels Check) to set:
.IsTreatmentExcluded = false
The Exclude Unfiltered Exclusion shape is connected from the last shape prior to the filtering process (Apply Final Action Limits), and excludes unfiltered treatments as shown below.
Exclude Unfiltered is then connected to a new Filtered Treatments Set Property shape to set:.IsTreatmentExcluded = true
.IsTestModeFiltered = true
Filtered treatments and unfiltered treatments are then merged into Test Mode Filtered,
which sets no properties, and finally the Test Mode Check Switch rule selects the
appropriate path.Select Test Mode Filtered if Primary.ExecuteInTestMode
Otherwise Inbound Channels Check
Data join deployment method
This method is used to detect actions that are removed using one or more Exclusion rules, and varies depending on the implementation.
As with the other methods, the basic concept is to switch between two paths based on the ExecuteInTestMode property, but in this case the test mode path replaces the exclusion rules with data joins (with an outer join) using the same join conditions as the exclusion conditions, and sets the filter flag in the data mapping. A final Test Mode Filtered Set Property shape sets IsTestModeFiltered appropriately.
The Constraints strategy is a good example.
The top section has the data joins for detecting the filtered actions, each of which sets IsSuppressed = true if a suppression is detected. In this case the contact policy is also retained for reporting.
The Test Mode Filtered Set Property shape sets:.IsTestModeFiltered = @or(.IsTestModeFiltered, . IsSuppressed)
The Test Mode Check Switch rule selects the appropriate path:Select Test Mode Filtered if Primary.ExecuteInTestMode
Otherwise Exclude Suppressed Actions
When to use IsTestModeFiltered
The IsTestModeFiltered property indicates that an action has been filtered by a prior process for some reason, and if ExecuteInTestMode were false it would not be present in the strategy results.
For many of the filtering processes, we evaluate whether an action will be filtered, even if it has already been filtered, as this can obviate having to perform iterative tests to see at which stage an action will be dropped.
However, for processes where actions are eliminated because of their rank (e.g. Final Action Limits, or Outbound Channel Limits), we need to exclude any actions that would otherwise not have been present, since this would change the ranking of the actions. In these cases, actions where IsTestModeFiltered is true need to be excluded from the ranking process.
Previous topic Testing the Next-Best-Action strategy framework Next topic Using test mode features