Understanding the NBAStrategyFramework component test results
After you test the NBA Strategy Framework strategy, use the following troubleshooting tips to help identify the component where actions are dropping out or where the strategy results are unexpected.
Pre-processing extension point
Unless this strategy has been customized, you should not expect any changes here.
Customer Journeys
This strategy assigns Journey Stage data to actions. If there are journey stages configured with actions assigned to them, the ActionRequriesJourney property is set to true on the action form, and there is currently no eligible journey for the action, then the action will be dropped, so it is possible to see a reduction in actions at this point.
Constraints
This strategy applies Contact Policies based on prior outcomes, so you could expect to see actions drop out if they are referenced as Suppressions in the ActionInsights page list in the Data Transform.
For example, actions that have treatments for multiple enabled channels will cause an increase, but actions that have a single treatment for a channel that has been disabled will cause a decrease, although this latter case will be rare.
If any transactional actions are defined, be aware of how they are treated differently.
The second part of the strategy executes adaptive models for each action and channel combination, and this should not in itself cause any changes.
If the Next-Best-Action Designer Channel treatment processing setting is set to false, there are other potential changes:
- For outbound interactions on channels other than Paid, where Outbound Model Maturity is enabled, and where a model is immature (that is, has less than 200 positive responses by default), a significant percentage of contacts (98% by default) will not receive the action. Since the percentage test is evaluated based on a hash of the customer ID and the action ID, this is virtually impossible to control in test scenarios, so to overcome this, consider switching Outbound Model Maturity off during testing, or set the direction to be Inbound and choose a suitable channel.
- Ensure you are familiar with any Model Propensity Thresholds DDR settings, since these may also cause actions to be dropped.
Treatments and channels
If the Next-Best-Action Designer Channel treatment processing setting is set to false, this whole strategy is bypassed and does not cause any changes. Otherwise, the following processing occurs.
The first part is where actions are joined to their treatments. Expect to see no change if each action has a single treatment per channel, or an increase if there may be multiple treatments per channel.
The second part of the strategy executes adaptive models for each action / treatment combination, and this should not in itself cause any changes.
However, as with Action Scoring, there are other potential changes:
- For outbound interactions on channels other than Paid, where Outbound Model Maturity is enabled, and where a model is immature (that is, has less than 200 positive responses by default), a significant percentage of contacts (98% by default) will not receive the action. Since the percentage test is evaluated based on a hash of the customer ID and the action ID, this is virtually impossible to control in test scenarios, so to overcome this, consider switching Outbound Model Maturity off during testing, or set the direction to be Inbound and choose a suitable channel.
- Ensure you are familiar with any Model Propensity Thresholds DDR settings, since these may also cause actions to be dropped.
Universal Control
You should only expect to see a change if the Is In Control Group and Handle Control Group strategies have been customized, which is not common.
Arbitrate
This strategy calculates the priority for each action and treatment, and prioritizes them. There should be no changes to the number of selected actions and treatments.
Channel Logic
This strategy applies channel specific processing, including applying actions limits which may reduce the number of actions and treatments. However, this is normally only done for outbound interactions where the default is one action or treatment per contact per context. Ensure that you are familiar with any Action Limits DDR settings, since these may cause actions to be dropped.
NBA Kill Switch
If the Next-Best-Action Designer Next-Best-Action active setting is set to true, there will be no change. In the unlikely event that the setting is set to false, all actions and treatments will be replaced by the output from the Fallback NBA Strategy.
NBA Post-process extension
Unless this strategy has been customized, you should not expect any changes here.
Previous topic Using the trigger strategy to test the NBAStrategyFramework component Next topic Testing the lower strategies