A / B Test Scenario

In a normal Campaign (without Cells and Splits), the Campaign screen presents the user with four chevrons across the top of the screen that represent the four key steps in a Campaign -- "Setup," "Content," "Review," and "Status."

When you add Cells and Splits to the Campaign, the Campaign screen presents the user with three additional steps, representing the Cells and Splits workflow -- "Setup Cell Tree," "Setup Testing Elements," and "Setup and Launch Campaign."

The Marigold Help Center provides access to a wide range of enablement videos on many Marigold solutions that you can take as needed. Click here to browse to the video on how to use A / B Tests.

To split your Campaign into multiple cells:

In the Tool Bar on the Campaign screen, click Actions > Cells & Splits. A confirmation dialog box is displayed; click yes. The "1. Setup Cell Tree" screen is displayed.
The "This Cell's Code" field is automatically populated with a default cell code consisting of "CELL_" followed by the Object Reference ID for this Campaign. For example: "CELL_15691." Optionally, enter a custom cell code in this field.

Note: If you created this Campaign by copying an existing Campaign, please note that the new Campaign will contain the same Cells & Splits, with the same Cell Codes as the base Campaign. If you don't modify those Cell Codes in the new Campaign, you may unintentionally reuse those Cell Codes, which may cause unexpected results to appear in your Campaign reports. In most cases, you'll want to modify the Cell Codes in the newly copied Campaign.

Click split into testing cells.
Select the "A/B Test" radio button.

Optionally, if you need to remove the cells and splits from a Campaign, click Actions > Undo Cells & Splits. A confirmation dialog box is displayed; click yes.

Split Methods

The next step in the process is to determine the method used to split your Campaign Audience, and to create the desired hierarchy of parent cells and sub-cells.

From the "Split by" drop-down menu, select the split method you want to use:

Split by PercentagesSplit by Percentages

Splitting by percentage allows you to assign a specific percentage of the parent Audience to each sub-cell.

From the "Split by" drop-down menu, select "Percentages."
The "Order By" drop-down menu allows you to determine how you want to order the recipients within the Campaign Audience. From this menu, select either "Random," or select a specific field on the Campaign's source table. If you select a field by which to sort, select a sort order -- either "Ascending" or "Descending."

Note: For an A / B Test, you likely want to keep the default option of "Random," so that you don't inadvertently skew your test results.

In the "Add Cell" text field, enter the desired number of sub-cells you want to create, then click the add button (plus-sign icon). The system adds the indicated number of sub-cells.
The new cells are given a default cell code consisting of the parent cell code (defined above), and appended with "CELL_" and a counter. For example" Parent_CELL_1." Optionally, enter a custom cell code in the "Cell Code" field.
In the "Percentage" field next to the sub-cell, enter the desired percentage value.
If you need to remove a cell, click the remove button ("X" icon) within the cell list.
The system will automatically create a remainder cell for you called the "Winner Cell." This cell represents the bulk of your audience, who will receive the winning conditions, once your A/B test results are complete.

Split by AmountsSplit by Amounts

Splitting by amount allows you to assign a specific record quantity to each sub-cell.

From the "Split by" drop-down menu, select "Amounts."
The "Order By" drop-down menu allows you to determine how you want to order the recipients within the Campaign Audience. From this menu, select either "Random," or select a specific field on the Campaign's source table. If you select a field by which to sort, select a sort order -- either "Ascending" or "Descending."

Note: For an A / B Test, you likely want to keep the default option of "Random," so that you don't inadvertently skew your test results.

In the "Add Cell" text field, enter the desired number of sub-cells you want to create, then click the add button (plus-sign icon). The system adds the indicated number of sub-cells.
The new cells are given a default cell code consisting of the parent cell code (defined above), and appended with "CELL_" and a counter. For example "Parent _CELL_1." Optionally, enter a custom cell code in the "Cell Code" field.
In the "Amount" field next to the sub-cell, enter the desired size of the sub-cell.
If you need to remove a cell, click the remove button ("X" icon) within the cell list.
The system will automatically create a remainder cell for you called the "Winner Cell." This cell represents the bulk of your audience, who will receive the winning conditions, once your A/B test results are complete.

Split by Field ValuesSplit by Field Values

Splitting by Field Values is a bit different than the other split methods, because the system generates all of the sub-cells for you, rather than you having to create them. When you split by Field Values, you select a field on the Campaign source table, and the system automatically generates a sub-cell for every discrete value in that field.

From the "Split by" drop-down menu, select "Field Values." Please note that if you switch to splitting by "Field Values" from any other split method, and you previously defined sub-cells, the system displays a warning that your existing cell list will be deleted and replaced with the system-generated cell list; click ok to confirm.
The "Order By" drop-down menu allows you to determine how you want to order the recipients within the Campaign Audience. From this menu, select either "Random," or select a specific field on the Campaign's source table. If you select a field by which to sort, select a sort order -- either "Ascending" or "Descending."

Note: For an A / B Test, you likely want to keep the default option of "Random," so that you don't inadvertently skew your test results.

From the "Add by Field" drop-down menu, select the field that you want to use to create the split, then click the add button (plus-sign icon).
If the selected field contains more than 25 unique values, a warning is displayed, alerting you to the number of cells that will be added to the Campaign, and asking for confirmation. This feature is intended to prevent accidentally adding a large number of cells to the Campaign. Click ok to confirm, or cancel to cancel. Records that don't have a value in this field will not be included. If no records in the entire table have any values in this field, an error message is displayed.
The system builds and populates the cell list, with a sub-cell created for each unique value within the selected field. The default cell code for each sub-cell is the parent cell code, followed by the value. For example: "Parent_Gold." Optionally, enter a custom cell code in the "Cell Code" field.
Optionally, if you need to remove a cell, click the remove button ("X" icon) within the cell list.

Note: The "Field Values" split method allows you to select only one field by which to split, within the same parent cell. If you want to select a different field by which to create the split, the system displays a warning message that your existing cell list will be deleted and replaced; click ok to confirm.

The system will automatically create a remainder cell for you called the "Winner Cell." This cell represents the bulk of your audience, who will receive the winning conditions, once your A/B test results are complete.

Split by FilterSplit by Filter

Splitting by Filter allows you to use a Filter to define the Audience for each sub-cell. Filters execute from the top-level of the cell hierarchy downward. This means that the selected Filter for a sub-cell will only query records that were selected by the parent Campaign Filter

From the "Split by" drop-down menu, select "Filters."
The "Order By" drop-down menu allows you to determine how you want to order the recipients within the Campaign Audience. From this menu, select one of the following options:
- Random: This option will sort recipients randomly. Also select a sort order -- either "Ascending" or "Descending."
- Use Filter Sort: This option allows you to override the Campaign-level sort options, and instead sort the recipients according to the sort sequence defined within the selected cell-level Filter. This feature allows you to set more complex sorting rules using joins at the Filter level, and have them applied in the Campaign. If a cell has an amount and a Filter assigned, and is the first cell in the Campaign (or the only cell with that Filter), the audience for that cell will be pulled from the top 'X' amount of the Filter. If a cell has an amount and a Filter assigned, and is not the first cell with that Filter, the audience for the cell will be pulled from the top 'X' amount of the Filter minus the audience for the cells with the same Filter that came before it.
- Sort Field: Select a specific field on the Campaign's source table by which you want to sort the recipients. Also select a sort order -- either "Ascending" or "Descending."

Note: For an A / B Test, you likely want to keep the default option of "Random," so that you don't inadvertently skew your test results.

In the "Add Cell" text field, enter the desired number of sub-cells you want to create, then click the add button (plus-sign icon). The system adds the indicated number of sub-cells.
The new cells are given a default cell code consisting of the parent cell code (defined above), and appended with "CELL_" and a counter. For example "Parent _CELL_1." Optionally, enter a custom cell code in the "Cell Code" field.
In the "Filter" field next to the cell, select the desired Filter using one of the following methods:

To search for an existing Filter, begin typing the Filter name, then select the desired Filter from the drop-down menu.
To browse to and select an existing Filter, click the browse button (magnifying glass icon).

Optionally, to view the Filter details, click the "jump-to" button (green up-arrow icon). The Filter screen is displayed and populated with the details of this Filter.
Optionally, you can further limit a cell audience by entering a maximum record quantity in the "Amount" field. If you don't enter a quantity in this field, then the system will populate the cell with the entire Filter result set.
If you need to remove a cell, click the remove button ("X" icon) within the cell list.
The system will automatically create a remainder cell for you called the "Winner Cell." This cell represents the bulk of your audience, who will receive the winning conditions, once your A/B test results are complete.

Note: You can't use different split methods within the same parent cell.

The "Channel" column allows you to identify a marketing Channel that will be used for each sub-cell, or to select "Control" to designate a sub-cell as a Control Cell. From the drop-down menu in this column, selected the desired Channel for each sub-cell, or select "Control." If you select "Control" for a sub-cell, a confirmation dialog box is displayed; click ok to confirm. Please note that once you make this Channel selection and save the Campaign, you can't later change the Channel for a sub-cell.

Note: The drop-down menu in the "Channel" column has "Email" selected, and is disabled. A / B Testing is available only in the email Channel, so this option can't be modified.

Optionally, if you want to rearrange the sub-cells, click on the gray box to the left of the sub-cell row, and drag-and-drop the row into its desired new position.
In the Tool Bar, click save.

Note: When creating the hierarchy of cells for an A / B Test scenario, the platform allows you to create only one layer of "child" cells beneath the main "parent" Campaign. In an A / B Test, you can't further divide those "child" cells into "grandchild" cells. Unlike a Split Cells Campaign (which can create any number of levels), an A / B Test Campaign can have only a single level of child cells beneath the parent Campaign.

Cell Tree Features

The Cell Tree is an important tool to help you keep track of WHAT you're editing. Within the Cell Tree, the currently selected cell is always highlighted in bold. In addition to serving as a navigation tool, the Cell Tree also provides quick access to the following feature:

Rename a Cell: Within the Cell Tree, hover your mouse cursor over the cell that you want to rename, and click the "edit" icon. The "Edit Cell Code" pop-up window is displayed. In the "Cell Code" field, enter a new cell code, then click ok.

Winning Options

The next step is to configure how you want the system to determine the "winner" in your A/B Test.

From the "Send Winning Campaign" drop-down menu, select one of the following deployment options:

ManuallyManually

If you select "Manually," the system won't determine a winner, and won't automatically deploy messages to the Winner Cell; see the "Send Winner" section below for more details on how to manually send messages to the Winner Cell.

With a manual deployment, you have the option to configure a reminder email notification. The platform will send this notification at the designated interval, if the Winner Cell has not yet been manually deployed. This notification is intended as a reminder, so that you don't forget to deploy the Winner Cell.

To configure the reminder notification:

Enter one or more email addresses for the reminder notification recipient(s). Multiple email addresses must be separated by commas.
Select when to send the reminder notification -- either "x" number of hours after launch, or "x" number of days after launch.

AutomaticallyAutomatically

If you select "Automatically," the system will determine the winning variable conditions based on the business rules you provide below; in addition, the system will automatically send out messages to the Winner Cell, using the winning variable conditions.

Identify the business rules for defining the winning conditions. From the "Based on" drop-down menu, select one of the following winning conditions:

Unique Reads (includes "opens" and "clicks")
Unique Clickers
Unique Opens
Web Events
Projected Open Rate

If you selected "Unique Reads" or "Unique Opens" option for the "Based on" drop-down menu, you can exclude the pre-cached opens by proxy server by selecting the "Use real open activity, excluding Pre-Cached opens" check box, so that winner selection is only based on the verified opens/reads.
If you selected "Unique Reads" or "Unique Clickers" option for the "Based on" drop-down menu, you can exclude the fake clicks by the Bots by selecting the "Use real click activity, excluding Bot clicks" check box, so that winner selection is only based on the real human clicks.
By default, the system will identify the winner as the sub-cell with the most activity of the type you selected in step 2. Optionally, you can define a threshold that states the winning sub-cell must surpass the other sub-cells by a certain percentage. To define this threshold, select "at least" from the "with improvement of" drop-down menu, then enter a percentage amount.
Define the testing period. Once this testing period is finished, the system will begin the process of identifying the winner. The testing period options are:

To define a testing window, click the after toggle button. Enter a duration in hours or days (the default is 3 hours). This window begins once all test emails have been sent, and ends once the specific duration has elapsed.
To enter a specific date / time, click the start on toggle button. Enter the date and time when the winner calculation process should begin.

Define the tiebreaker rules in the event that no sub-cell can be determined to be the winner. From the "If no winner (tie) send" drop-down menu, select one of the following options:

Random leader -- The system randomly selects one of the tied sub-cells to be the winner.
None -- The system will not break the tie; instead, you'll have to manually select the sub-cell to be the winner. See the "Send Winner" section below for more details on how to manually send messages to the Winner Cell.

Select a specific sub-cell to be the winner in the event of a tie.
Optionally, if you want the system to send out a notification email when the A / B Test is completed, enter one or more email addresses in the "When test complete alert" field.

In the Tool Bar, click save.

Variable Conditions

The next step is to select what Campaign element or elements are going to be different between each of the sub-cells.

Beneath the cell list, from the "Elements to split" drop-down menu, select one of the following options:

None: All sub-cell values are inherited from the parent cell.
Subject line: Set a subject line for each sub-cell in order to test the effectiveness of a subject line.
Content: Define message content for each sub-cell in order to test the effectiveness of different content.
Any: This option presents you with a list of all the possible Campaign elements that can be varied, grouped into different categories. Place a check mark next to one or more elements. The possible elements that can be varied are:

General Settings		Email Components		Response Options		Sending Options
Content / Links Seed List Deduping Exclusion List Metadata Special Processing Step		From Name and Sender Profile To Name Subject Line Reply Handlers Email Headers Preheader		PURL		Sending Schedule Messages Per Hour

In the Tool Bar, click save.

Populate Sub-Cells

After you've defined all of the options as described above, the next step is to actually populate the different Campaign variants in each of the different sub-cells.

Sub-cells automatically inherit the characteristics of their parent cell, except for the particular element, or elements, that you're changing. For example, let's say you chose to modify the "Subject line." You would enter unique Subject Lines into each sub-cell. These sub-cells would inherit all of the other Campaign elements (From Name, From Address, content, etc.) from the parent cell.

The system supports two different methods of populating the sub-cells in your Campaign. Depending on the complexity of your cell hierarchy, and the number of elements being tested, you may find one method works better than the other. The first method is to select the Campaign element being tested, and to define that element in every sub-cell. Then you would select a different Campaign element, and repeat that process.

The other method is to focus on a single sub-cell, and to populate all the different variable elements in that sub-cell. Then, you would move on to the next sub-cell and repeat that process.

Both of these methods are described below.

Define a Single Element Across All Sub-Cells

This method allows you to edit multiple sub-cells from the same screen, by first selecting the Campaign element that you want to test.

At the top of the screen, click "2. Setup Testing Elements." This screen allows you to set up the different variants for each sub-cell.
From the "Elements to Test" drop-down menu, select the element that you want to set up (or use the up-arrow and down-arrow buttons to navigate within this menu). This menu contains all of the Campaign elements that you selected on the previous screen. For example, if you selected "Subject Line" and "Seed List" as the elements that you want to test, then you'll see both those items listed in this drop-down menu.
For a sub-cell, select or enter the desired option. If a test element wasn't selected for a particular sub-cell, the message "Inherited from parent" is displayed next to that sub-cell. The different options for the different Campaign elements are as follows:

Content / Links: The system displays a thumbnail preview of the content (when available). Optionally click preview to see a larger preview version of the content. Optionally click edit to edit the content within the Advanced Editor, then click "2. Setup Testing Elements" to return to this screen. Optionally select the desired character encoding method from the "Message Encoding" drop-down menu.
Seed List: To select a Seed List, click add seed list, and browse to and select the desired Seed List. To use a Filter as the Seed List, click add filter as seed list, and browse to and select the desired Filter. To remove a Seed List, click the "X" icon next to the Seed List name. Please note that a sub-cell can have only one Seed List selected.
Deduping: Select your deduping logic, including a field by which to dedupe, a Filter, and / or a sort method and sort direction.
Exclusion List: To select an Exclusion List, click add exclusion list, and browse to and select the desired Exclusion List. Repeat this step as needed to add additional Exclusion Lists to a sub-cell.
Metadata: If you selected Metadata as an element to test, every Metadata field in your account will be displayed within the "Elements to Test" drop-down menu. Select the desired Metadata field. For string fields, enter the desired value (including optionally dragging in personalization fields from the Personalization Pane). For date fields, select the desired date from the calendar pop-up. For pre-defined fields, select the desired value from the drop-down menu. For integer fields, enter the desired number.
Special Processing Steps: From the "Add Special Processing Steps" drop-down menu, select the desired stored procedure, then click the add button (plus-sign icon). The drop-down menu contains every custom stored procedure that has been configured for your account. Repeat this step as needed to add multiple stored procedures.
From Name and Sender Profile: From the "Sender Profile" drop-down menu, select the desired Sender Profile. Optionally, from the "From Address" drop-down menu, select a From Address. Optionally, in the "From Name" field, enter the desired value (including optionally dragging in personalization fields from the Personalization Pane).
To Name: Enter the desired "To Name" value (including optionally dragging in personalization fields from the Personalization Pane).
Subject Line: Enter the desired Subject Line value (including optionally dragging in personalization fields from the Personalization Pane).
Reply Handlers: Optionally select a Forwarding Handler from the "Select a Forward Profile" drop-down menu. Optionally, select an Auto-Reply Handler from the "Select a Reply Profile" drop-down menu.
Email Headers: Optionally, enter one or more Blind Carbon Copy email addresses and / or a Reply To email address.

Preheader: The email preheader of the campaign, the invisible text that is added to the top of an email’s HTML to become the preview, instead of the first email text. To find and use the best combination of Preheader and Subject Line, you can now select them both as shown below, and use Projected Open Rate as the winner criteria to find the combination that received the highest open rate without noise from Apple MPP.

Note: Because the preheader is automatically inserted directly after the opening body tag, if your HTML email is missing the body tag, the preheader will not appear.

PURL: To search for an existing PURL Domain, begin typing the PURL Domain name, then select the desired PURL Domain from the drop-down menu., Or, click the browse button (magnifying glass icon) to browse to and select the desired PURL Domain. From the drop-down menu, select either "Prefix" or "Suffix."
Sending Schedule: Define the Message Deployment Schedule for a sub-cell. Select "Start after queuing" to deploy immediately after message creation is finished, or select "Start on" to define a custom deployment start date / time. To define a "send window," click the "Advanced Options" link, then select the "Send Only Between" tab, and enter a specific start time and end time. Also, within the "Advanced Options" section, you can select either "Send All Messages," or "Stop Sending On." If you select "Stop Sending On," enter a stop date and time.
Messages Per Hour: Define the flow rate for a sub-cell by indicating the number of messages (in an increment of 500) to be sent per hour.

The Campaign screen supports an optional "push" feature, that allows you copy test values from a parent cell down into all of its sub-cells. First enter or select the desired test value in the parent cell, then click the "Copy" icon. A confirmation dialog box is displayed; click ok.
Repeat steps 2 through 4 as needed for any additional Campaign elements that you need to define.
In the Tool Bar, click save.

Note: For sub-cells that you designated as Control cells, you won't be able to edit anything, as the Control group doesn't actually receive any Campaign messages.

Define All Elements Within a Single Sub-Cell

This method allows you to define all the variable elements within a specified sub-cell.

At the top of the screen, click "3. Setup and Launch Campaign."
This screen displays the Cell Tree within a pane on the left-hand side of the screen. If you want more horizontal screen space, you can optionally "dock" the Cell Tree at the top of the screen. Within the Cell Tree pane, click the "dock" icon. The system moves the Cell Tree to a drop-down menu at the top of the screen. Conversely, to move the Cell Tree back to the left-hand pane, click the "undock" icon.
Select the desired sub-cell, either from the Cell Tree pane, or from the Cell Tree drop-down menu (if you docked the Cell Tree). The system displays a "stripped down" version of the regular Campaign screen, displaying only those Campaign elements that you defined as being variable for the selected sub-cell. The different options for the different Campaign elements are as follows:

Content / Links: Click the "Content" tabview the Advanced Editor, where you can enter the content for this sub-cell. Optionally select the desired character encoding method from the "Message Encoding" drop-down menu.
Seed List: To select a Seed List, click add seed list in the "Audience" section, and browse to and select the desired Seed List. To use a Filter as the Seed List, click add filter as seed list, and browse to and select the desired Filter. To remove a Seed List, click the "X" icon next to the Seed List name. Please note that a sub-cell can have only one Seed List selected.
Deduping: In the "Audience" section, select your deduping logic, including a field by which to dedupe, a Filter, and / or a sort method and sort direction.
Exclusion List: In the "Audience" section, click add exclusion list, and browse to and select the desired Exclusion List. Repeat this step as needed to add additional Exclusion Lists to this sub-cell.
Metadata: Click the "Set Metadata" link near the top of the screen. The "Metadata" pop-up window is displayed, and populated with all of the Metadata fields in your client account. Enter or select the desired Metadata values, then click ok.
Special Processing Steps: From the "Add Special Processing Steps" drop-down menu, select the desired stored procedure, then click the add button (plus-sign icon). The drop-down menu contains every custom stored procedure that has been configured for your account. Repeat this step as needed to add multiple stored procedures.
From Name and Sender Profile: In the "Envelope" section, select the desired Sender Profile from the "Sender Profile" drop-down menu. Optionally, from the "From Address" drop-down menu, select a From Address. Optionally, in the "From Name" field, enter the desired value (including optionally dragging in personalization fields from the Personalization Pane).
To Name: In the "Envelope" section, enter the desired "To Name" value (including optionally dragging in personalization fields from the Personalization Pane).
Subject Line: In the "Envelope" section, enter the desired Subject Line value (including optionally dragging in personalization fields from the Personalization Pane).
Reply Handlers: In the "Envelope" section, optionally select a Forwarding Handler from the "Select a Forward Profile" drop-down menu. Optionally, select an Auto-Reply Handler from the "Select a Reply Profile" drop-down menu.
Email Headers: In the "Envelope" section, optionally, enter one or more Blind Carbon Copy email addresses and / or a Reply To email address.
PURL: In the "Envelope" section, search for an existing PURL Domain by typing the PURL Domain name, then select the desired PURL Domain from the drop-down menu., Or, click the browse button (magnifying glass icon) to browse to and select the desired PURL Domain. From the "URL Type" drop-down menu, select either "Prefix" or "Suffix."
Sending Schedule: Click the "Review" chevon. Within the "Campaign Schedule" section, select "Start after queuing" to deploy immediately after message creation is finished, or select "Start on" to define a custom deployment start date / time. To define a "send window," click the "Advanced Options" link, then select the "Send Only Between" tab, and enter a specific start time and end time. Also, within the "Advanced Options" section, you can select either "Send All Messages," or "Stop Sending On." If you select "Stop Sending On," enter a stop date and time.
Messages Per Hour: Click the "Review" chevon. Within the "Campaign Schedule" section, enter the hourly flow rate in the "Per Hour Limit" field (in an increment of 500).

Repeat step 3 as needed for any additional sub-cells that you need to populate.
In the Tool Bar, click save.

After you've defined all of the cells and sub-cells using either of the above methods, click "3. Setup and Launch Campaign" to proceed to the regular Campaign life cycle of Proofing, Auditing, and Launching. You can send proofs from within a sub-cell (which sends proofs for only that sub-cell), or from the top-level Campaign (which sends proofs to all sub-cells). Auditing and Launching, however, must occur from the top-level Campaign, and not from a cell.

Send Winner

If you set up automated winning conditions (see "Winning Options" above), the system will determine the winning sub-cell based on the rules you defined. The system will them populate the Winner Cell with the winning variable conditions, and deploy messages to the Winner Cell.

If you opted to manually determine the winning sub-cell, you'll need to select which sub-cell was the winner, then deploy messages to the Winner Cell.

Navigate to and select the Campaign. An A / B Test Campaign that's waiting to send to the Winner Cell audience will have a Campaign status of "Pending Winner."
Click "3. Setup and Launch Campaign."
Click the "Status" tab
The "Review & Approve" section will display your cell tree with all of your child cells, including the Winner Cell. Next to the Winner Cell, click send winner. The "Launch Winner Cell" pop-up window is displayed.
From the drop-down menu, select the winning sub-cell. The system will use the variable conditions in this sub-cell (subject line, content, etc.) to build the messages that get sent to the Winner Cell audience.
Click launch winner.