A/B Testing for JO

1. Fetch the Account ID and Account Name of each account for consideration (along with any other fields you'd want to carry forward)
2. Use a transformation task to add "Account Name Length" to your data set (using LENGTH(field) function).
3. Use a second transformation task to bring forward the data in the second task and add a formula field using the MEDIAN(field) function to calculate the mid point length of characters for all your customer names
4. Use a third transformation task to add a calculated field that is equal to the Account Name Length - Median Account Name Length. 
5. You'll now have a field for each account that you can use that will be a positive number if the account name length is greater than the median, negative if the account name length is less than the median and a few that will be zero where the account name length equals the median.
6. This will split your data set into two roughly equal groups (with a few in the zero spot). 

@dan_ahrens

Would you have a minute to look at my transformations? For some reason this survey failed to send… was trying to use this A/B suggestion.

I’ve included the sources attached here

HI @scott_sigsci - at your last step you are aggregating by company ID name, which instead of pulling in the email address and other relevant data of your contacts, is showing the count and sum of those values. You’ll want to remove that aggregation.

So close... I’ll remove that and have another go. Thank you @dan_ahrens

@dan_ahrens now I remember why I put that in… I keep getting this error. Any thoughts? (Note: I didn’t see this error in the first two transformations)

Hi Scott, without working in your org specifically, it’s hard to dig very deep on troubleshooting and advising using screenshots, but you might find it helpful to do that field length calculation in it’s own step and then merging that data back in with your other company data. Your Gainsight Customer Success Manager can probably work 1 on 1 with you to tune this query. 

Thank you @dan_ahrens , much appreciated.

@dan_ahrens I figured out what the issue is. In this case, you don’t use a “calculated field,” you use “numeric expression.”

@dan_ahrens: Thanks a lot for the idea with LENGTH and MEDIAN above. Very interesting angle to look at it! Is it, however,  – statistically speaking – random enough? Have you maybe considered adding a distinct random function, i.e. similar to RAND() function in MySQL? We currently use exactly that, just with a bit of extra complexity and pushing it to Gainsight from our internal data warehouse.

In any case, I think it would be great to have the fork functionality that @dale_parent suggested above. Having that option directly in the flow and, ideally, also being able to compare the results directly in the Analytics would be absolutely amazing!

Do posts tagged ‘Work around available’ ever get revisited? This would be nice to have as an actual feature and not a 3 task transformation. It also doesn’t solve for powerlists or segments, as that feature grows in adoption.

We are looking at some options to do this in Journey Orchestrator and might have some more updates closer to the start of the new year. 

Is there any update to make this a feature? All of our account IDs are exactly the same length, so this crazy complicated workaround is really not feasible for us. It would be really helpful to have this as a feature with similar parity to other email systems where I can just configure the A/B settings and let it deploy with randomization. 

This would also help with company conversion. My marketing team is showing more interest in using JO rather than Marketo for certain drip campaigns and surveys, but this is a basic functionality they use all the time. 

This is a feature that is being planned for 1H 2021. 

Hello! I wanted to follow up and see if there was still hope for this to be a feature in JO? Our company would love to start doing some A/B testing as we bring more complicated programs into Gainsight and adding the additional transformations would complicate things a bit leaving more room for error when trying to complete the testing. 

@PavanCh could you comment?

@PavanCh any comments on this?

@vmallya could you pl address the latest with this? 

Hello @anirbandutta and everyone on this thread!! 

Despite being on our backlog, this requirement was not picked up due to other conflicts. Although this is a valuable addition to the platform, we will not be able to implement it anytime soon.

If it helps, I’d like to suggest an alternative solution (can share more details if needed) - 

1. We can create a custom dropdown field in the Company object (can have A, B.. values as needed)
2. We then create a rule to update the field value of this field based on some conditions
   • SUBSTRING(field, position, length) - finds the substring based on the position and length.
     • Last digit of GSID (Alphabet or number)
     • If number (odd or even)
     • etc.
3. Create a data set filtered on this field value (A, B, ..etc.)
4. Then use appropriate data sets for respective JO programs.

Thank you. 

Revisiting this thread, as this still seems to be a gap with no built-in solution. Not having A/B testing is a large gap for an email tool, so hoping there is some plan for this since the last post two years ago.