Skip to main content
steve_davis
steve_davis
Gainsight Employee ⭐️⭐️
March 29, 2017
Discovery

A/B Testing for JO

Related products:CS Journey Orchestrator
  • March 29, 2017
  • 31 replies
  • 965 views

We are looking for an easy way to split a power list in half to send a message to one sub group and a slightly different one to another sub group to compare the results. Of course, we can set up a flag and populate it. I am just looking to see if we can do it easier than that. Is this something we have considered as a feature for JO?

      31 replies

      dan_ahrens
      dan_ahrens
      Expert ⭐️⭐️⭐️
      March 29, 2017
      Could you use the variant feature (with different message content instead of language)?
      VP of Customer Success at Forcepoint
        dstokowski
        dstokowski
        Gainsight Employee ⭐️⭐️
        March 29, 2017
        I was thinking the same Dan.  You would want to pick an attribute to use to split the list.
        nora_soza
        nora_soza
        Helper ⭐️
        March 29, 2017
        Beauty of supporting A/B testing is the powerlist is randomized in system. I do think variant is the right mechanism to leverage though!
          praneet
          praneet
          Helper ⭐️⭐️⭐️
          March 30, 2017
          Totally agreed to all of your request, however today we don't have randomizing email feature available. We have this as our roadmap items.
            marianne
            marianne
            Contributor ⭐️⭐️
            April 17, 2017
            Hi Praneet, is there an ETA for adding this feature? We're eager to move more of our customer emails to Gainsight vs the email marketing automation platform our Marketing team uses, but the lack of easy, random A/B testing functionality is a major obstacle to this.
              dale_parent
              dale_parent
              Helper ⭐️
              June 11, 2018
              We also have a use case for an A/B testing feature combined with the Survey tool and advanced outreaches.  It would be great to have the ability to configure a fork in the AO workflow so participants receive one of two possible email template/survey combinations.  Perhaps build this into the conditional wait configuration.
                dan_ahrens
                dan_ahrens
                Expert ⭐️⭐️⭐️
                June 11, 2018
                So here's an idea for doing some A/B style testing using automation. This leverages the LENGTH(field) and MEDIAN (field) functions in rules. https://support.gainsight.com/Product_Documentation/Rules_Engine/Admin_Configuration/Formula_Builder...

                Steps below as follows:
                1. Fetch the Account ID and Account Name of each account for consideration (along with any other fields you'd want to carry forward)
                2. Use a transformation task to add "Account Name Length" to your data set (using LENGTH(field) function).
                3. Use a second transformation task to bring forward the data in the second task and add a formula field using the MEDIAN(field) function to calculate the mid point length of characters for all your customer names
                4. Use a third transformation task to add a calculated field that is equal to the Account Name Length - Median Account Name Length. 
                5. You'll now have a field for each account that you can use that will be a positive number if the account name length is greater than the median, negative if the account name length is less than the median and a few that will be zero where the account name length equals the median.
                6. This will split your data set into two roughly equal groups (with a few in the zero spot). 
                Then just use the compare field you created in step 5 to segregate your email variants where those where the compare field is greater or equal to 0 gets one variant and those where the compare is less than zero get the other variant. 

                This is just one example. You could use the basic principle on any field data that is present on all accounts and I'm sure some may even devise more complex methods using the other statistical functions like Correlation, Covariance, Std Deviation/Variance and Median. 

                Average should be used with caution as a single large outlier can skew the entire dataset's mid point. 
                VP of Customer Success at Forcepoint
                  scott_sigsci
                  scott_sigsci
                  Helper ⭐️
                  December 9, 2019

                  @dan_ahrens

                  Would you have a minute to look at my transformations? For some reason this survey failed to send… was trying to use this A/B suggestion.

                   

                  I’ve included the sources attached here

                   

                    dan_ahrens
                    dan_ahrens
                    Expert ⭐️⭐️⭐️
                    December 17, 2019

                    HI @scott_sigsci - at your last step you are aggregating by company ID name, which instead of pulling in the email address and other relevant data of your contacts, is showing the count and sum of those values. You’ll want to remove that aggregation.

                    VP of Customer Success at Forcepoint
                      scott_sigsci
                      scott_sigsci
                      Helper ⭐️
                      December 21, 2019

                      So close... I’ll remove that and have another go. Thank you @dan_ahrens

                      Show more replies
                      Terms & ConditionsAccessibility statement