@jriva I’ve used milestones to do this.  So, for example, if you have a risk alert because usage falls below a certain threshold, create a rule to check for a completed CTA, and then, an increase in usage over whatever period of time you might attribute to that alert, and then, create a milestone.  I then built a report to show which risks were considered mitigated via the milestone and reported on the percentage of CTAs that were resolved positively.  The report took a little time, but was definitely worth it to evaluate the effectiveness of not only the CTA, but also the playbook used to mitigate.  

We also ran into a lot of instances where our CSMs thought that a customer should not have received a CTA, so we used a status “Not Valid”, and the CSM was required to enter a reason why they considered it not valid.  This also gave us the opportunity to evaluate the effectiveness of alerts, whether or not there was a training need for the reason behind the alert, etc.

Hope those help!

Thanks @heather_hansen!  We haven’t started using Milestones yet, but this is definitely something I want to investigate further. 

What about lifecycle CTAs like EBRs? How do you know if the cadence, due dates, playbooks, etc. are optimized? 

@jriva For the more event based ones, we did a couple of things:  1. Look at days to close versus due date to continue evaluate whether we were giving the appropriate amount of time and as a KPI. 

To elaborate a little more on the first one, we wanted the key parts of the customer journey to be accomplished at set times, so we were looking more for non-compliance.  We created a field to show when the next EBR was due so that helped with planning for the CSMs and compliance for the leadership team.

For Success Plans, we were looking for updates, so we created a dashboard to show how many updates had been made as well as when the last one was.  

2.  We started sending out post-EBR surveys to ask the customer to rate the CSM on how they performed the EBR.  So, the goal of the EBR would be to reinforce the value that the customer is receiving towards their initial success metric, so we asked a question around how well the CSM did a gearing the EBR towards what they cared about.

Depending on how many Playbooks you have, this might be an easy task or not. It also really depends on what types of playbooks you have and you want to assess. What I would recommend is to first organize your playbooks into the different categories: opportunities, risks, lifecycle. 

Lifecycle Playbooks

Lifecycle playbooks are things that occur during a customer’s lifecycle at a particular point in time. For example, post-sales, they might start off with an onboarding or kickoff playbook. And post-onboarding, there might be a QBR playbook that fires every 3 months.

What I have typically seen customers do is assess wether these playbooks are firing at the right time and whether or not they are being completed. To evaluate these types of playbooks, I would recommend 3 reports:  

1. All customers and their start date
2. All completed or incomplete lifecycle playbooks
3. All Timeline Activities

What you will want to analyze is the timeline of activities across your key accounts. For example, if you expect to have a BR with high touch customers every quarter, then there should be an associated CTA. I have used Excel formulas and pivot tables in the past to do this analysis. 

Risk Playbooks

These types of playbooks, as Heather mentioned, are for issues like loss of a champion, reduction in usage, no logins in a number of days, customer was just purchased by another company, etc. These are for risks that will impact adoption, retention, and potentially expansions. 

As Heather mentioned above, you will need to architect a system to track the CTA, the CSM’s action taken, and the expected impact on the metrics. For usage, this a bit simpler. You can compare what usage looked like before and after the CSM action was taken to assess whether it was effective.

For items like loss of a sponsor, you will likely need to get a bit more crafty. I typically compare usage and you can compare renewal and expansions over time. If there was no change in usage and no impact to renewals or expansions opps, than the playbook was likely successful. 

Opportunity Playbooks

Opportunity playbooks are often used to track expansions (CSQLs) and adoption promotion activities. For example, your customer might have a new division of employees who want to use the software. There may not be an upsell because they don’t need any new licenses, but nonetheless, this is an opportunity that you will want to track in your Cockpit.

To assess how effective this playbooks are, you will likely need to follow each process and compare data along the way. For example, if you want to assess CSQLs playbooks, you can pull reports of all the CTAs and associated expansion opportunities to see how many converted, the time it took to convert them, and how effective these campaigns were. For this analysis, it is very similar to analyzing a sales pipeline and understanding the time it takes to complete an sale and your conversion rates at each stage. 

Hope this helps provide a bit of a framework to get started on this initiative. Its a bit of work to get started but once you have a framework up and running, it will be much easier to analyze this in the future. 

Wow, thank you @heather_hansen and @jean.nairon this is really great! A lot to unpack but great directions for me to explore. 

@jean.nairon and @heather_hansen  Thanks for your inputs here!

You’re most welcome @jriva and @sai_ram!