@jason_metzler  Definitely will be an interesting thread.  So, our health score has 3 different components: Customer Sentiment, Product and Experience.  For the product portion, we have several different products, so we compare the benchmarks for performance and usage for each of those products separately and combine into one score for product.  Then, for experience, we look at things like cases and voice of the customer survey results.  Then, for Customer Sentiment, that’s the value the CSM inputs, and we’ve actually given them some guidelines as to what should potentially be red versus yellow or green.  Product is weighted the most heavily, and then, Experience and lastly Customer Sentiment.  We definitely try to use ours more as a guide to whether or not the customer will renew.  Happy to email you the breakdown in more detail as it’s pretty complicated.  :)  We haven’t really ever held the CSMs accountable to improving the health score, but we do hold them accountable for addressing risk alerts.  Our health score is only calculated monthly so the risk alerts are meant to be more proactive than waiting for a health score update.

Hi @jason_metzler - you bring up a super important topic as it relates to health scoring and one that is often glossed over.

As you noted, if a health score does not accurately predict the likelihood of continued “healthy” business between a customer and your company, then the health score is meaningless. A good way to improve the accuracy of your health score is to do a historical analysis of your accounts that churned, as well as accounts that performed above average. You want to identify common traits that are tied to poor health and great health.

For example, many companies have a dimension of their health score that looks at the number of support cases currently open. Which is a fine metric to track and elevate visibility around, but what if your historical analysis shows no correlation between high or low support case volume and likelihood to renew or expand or churn? It might still make sense to track that metric for overall visibility, but you may decide to weigh it at a very low percentage, if at all.

While a historical analysis is best, you may find it easier just to do an analysis on customer renewal outcomes in the next quarter and for each customer that churned, renewed, upsold, etc - analyze if the health score was predictive of that outcome, or if it gave a false predictor. If it did not align with the outcome, you should look at the business reasons why that customer churned, renewed, upsold, etc and make sure they are accurately captured in your scoring rubric. They might also be in that rubic, only weighted incorrectly. 

This where you’d want to play around with Google Sheets or Excel a bit and tweak your health score metrics and percentages using the actual data from the customers that had a renewal in that time period that you’re analyzing and adjust the health score formula with actual customer input to see if your revisions will capture the true health. 

If you have a data science team, they can accelerate this process, as doing a correlation analysis is a great start. 

Happy to chat further on this topic. It’s something that’s a bit of a personal passion for me, having built health scores from scratch before and realizing that it’s not a “set it and forget it” but something you have to continually validate and adjust. 

Yeah, @dan_ahrens we built out a version of health scores a few years back when DEAR was new and haven’t iterated much from there. I personally see risk as a separate issue from health, and why I haven’t tried to blend the two together. Plus it would really torpedo my analogy of the hang-gliding CEO.

We are gearing up to develop a new scoring methodology here, would definitely like to hear your thoughts, and see if there’s any latest and greatest out there we can piggy back off of.

Hi @jason_metzler what might be clouding things here is if you view health alone as the predictor of renewal likelihood. 

Take your hang gliding CEO example. If he wanted to apply for life insurance and was candid with both his medical history as well as his adventurous hobbies, his life insurance premium would be higher than another similar healthy person who had a stamp collecting hobby instead. ;)

So in this example, you are the life insurance actuary who has to assign an overall death (churn) risk to your customer which is a weighted combination of physical health and risk behaviors. You can’t properly price the policy premium (renewal risk score) on either measure without considering the other. 

Hope this helps!