Skip to main content
ben
ben
Helper ⭐️⭐️⭐️
February 1, 2018
Question

Response Rate calculations for surveys

  • February 1, 2018
  • 6 replies
  • 299 views
I was asked to provide response rate data to our board earlier this month. During this process, I started to notice some inconstancies in how the "Response Rate" is calculated under the Surveys section. I think I identified the situation and here is my example:

Participants: 1000
# of Survey Emails: 2 (initial and follow up)
Maximum # of Emails: 2000

Say that 100 people reply to the survey and all of those wait to reply until they receive the follow-up email. This would me we have 2000 emails sent and 100 survey responses. I would expect that the response rate for this survey is 10% (100 submissions divided by 1000 invited participants).

HOWEVER, I am seeing the response rate in a scenario like this be 5% (100 responses divided by 2000 emails sent). In my opinion, the denominator should be the # of participants, not the number of emails.

Am I missing something?
Do others see this as well?

Thanks!

    6 replies

    tzundelgainsightcom
    tzundelgainsightcom
    Helper ⭐️⭐️
    February 2, 2018
    Your are absolutely correct, Ben. That is how it should work. I'm requesting that Product Management weigh in on plans ot address this. 
    ben
    benAuthor
    Helper ⭐️⭐️⭐️
    February 2, 2018
    Thanks Tracy! I found a way around it with some reports and Excel magic, but not very handy. Our VP wants quick access to Response Rates for our board, so keeping this manageable for me is important.
    -Ben
      abhishek_sivaraman
      abhishek_sivaraman
      Helper ⭐️⭐️⭐️
      February 7, 2018
      Hi Ben,

      Can you elaborate on where you see this discrepancy?



      I have attached this screenshot from Advanced Outreach analytics for a survey and as you can see the survey response rate is calculated based on the number of participants and not the email sent. In this case, 20 participants responded compared to the 108 participants who were sent the email which comes out to 19%.

      Thanks
      Abhishek S
      ben
      benAuthor
      Helper ⭐️⭐️⭐️
      February 7, 2018
      Hey Abhishek,

      Thanks for the note! That actually makes a lot more sense when looking at the Advanced Outreach side of things. Based on a review of your notes/screenshot and our system, I think the AO section is correct and easily reference able.

      I had been looking at the "Analyze" tab inside the Survey area. I still feel that this may be broken. The top horizontal column is titled "Sent". The next one is "Responded". There are no issues for a survey with one email in the chain. However, if a chain has the potential for multiple "Sent" emails to a single participant, you cannot trust the current Response Rate calculated here. Most of our survey workflows have an initial email and a delayed follow up if they have not completed the survey. This dilutes the pool and artificially lowers Response Rate calculated under Surveys >> your survey in question >> Analyze.

      I am very happy to see that, for the time being, I can reference the AO section. I would still encourage more conversation around this because, for our org, we don't always have a 1:1 ration of Survey : AO. Sometimes we have used the same survey in multiple AOs. This would make response rate slightly more difficult. It would be nice if we could get the data from either the AO or the Survey itself (and have it be the same).

      Thank you!
      -Ben
      cmultanen
      cmultanen
      Helper ⭐️
      August 27, 2018
      +1 to this idea, it would be really helpful if we could pull response rate data and aggregate across multiple Programs. 
      ben
      benAuthor
      Helper ⭐️⭐️⭐️
      August 27, 2018
      So I have developed some ways to do this via reports and rules. It's hacky, but doable.
      -Ben
      Terms & ConditionsAccessibility statement