Skip to main content

Our team is reporting a worse experience with Copilot after the full release than during the beta. I’m also experiencing issues using Copilot. 

Here is an example:

You

what are the verified outcomes for media companies similar to tcompany ABC]? provide examples of customers in the same industry as acompany ABC] and outline what they have achieved using amplitude

Copilot

I can't provide specific examples or outcomes for media companies like kcompany ABC] using Amplitude, as my focus is on assisting with customer success-related queries within the Gainsight platform. If you have questions about how Gainsight can support your customer success efforts, feel free to ask!

 

Users have reported to me that a prompt they used months ago while in beta, that worked great at the time, is now no longer behaving the same as it did, and the responses are worse. For example, the team asked for use cases by industry, and copilot would respond with meaningful answers. Now, when they ask the same prompt, it’s only returning use cases to a specific company.

  1. What would cause me to get this prompt back from Copilot?
  2. It states it is for “...questions about how Gainsight can support your customer success efforts.” Am I misinterpreting that?
  3. Has the Copilot LLM changed since beta?

Hey ​@sarahmiracle 

Thanks for sharing. Let me get someone from the team to look at this.


FYI 
@Molly.McQ ​@vineshgvk 


@sarahmiracle Thanks for the feedback! We did move some parts to Gemini for faster responses, but we did test it thoroughly to ensure that the performance does not degrade.

We’ll look into this feedback and get back to you. Can you possibly share a few more such examples, so we can investigate these things in detail?

 

Regards,

Shantan


Reply