Skip to main content

UPDATE 6/9/22 3:15CT

 

A brief update to this thread as the investigation and resolution planning continues.

  1. Root Cause Analysis: Product, Engineering, Support and CS Operations leadership are leading this effort with syncs multiple times in every 24 hour period.  

  2. Impacted Users: We are finalizing a way to request and access the list of impacted users for your organization.  

  3. Communication To Date  

    • 6/7 11pm PT: Issue is identified and email queues were shut off to stop any more errant emails

    • 6/8 5am PT:  Notification posted in Global Gainsight Admins slack and Community

    • 6/8 12pm PT: Email Notification emailed to all users who were sent the errant email

    • 6/8 5pm PT: RCA Status Update email sent to all users who were sent the errant email

    • 6/9 1pm PT: This Status Update to Community Page

  1. Secondary Issue with Email Notification on 6/8 12pm PT

    • Unfortunately, due to a mistake in our process (that has now been fixed), the email introduced more confusion with different From Name (Dan Wiegert) and From Email Address (Kellie Capote email address).

    • This was a mistake during the process, and was not a downstream impact of the original issue.  But given the nature of the original issue, we know this was a disruptive notification to make sense of.

  1. Communication Going Forward:

    • Summary Status Updates to this Community Post

    • RCA and Resolution Plan details will be shared with key contacts at each account 

      1. CS Leadership (CCO/Success Leader)

      2. Operations Contact

      3. Gainsight Administrator(s)

    • We do not intend to email impacted users going forward and will centralize through the channels listed above

ORIGINAL POST BELOW

URGENT ISSUE:

On the early hours of June 8 CT, a process inadvertently kicked off and sent out an email with only an email address and a token. Please be advised this was done in error while our team was testing functionality.  We sincerely apologize for this incident and are working with our engineering team to determine the root cause for how this got sent out and also what steps need to be taken to prevent an issue like this from happening in the future.

Please reach out to Gainsight Support with any questions/concerns.  We will update this thread once we know more about root cause and preventative actions.

Would appreciate a list of individuals at my company that were sent this so i can appropriately communicate this is not spam. 

@elliot_hullverson are we impacted? If so, how many?

Thank you @mandy_major for sharing. All, let me get the right sort of people view this thread to help us along.

Our customers are still waiting for the root cause analysis -- when will that be available?

I’m a Gainsight Admin at my company, and I have received zero communication from Gainsight about this. I found out on the Global Slack community (which was nice to see it there but not a substitute) before this post was up. 

 

Where was the immediate in-app notification to admins? Email to Admins? Why are some admins getting updates and others are not? Mistakes happen, but the response to this has been more alarming than the issue itself to be honest.

 

Edit: deja vu from a year ago 

 

@bradley - More details to come, including how we will provide you the list of impacted users, but to respond to your specific question:

 

We have the full list of contacts that had an email sent via our email provider.  A subset of those emails were filtered out by:

  • Corporate-wide automated filters before getting to someone’s inbox
  • User-level Inbox SPAM filters  
  • IT teams seeing unusual behavior and isolating those emails 

I can confirm that your email address was not on the list of sent emails.  That is likely why you did not get a direct email via the first two messages we sent out.

Going forward - per Mandy’s update - we will be focusing updates only on our key contacts including Gainsight Admins.

Thanks for the reply on this @tyler_mcnally . I realize I maybe wasn’t super clear in my question - not really concerned with why I didn’t get the errant emails. More concerned with why I didn’t get an email notifying me of the issue or any subsequent email or alert to that effect, or even a copy of the email that went out to our impacted users (the ‘ignore previous email’ and any ‘update’ email, not the original errant one). Also I haven’t been at Iredacted] since February :)

Don’t forget the tertiary issue - the follow up email had support@gainsight.com as the reply-to address… so anyone that replied to the email automatically had a support ticket created, as a few of our users have done. 

UPDATE 6/27 200pm PT

 

Status

  • Our product, engineering and security teams have confirmed the full impact of the email event and the root causes have been addressed to ensure this type of situation does not occur again. 

Next Steps

  • Beginning today and over the next 48 hours, direct communication to our main contacts at each account are going out. 
  • Your CSM will be able to follow up with a list of contacts that were mistakenly emailed
  • The formal RCA is being finalized - we will post an update to this thread when it is ready  

We will continue to update this community thread with future status updates.

Closing Thread on one outstanding item: RCA is available upon request.  Your CSM can help facilitate.

 

thank you!

 

tyler

 

 

Reply


Terms & ConditionsCookie settings