Skip to main content
mahesh_patil
Gainsight Employee ⭐️⭐️
October 21, 2016

[INCIDENT] Elevated API Errors in MDA (Backed Services)

  • October 21, 2016
  • 9 replies
  • 121 views
SUMMARY

We're experiencing an elevated level of API errors. Initial analysis confirms DNS related issues with our Upstream service provider and they are investigating the same. 

FOR MORE INFORMATION

Please subscribe to this article for regular updates until the issue is resolved. If you have questions about this issue, please open a ticket with us by sending a note to support@gainsight.com.

UPDATE

Friday, October 21 9:20AM CST: Our service provider has resolved the DNS issue. During the outage MDA services including Rules, Outreaches and Data Processor were paused. We've restarted all services and are replaying any jobs that failed during the service incident. 

    9 replies

    mahesh_patil
    Gainsight Employee ⭐️⭐️
    October 21, 2016
    Our Upstream providers have identified the root cause of the issue causing DNS Resolution errors and are currently working to resolve.
    emily_mcdaniel_629a86
    Gainsight Employee ⭐️⭐️
    October 21, 2016
    Our service provider has resolved the DNS issue. During the outage MDA services including Rules, Outreaches and Data Processor were paused. We've restarted all services and are replaying any jobs that failed during the service incident. 
    arnold_price
    Contributor ⭐️⭐️
    October 21, 2016
    A customer rec'd a denial of service error message while attempting to complete a NPS survey yesterday at 2:20pm is this related to the API/MDA DNS issue?
    emily_mcdaniel_629a86
    Gainsight Employee ⭐️⭐️
    October 21, 2016
    Arnold - I am not sure if this is related. I know you're working with our support team to investigate and they'll be able to help. 
    mandy_major
    Gainsight Employee ⭐️⭐️
    October 21, 2016
    UPDATE

    We are experiencing this issue once again, and seeing errors "Unexpected end of JSON error" in Data Management, Copilot, Connectors tabs.  Will update this thread with the latest information as it comes in.
    darkknight
    Expert ⭐️
    October 21, 2016
    Thanks Mandy...that explains why I was experiencing this:  https://community.gainsight.com/gainsight/topics/timeouts-and-delays-in-gainsight-application

    VERY unnerving!
    Jeff Kirkpatrick
    kristin_f3c469
    Contributor ⭐️⭐️⭐️⭐️⭐️
    October 21, 2016
    Might have something to do with this
    http://www.wsj.com/articles/denial-of-service-web-attack-affects-amazon-twitter-others-1477056080
    Cyberattack Knocks Out Access to Websites
    darkknight
    Expert ⭐️
    October 21, 2016
    Thanks for the info Kristin!
    Jeff Kirkpatrick
    mahesh_patil
    Gainsight Employee ⭐️⭐️
    October 22, 2016
    Resolved - All services and queues are processing normally. 

    Please reach out to support@gainsight.com if you are still experiencing any issues.