ScootKit - Connection to Bot-Host #17 unstable – Incident details

Experiencing partially degraded performance

Connection to Bot-Host #17 unstable

Partial outage
Started about 2 months agoLasted 42 minutes


SCNX Bot-Hosts

Operational from 5:12 PM to 5:27 PM, Partial outage from 5:44 PM to 5:54 PM

Bot-Host #17

Operational from 5:12 PM to 5:27 PM, Partial outage from 5:44 PM to 5:54 PM

  • Postmortem

    A small typo in a DNS record created due to the manual setup steps of another bot-host made this server unreachable for a prolonged period of time. Our processes after the initial report of the downtime focused on identifying the issue on our hosting infrastructure, instead of reviewing recently taken actions which could have highlighted configuration changes in our DNS configuration. The processes have been made optimized for this case and call for a proper review of recent changes to infrastructure in addition to the already existing issue identification steps.

  • Resolved

    This issue has been resolved, Bot-Host #17 is reachable again. Please note that hosted bots were not affected.

    This incident was caused by a misconfiguration of DNS records, made in error when setting up infrastructure for another Bot-Host. We apologize for the inconvenience caused.

    As no bots were offline, this incident is ineligible for our voluntary refund policy.

  • Monitoring

    We've rolled out a fix for this issue and are monitoring results. Affected customers should be able to access their bots again. Hosted bots were not affected in this incident.

  • Identified

    We've identified a potential misconfiguration that might cause this issue. We're rolling out a fix.

  • Investigating

    We've identified that connectivity issues are limited to Bot-Host #17 - Bots hosted on Bot-Host #17 are not affected.

    We're continuing to investigate the root cause of this issue.

  • Investigating

    We've received reports of an increased number of connections to Bot-Hosts failing. We're investigating.