ScootKit - Connection to Bot-Host #17 unstable – Incident details

All systems operational

Connection to Bot-Host #17 unstable

Resolved
Partial outage
Started 6 months agoLasted 42 minutes

Affected

SCNX Bot-Hosts

Operational from 5:12 PM to 5:27 PM, Partial outage from 5:27 PM to 5:54 PM

Bot-Host #17

Operational from 5:12 PM to 5:27 PM, Partial outage from 5:27 PM to 5:54 PM

Updates
  • Postmortem
    Postmortem

    A small typo in a DNS record created due to the manual setup steps of another bot-host made this server unreachable for a prolonged period of time. Our processes after the initial report of the downtime focused on identifying the issue on our hosting infrastructure, instead of reviewing recently taken actions which could have highlighted configuration changes in our DNS configuration. The processes have been made optimized for this case and call for a proper review of recent changes to infrastructure in addition to the already existing issue identification steps.

  • Resolved
    Resolved

    This issue has been resolved, Bot-Host #17 is reachable again. Please note that hosted bots were not affected.

    This incident was caused by a misconfiguration of DNS records, made in error when setting up infrastructure for another Bot-Host. We apologize for the inconvenience caused.

    As no bots were offline, this incident is ineligible for our voluntary refund policy.

  • Monitoring
    Monitoring

    We've rolled out a fix for this issue and are monitoring results. Affected customers should be able to access their bots again. Hosted bots were not affected in this incident.

  • Identified
    Identified

    We've identified a potential misconfiguration that might cause this issue. We're rolling out a fix.

  • Update
    Update

    We've identified that connectivity issues are limited to Bot-Host #17 - Bots hosted on Bot-Host #17 are not affected.

    We're continuing to investigate the root cause of this issue.

  • Investigating
    Investigating

    We've received reports of an increased number of connections to Bot-Hosts failing. We're investigating.