Between August 23, 2022 17:54UTC and August 24, 2022 03:33UTC, our customers experienced DNS resolution failures in all regions of Private Spaces and Common Runtime. We sincerely apologize for the negative effects our customers experienced.
Applications in Private Spaces a…
Resolved
This incident has been resolved.
Monitoring
Our engineers believe that impact has been mitigated, and we are continuously monitoring and verifying connectivity.
Issue
Customers are experiencing DNS resolution failures for custom domains and the default domain (*.herokuapp.com) in Common Runtime and Private Spaces across the US and EU regions. Our upstream vendor have applied a fix and our engineers are verifying connectivity.
Investigating
Upon further investigation, our engineers found the issue started at 17:54 UTC Aug 23rd. Our engineers are in communication with our upstream DNS provider and are currently working to fix the issue. We will be providing updates on a 1 hour cadence.
Investigating
Our Engineers are continuing to investigate this issue.
We will be providing updates on a 1 hour cadence.
Investigating
Our Engineers are continuing to investigate this issue.
Investigating
Engineers identified and are investigating an issue with an upstream DNS provider starting around 19:33 UTC Aug 23rd, leading to increased failure of insert/delete/changes to DNS.
Customers may have seen delays in any DNS actions. This might include creating/deleting apps, or adding/removing custom domains.