Qovery - 403 received Nginx customer side – Incident details

Third Party: Scaleway → Elements - Products → Object Storage experiencing degraded performance

403 received Nginx customer side

Resolved
Partial outage
Started about 1 month agoLasted about 3 hours

Affected

Qovery

Partial outage from 2:00 PM to 4:40 PM

Customer infratructure

Partial outage from 2:00 PM to 4:40 PM

Updates
  • Resolved
    Update

    Following a global cluster update triggered by our team, some configuration of the Nginx Ingress controller was wrongly modified on a subset of these clusters.

    Our team identified the root cause of the problem and deployed a fix on all clusters.

    The websites hosted on impacted clusters were unreachable during the outage.

    You can find more information on this forum thread

    We apologize for any inconvenience caused.

  • Resolved
    Resolved

    Our team has identified the root cause of this issue; however, further investigation is required, as the bug appears to have impacted configurations that should have remained unaffected. We are actively working to fully understand the scope and prevent any recurrence.

    We apologize for any inconvenience caused and appreciate your patience as we work to investigate this matter promptly.

  • Identified
    Identified

    We want to inform you of a recent incident involving customers' Nginx configuration that affected some customers. Specifically, a conflicting setting was released that impacted customers with IP whitelists configured on their services, resulting in their requests being denied with an HTTP 403 error.

    Investigation is in progress