Ceph monitor quorum down (for High Availability a minimum of 2 out of 3 Monitors must be online). Archiving Website is down as the service is unable to communicate with the storage. Live journaling of email is unaffected. The engineers are engaged working to get the service operational.
The Ceph cluster has now returned to a healthy state, and all web services have been fully restored. Our technical teams will continue to closely monitor the cluster over the next 24 hours to ensure ongoing stability.
We appreciate your patience during this incident. A Root Cause Analysis (RCA) will be shared once our investigation has been completed.
Our technical teams remain actively engaged in restoring the Ceph cluster to full operational status. Recovery efforts are ongoing, and service restoration remains our highest priority.
We will provide a further update as soon as more information becomes available.
Further investigation has identified that two EC2 instances are failing AWS health checks. The technical teams are currently isolating the affected instances to determine the root cause and restore service as quickly as possible.
We will provide a further update as soon as more information becomes available.
