[Important Notice] Storage Disruption on Tasmania-02 instances
G
Geli Kourakis
started a topic
3 months ago
Important Notice: Storage Disruption on Tasmania-02 instances.
We have identified an issue that has occurred from yesterday to this morning with the storage system that supports Nectar’s Tasmania-02 availability zone.
What Happened:
During the period between Thursday 13th June 3:28 PM to Friday 14th June 10:59 AM some reads and writes to Tasmania-02 storage were blocked. This issue may have led to data corruption on the root disk and storage volumes of instances that were running during this time period. Our team has identified and resolved the cause of the blocked reads/writes and are working to assist users that have been affected.
Please note that this does not have any relation to the NFS filesystem service provided for RDSI.
Impact:
Potential Data Corruption: Some of your data stored on the affected instances may have been corrupted by blocked reads/writes.
Service Disruption: There may have been disruption in service or degraded performance on the affected instances depending on their function.
Our Actions:
Immediate Mitigation: Our team has identified the affected storage systems and resolved the underlying issue.
System Audit: We are contacting the owners of potentially affected instances for the need to check data integrity.
Actions for those with running instances in Tasmania-02:
Data Backup and Disk Check: We strongly recommend that you immediately snapshot your instance and run a disk check on your boot drive and any locally mounted volumes. Note this is not required for RDSI mounts. If you are unfamiliar or need advice with this, please contact us with what operating system version you are running and what the type of file systems you are using (if known).
We understand the inconvenience this may cause and assure you that we are taking all necessary measures to minimise service disruptions. Your data security and service reliability are our top priorities.
We apologise for any inconvenience this may have caused and appreciate your understanding and patience as we work through this situation.
Geli Kourakis
Important Notice: Storage Disruption on Tasmania-02 instances.
We have identified an issue that has occurred from yesterday to this morning with the storage system that supports Nectar’s Tasmania-02 availability zone.
What Happened:
During the period between Thursday 13th June 3:28 PM to Friday 14th June 10:59 AM some reads and writes to Tasmania-02 storage were blocked. This issue may have led to data corruption on the root disk and storage volumes of instances that were running during this time period. Our team has identified and resolved the cause of the blocked reads/writes and are working to assist users that have been affected.
Please note that this does not have any relation to the NFS filesystem service provided for RDSI.
Impact:
Our Actions:
Actions for those with running instances in Tasmania-02:
Next Steps:
We understand the inconvenience this may cause and assure you that we are taking all necessary measures to minimise service disruptions. Your data security and service reliability are our top priorities.
We apologise for any inconvenience this may have caused and appreciate your understanding and patience as we work through this situation.
Regards,
Digital Research team.