Deploying a Resilient Usage Engine Private Edition Application on AWS with Disaster Recovery

Deploying the Usage Engine Private Edition application on public cloud providers requires attention to file systems and databases, in order to ensure high availability and disaster recovery.

In this blog post, we will explore how to deploy on AWS using Amazon Elastic Kubernetes Service (EKS) with a focus on persistent storage and database solutions for disaster recovery. We will touch on setting up persistent volumes that span multiple availability zones, selecting the right file system, and implementing strategies to ensure that your data remains available and secure even in the face of a disaster.

We’ll be looking at disasters on two levels, and strategies for recovering from them:

• Loss of one AWS Availability Zone, and how being resilient to this is part of normal high availability setup of Usage Engine Private Edition.
• Loss of an entire AWS Region. Something that could be caused by a catastrophic failure of the underlying infrastructure, but also by malicious incursions into your AWS account by malicious actors and ransomware style attacks.

Ensuring High Availability and Disaster Recovery

A disaster recovery strategy encompasses both persistent file storage and database management. Amazon Elastic File System (EFS) and Amazon Relational Database Service (RDS) provide complementary solutions to ensure data integrity, security, and availability.

Persistent File Storage with EFS

Amazon EFS provides a scalable and managed NFS file system that integrates seamlessly with Amazon EKS for persistent storage. For an active-passive deployment strategy, EFS Access Points and EFS replication offer advanced features to simplify management and ensure faster recovery times.

When it comes to resiliency in the case of an AZ failure, EFS already supports multi-AZ support with automatic failover with minimal or no down time. However, in order to recover from a region failure complementary strategies are required.

For region disaster recovery, file systems need to be backed up to a fallback region, so that the filesystem can be restored in order to do disaster recovery. One feature of EFS that simplifies this greatly is that of EFS Access Points. With Access Points, applications can consistently mount specific directories within the EFS file system, regardless of the underlying file system structure. This should simplify the restoration process in a fallback region, as the EKS persistent volumes can be created without reference to the old EFS file system. With the EFS replication strategy in place, applications can quickly re-mount the EFS access points and have access to the latest data almost immediately.

Database Management with Amazon RDS

Although Amazon RDS for postgresql is not automatically resilient to AZ-failures, it is possible to set up RDS with a standby instance in an alternate AZ. In case of AZ failure, RDS will automatically fail over to this standby instance.

Similar to EFS, the key to region disaster recovery is to perform backups to a fallback region where a new database can be created from the backups in case of disaster. Care must be taken in scheduling the Database backups, because they can have an impact on IO, which can conflict with normal workloads.

AWS Backup for Interval and Cross-Region Backups

AWS Backup simplifies and automates the process of backing up data across AWS services, including Amazon EFS and RDS. By scheduling regular interval backups, you can ensure that your data is consistently protected and can be restored quickly in case of data loss or corruption. Additionally, AWS Backup supports cross-region backups, enabling you to replicate your data to a different AWS region.

AWS Backup also lets you set your backup settings on an organization level, that is one step above accounts, which can simplify or unify governance of your AWS environment, not only Usage Engine Private Edition.

Immutable Backups for Ransomware Protection

To protect against ransomware attacks, enable immutable backups, a feature of AWS Backup. Immutable backups cannot be altered or deleted, ensuring the integrity of your data.

Recovery Scenarios

AZ Failure Recovery: In the event of an AZ failure, EFS should continue operating and AWS RDS automatically promotes the standby instance in another AZ to primary. This process is managed by AWS, minimizing downtime.

Region Failure Recovery: For regional failures, restore your PostgreSQL database or EFS from a cross-region backup. This involves creating a new RDS instance or EFS file system in the fallback region and restoring data from the cross-region backup.

Ransomware Attack Recovery: In the event of a ransomware attack, use the immutable backups to restore your PostgreSQL database or EFS to a state before the attack. This ensures data integrity and minimizes data loss.

Conclusion

Implementing these strategies for EFS and RDS should ensure that your Usage Engine Private Edition on AWS remains resilient and secure, capable of recovering from both AZ and regional failures as well as ransomware attacks.