Case Study on Primary Storage
About the company
Deshsanchar.com is one of the most popular news portals from Nepal and has made a landmark achievement in reaching the Nepalese community the world over in a short span of time. They are the multimedia venture led by senior journalist Mr. Yubaraj Ghimire, that has a team of dedicated journalists for updating political, economic, social and entertainment-related news and events. It is particularly viewed within Nepal and in Nepalese-populated community in different 140 countries. with a team of young, energetic and talented journalists.
Deshsanchar is running their workloads and application in Datahub Nepal. Currently, they have been using Dedicated server with 8 core Vcpu, 32Gb Ram, 500Gb SSD storage and data transfer of 15 TB. With the exponential increase of the data day by day, using on-premise infrastructure could result in high investment for handling the data as well as for maintenance of the infrastructure. Being one of Nepal’s biggest news portals, the website is guaranteed to get a high amount of hits and high daily log data.
As storage size is going bigger, it is difficult for them to manage their storage infrastructure. In addition to that, the cost of running and maintaining the data center is increasing. They face problems in scaling their infrastructure as well. They also face the unavailability of data under the circumstances of system failures. Currently they are using the traditions Hard Disk drives for their primary storage.
As an advanced consulting partner of AWS, Genese Software Solution proposed a solution to facilitate their overall infrastructure and primary storage of their system in the AWS cloud.
We proposed the following options for primary storage in the AWS cloud.
- Use of Amazon Simple Storage Service (S3) for storing static files and documents with proper security implementation to store data at rest as well as in transit.
- We proposed the options to configure fine-grained control of access of S3 resources using Bucket and object level policies along with IAM policies.
- Use of S3 storage class applying Lifecycle management policies for proper management of objects in S3 for cost optimisation.
- Proposed the use of versioning in S3 buckets to keep multiple versions of objects when modified or deleted.
- Use of Amazon EBS backed instances for durable block-level storage.
- Suggested the use of SSD for better performance and efficiency.
- For high availability, we suggested taking frequent snapshots.
- Suggested the use of managed Relational Database Service (RDS) for databases.
- The regular snapshots of the RDS for data backup and recovery purposes.
- Monitoring of the performance of the database instances using cloudwatch metrics.
- Multi-AZ setup for high availability and suggested use of Read replicas when read
The main focus with Deshsanchar was to move the primary on premise environment to AWS along with primary storage. For this, first a custom isolated VPC network in multiple availability zones and multiple subnets was created. This setup maintain the availability of their AWS environment in case of any outages, failures or natural disaster.
The application servers deployed in multiple Availability Zones reside behind an Application Load balancer. The servers communicate with AWS managed database that resides in the private subnet. Also for caching we have provisioned the Redis cluster. All the static files are stored in S3 which is served using Cloudfront. The web servers are configured as EBS backed instances and use EBS volume as the permanent storage.
The system uses multipart upload to S3 for larger files. The destination buckets have been encrypted using S3 managed keys. The bucket’s access has been controlled using bucket policy as well as from the IAM. We have implemented object based tagging and for proper selection of storage class in S3. S3 Lifecycle Management policies also transition objects from one storage tier to another. Versioning has been enabled to prevent accidental deletion of objects.
For volumes in web servers we have selected high performing SSD EBS volumes encrypted using AES-256. Snapshots are taken regularly using a lambda function that is scheduled to trigger once a day. The function takes snapshots of the EBS volumes and also cleans up any snapshots that are older than 30 days.
The POC was successfully concluded within one month, after the environment was completely implemented in AWS.
As an advanced consulting partner of AWS, Genese Software Solution proposed a solution to facilitate Disaster Recovery environment on AWS cloud for disaster scenarios with minimal disruption and downtime.
For this, we proposed different options for disaster recovery available in AWS.