Deshsanchar CloudFront

Deshsanchar.com Case Study on AWS

About the company

Deshsanchar.com is one of the most prominent and popular news portals of Nepal. It has made a landmark achievement in reaching the Nepalese community all over the world, in a short period of time. This multimedia venture is led by senior journalist Mr. Yubaraj Ghimire, it has a team of dedicated journalists for updating content on political, economic, social and entertainment-related news and events. The contents are particularly viewed within Nepal and also in the Nepalese-populated community in 140 different countries. The Deshsanchar’s team includes young, energetic and talented journalists.

Problem Statement

The Deshsanchar were running their workloads and application in Datahub Nepal. Currently, they have been using a Dedicated server with 8 core Vcpu, 32Gb Ram, 500Gb SSD storage and data transfer of 30 TB. With the exponential increase of the data day by day, using on-premise infrastructure resulted in high investment for handling the data as well as for maintenance of the infrastructure.

Being one of Nepal’s largest news portals, its website was guaranteed to get a large number of visitors from the users, both from Nepal and abroad. Furthermore, the Deshsanchar users outside of Nepal were complaining about the response time of the website. Deshsanchar had the biggest fear of losing their customers because of the high latency and response.

Our Proposal

As an advanced consulting partner of AWS, The Genese Solution proposed a solution to facilitate their overall infrastructure and primary storage of their system in the AWS cloud.

The following options were proposed for primary storage in the AWS cloud.

• Amazon S3:
  • Use of Amazon Simple Storage Service (S3) for storing static files and documents with proper security implementation to store data at rest as well as in transit.
  • We proposed the options to configure fine-grained control of access of S3 resources using Bucket and object-level policies along with IAM policies.
  • We proposed the options to configure fine-grained control of access of S3 resources using Bucket and object-level policies along with IAM policies.
  • We proposed the options to configure fine-grained control of access of S3 resources using Bucket and object-level policies along with IAM policies.
  • Use of S3 storage class applying Lifecycle management policies for proper management of objects in S3 for cost optimization.
    Proposed the use of versioning in S3 buckets to keep multiple versions of objects when modified or deleted.
• Amazon EBS:
  • Use of Amazon EBS backed instances for durable block-level storage.
  • Suggested the use of SSD for better performance and efficiency.
  • For high availability, we suggested taking frequent snapshots.
• Amazon RDS:
  • Suggested the use of managed Relational Database Service (RDS) for databases.
  • The regular snapshots of the RDS for data backup and recovery purposes.
  • Monitoring of the performance of the database instances using Cloudwatch metrics.
  • Multi-AZ setup for high availability and suggested use of Reading replicas when read.
• Amazon CloudFront
  • Suggested to use the Cloudfront for the fast and secure delivery of data to their global customers.
  • Suggested the Cloudfront for handling the response as demand increases or decreases.

Solution

The main focus of Deshsanchar was to move the primary on-premise environment to AWS along with the primary storage. For this particular problem, first, a custom isolated VPC network in multiple availability zones and multiple subnets was created. This setup maintains the availability of their AWS environment in case of any outages, failures or natural disasters.

The application servers deployed in multiple Availability Zones reside behind an Application Load balancer. The servers communicate with AWS managed database that resides in the private subnet. In addition to caching, the Redis cluster was provisioned. All the static files were stored in S3 which is served using Cloudfront. The web servers were configured as EBS backed instances and EBS volume was used as the permanent storage.

There was a Cloudfront set up to take the user request and response from the server within a seconds from the nearest Cloudfront POP in terms of latency which then routes the user to the Edge location. The advantage of using the Cloudfront was that it could be easily carried even at peak times due to its capacity to take many requests in a second.

The system uses multipart upload to S3 for larger files. The destination buckets have been encrypted using S3 managed keys. The bucket’s access has been controlled using bucket policy as well as from the IAM. It has been implemented for object-based tagging and also for the proper selection of storage class in S3. The S3 Lifecycle Management policies also transition objects from one storage tier to another. The versioning has been enabled to prevent accidental deletion of objects.

For volumes in web servers, high performing SSD EBS volumes encrypted using AES-256 was selected. Snapshots were taken regularly using a lambda function that was scheduled to trigger once a day. The function takes snapshots of the EBS volumes and also cleans up any snapshots that were older than 30 days.

The application servers deployed in multiple Availability Zones reside behind an Application Load balancer. The load balancer’s domain has been pointed in the Cloudfront origin which takes every request comes to it. We have implemented Signed-Urls in the Cloudfront which gives the dashboard access to the editors of Deshsanchar via a pre-signed URL. After authentication, the images inside the S3 were kept private images, in order to load those images pre-signed URL was required.

The POC was successfully concluded within one month after the environment was completely implemented in AWS.

Results

The environment was deployed to AWS and during the POC period, there were no performance disruptions at any stage. After the deployment, the client was pleased with the heavy reduction in costs and server management. Deshsanchar have total of 37.6752 GB data transfer with 1.9257 M number of requests for the month of August.
Deshsanchar has been saving more than $5000 monthly in comparison to On-premise. The Genese Solution did the performance testing as per the client’s requirement of the Cloudfront and the following result was achieved and could be observed in the table below.

No of users	No of Request	Minimum response	Max response	Number of Failed request	Error	Total TimeTaken	Received kb/s	Sent Kb/s
500	0	702	73546	0	0.00%	2:45	1653.913	9.75
1000	4,000	719	1140685	10	0.26%	19:03	370.997	2.19
1500	12,000	506	671702	57	0.48%	12:36	605.86	3.48
2000	16000	738	644202	110	0.69%	15:35	592.36	2.75

During the load testing, the highest demand designated by the client was 16000 hit in the Cloudfront and the tolerable error rate was 1%, which we successfully achieved and the maximum error rate was 0.69%.