AWS cost optimization is an ongoing process. AWS cloud resource utilization needs to be continually monitored to determine when the resources are being under-utilized or not being utilized or idling to reduce the costs by deleting/terminating/freeing the unused resources.

It’s also helpful to consider the Saving Plan or Reserved Instances to ensure full utilization as per the anticipated constant level of consumption.

While the fundamental process of cost optimization on AWS remains the same – monitor AWS costs and usage, there are a number of tactical ways to analyze the operational data to find the opportunities for savings and take actions to realize the savings.

Pillars of Cloud Cost Optimization

1. Right-Sizing

Identify the resources with low-utilization and reduce the cost by stopping or rightsizing.

• Use AWS Cost Explorer Resource Optimization report to get the list of idle or low utilized resources. Reduce the costs by either stopping or downsizing the instances.
• Use AWS Compute Optimizer for downsizing recommendations within or across instance families, upsizing recommendations to remove performance bottlenecks and recommendations for EC2 instances that are parts of an Auto Scaling group.
• Identify Amazon RDS, Amazon Redshift instances with low utilization and reduce the cost by stopping (RDS) and pausing Redshift outside of business hours or non-processing timeframe.
• Use Amazon EC2 Spot Instances to reduce EC2 costs where possible – SPOT instance mgmt. can be effectively managed by 3rd part tool, such as Spotinst, for automatic termination and new instance availability without impacting end users.
• Review and modify EC2 Auto-Scaling Groups configuration to ensure scaling is happening on the right thresholds instead of setting it at the lower thresholds.
• Try using Elastic Kubernetes Services (EKS) and Elastic Compute services (ECS – EC2 Model) worker nodes under auto-scaling using SPOT instances (instead of on-demand/RI) for reducing the cost.
• Consider using Elastic Compute services (ECS -Fargate) to start the tasks with lower configuration (e.g. 0.5 vCPUs & 1GB RAM) per task and use auto-scaling instead of higher configuration per task.
• Multi-AZs configuration for DEV, test, UAT, or DR environment may not be necessary or useful for RDS, Redis, NAT, or other PAAS services (unless business really requires achieving any objective); therefore, it’s important to design wisely.

2. Instance Family Refresh

Each instance/series/family offers different varieties of computing, memory, and storage parameters. Instance types within their set groupings are often retired as a unit when the hardware required to keep them running is replaced by newer technology.

• Upgrade instances to the latest generation offering lower costs.
• For PAAS services, such as RDS, Redis, ElasticSearch, MSK, use instance type wisely, particularly for pre-PROD or DR and appropriate AZs to scale the instances. It may not require more than 2 AZs even while considering business SLAs. Keep an eye on the latest family of infrastructure and refresh to take the benefits of lower costing.
• Use low costing instance type for Development, QA, or the environment (e.g. T3a) wherein performance benchmarking certainly is not required for the business SLA.
• Always start with low and upgrade the right size to a suitable family considering the business use cases and traffic patterns.

3. Compute Savings Plans to reduce EC2, Fargate and Lambda costs

Compute Savings (Compute Savings Plans and EC2 Instance Savings Plans) Plans automatically apply to EC2 instance usage regardless of instance family, size, AZ, region, OS or tenancy, and apply to Fargate and Lambda usage. Use one year, no upfront Compute Savings Plans to get a discount of up to 54% compared to On-Demand pricing. Once you sign up for Savings Plans, your computer usage is automatically charged at the discounted Savings Plans prices. Any usage beyond your commitment will be charged at regular On-Demand rates. Please follow Point Nos. 1 and 2 before adopting Saving PLAN. Saving Plan has a lot of advantages over AWS Reserved Instances with one condition – “Usage commitment/Hour”

Reference link – https://aws.amazon.com/blogs/aws-cost-management/getting-started-with-aws-savings-plans/

4. Reserved Instances (RIs)

Even though the Saving plan supersedes the RI option, some of the PAAS services are outside of the Saving Plan program as of today in April 2020.

• Purchase reserved nodes for RDS, Redshift, Elasticsearch and ElastiCache Services to reduce the cost.
• Many instance type discounts are larger and at the top end may be over 60% in the case of some 3 year all upfront terms. Identify the instances & take call saving plan vs RI intelligently.
• You can get shorter term RIs on the marketplace.

5. Scheduling on/off times

It’s worth scheduling on/off for non-production instances used for development, staging, testing, and QA as it can save up to 65% of running these instances if you apply an “on” schedule of 8.00 a.m. to 8.00 p.m. from Monday to Friday. However, it’s possible to save a lot more — especially if development teams work in irregular patterns or irregular hours. Plan more aggressive schedules by analyzing utilization metrics to determine when the instances are most frequently used or apply an always-stopped schedule which can be interrupted when access to the instances is required.

6. Orphaned resources

Identifying waste takes time and accurate reporting. It is a great reason to invest time and energy in developing a proper tagging strategy to make this an easy process.

• For unutilized AWS EC2s, use Cost Explorer Compute Optimizer report or extract CW stats and take actions.
• Terminate VMs that were spun up for training or testing.
• Delete unattached EBS volumes – Check Volumes page and available EBS volumes status.
• Delete obsolete snapshots and lifecycle as required to meet the business demand.
• For idle load balancers, try to use ALB with path/content-based routing.
• Release unattached Elastic IP addresses.

7. Storage

• EBS volumes that have very low activity (less than 1 IOPS per day) over a period of 15 days indicate that they are probably not in use. Identify these volumes using the Trusted Advisor Underutilized Amazon EBS Volumes Check. To reduce costs, first, snapshot the volume (in case you need it later), and then delete these volumes.
• Use S3 Analytics to analyze storage access patterns on the object data set for 30 days or longer. It makes recommendations on where you can leverage S3 Infrequently Accessed (S3 IA) to reduce costs. You can automate the process of moving these objects into lower-cost storage tier using Life Cycle Policies. Alternately, you can also use S3 Intelligent-Tiering, which automatically analyzes and moves your objects to the appropriate storage tier.
• Move infrequently accessed data to lower-cost tiers.
• Use S3 One Zone if business SLA allows you to do so.

8. Containerization

Containers can help to get the most out of available computer hardware and software resources which are Lightweight, have start-up time in milliseconds, and require less memory space. Containers help to achieve the scale of the economy by reducing the IT management resources, snapshot size, spinning up applications, reduced & simplified security updates, etc. Containers are a better choice when your biggest priority is maximizing the number of applications running on a minimal number of servers.

9. Local Caching

If data transfer from EC2 to the public internet shows up as a significant cost, consider using Amazon CloudFront. Any image, video, or static web content can be cached at AWS edge locations worldwide, using the Amazon CloudFront Content Delivery Network (CDN). CloudFront eliminates the need to over-provision capacity in order to serve potential spikes in traffic. Use CloudFront when your userbase is geographically distributed.

10. VPC endpoints

Heavy data lifting to S3 from private subnets (e.g. static contents, backup, videos, etc.) require NAT gateway. Use VPC endpoint to reduce the NAT gateway data out the cost and move data securely over the AWS backbone. This will reduce the NAT Gateway data out the cost that occurs to S3.

11. Regional and AZs Cost

AWS AZ and regional cost can be eliminated wisely through right solutioning.

With-in availability zone (AZ)

• Data transfer costs for transferring data in the same region and within the same availability zone are zero, however with one requirement that you must be using a private IP address.
• If you are using a public or Elastic IPv4 address or IPv6 address, data transfer out from EC2 will be charged at 0.01/GB. In the same way, data transfer into AWS EC2 will be charged at 0.01/GB if you are using a public or Elastic IPv4 address or IPv6 address.

Across availability zones in the same region

• Data transfer between AWS services located in the same region but in different availability zones is considered as regional data transfer and is charged at $ 0.01/GB (outgoing data transfer).
• In the same way, data transfer into EC2 from an AWS service in another availability zone is charged at $ 0.01/GB.

This is only true for some AWS services like Amazon EC2, Amazon RDS, Amazon Redshift or Amazon ElastiCache instances, etc.

• Architect your systems so that there is minimal data transfer across AWS regions or availability zones.
• Architect your AWS environment such that data transfer is restricted to within an availability zone or within a region at the most.
• Try to use private IP addresses instead of public or elastic IP addresses wherever possible.

Conclusion

Fine-tuning your cloud infrastructure is critical to make sure that your overall bill stays in the limit. Proven cloud cost optimization strategies outlined in this blog will help you cut down your cloud costs by eliminating unused resources and/or choosing the right resource plan. Sify has been meticulously managing the cost optimization projects of large enterprise customers for many years to reduce their AWS bills substantially. Sify, with its highly dedicated, well experienced and AWS certified SMEs, can help you realize your business objectives by fine-tuning your environment.

If you are concerned about your ever-increasing AWS cost, the above-given strategies will help you optimize the cost. You can also choose Sify – the experienced AWS Managed Service provider – for yielding the best results.

Emerging technologies such as IoT, Artificial Intelligence, and Machine Learning have not only changed the business dynamics but have been generating an exponential volume of data, posing difficulties in its management. Processing this exponential burst of data with varying workloads in a timely and cost-efficient manner requires modern systems like High Performance Computing (HPC). Despite being in its early adoption stage, HPC is quite promising, as it has the capability to solve emerging business problems. Many business use cases have been developed, capturing the attention of the enterprises across the globe. Some of these use-cases where HPC has helped enterprises solve their problems are discussed here.

Fraud detection in the financial industry

Financial frauds have rocked the world with their inventiveness and scale. Some high-profile frauds such as Ponzi schemes, misleading investment schemes, identity fraud, phishing, card fraud, counterfeiting, skimming, fake prizes, and inheritance scams have duped many unsuspecting investors or users. The Global Economic Crime and Fraud Survey conducted by PwC in 2018 reveals that 49% of the organizations faced an economic fraud in the previous two years. While 52% of these threats were internally born, 40% were from external sources.

With a centralized system of fraud detection, information can be gathered from all the available sources. This system can help detect and prevent cyber-attacks, improve data privacy, enhance data protection, and support the digital forensic investigation. Given the humongous volumes of data, the system that is installed must have the capability to detect anomalies and intrusion attempts. Legacy data centers, despite their high capacities, are incapable of handling such large volumes of data. In this scenario, High Performance Computing can prove to be a useful aid in modern fraud detection and risk modeling, thereby helping financial organizations detect attacks on their systems as they occur, preventing financial fraud. PayPal implemented an HPC environment, and within a year of its adoption, it was able to save $710 million that would have been stolen by cyber thieves.

Personalized healthcare and clinical research

Modern advancements in healthcare have helped in the development of many pharmaceutical drugs and in identifying treatment options for a variety of ailments. The adoption of information technology by physicians, surgeons, and other medical professionals has helped them deliver accurate and timely medical treatment to their patients. Advanced medical simulations are often employed to ascertain the positive and negative effects of medicines on a patient profile to determine the most effective treatment option. These simulations demand high computing power as a number of factors need to be taken into account. This is another area where High Performance Computing can come to the aid of medical professionals with its significant computational power. Medical professionals can drill down into patient information at the genomic level, using millions of data points to help diagnose and identify personalized treatments for the patient.

Large hospitals have been using HPC for research. TGen and NMTRC have together developed an HPC system to obtain insights in much faster time into cancer and pharmaceutical research. By using these systems, new drugs can also be discovered, and life-saving medical treatments can be identified and personalized.

Smart energy grids

Smart energy grids are deployed to reduce energy consumption and offer more flexibility and reliability than traditional grids. These grids help in supplying energy to millions of households by integrating multiple energy sources. In order to supply, optimize, and maintain energy efficiency for multiple cities and neighborhoods, a huge volume of data is captured from millions of devices, including individual meters and consumption devices. These devices can generate exabytes of data, for which enormous computing power is needed for processing. Traditional servers cannot fulfill this need. However, with HPC, a huge volume of data can be processed and analyzed with efficiency in real-time.

Manufacturing Excellence

Large manufacturing enterprises have already begun to make use of the power of HPC, which is used for IoT and Big Data analysis. Based on the analysis results, real-time adjustments are possible in processes and tools to ensure an improved design of a product, increased competitiveness, and faster lead times.

High Performance Computing is capable of running large simulations, rapid prototyping, redesigns, and demonstrations. An example could be a manufacturing unit that would improve its manufacturing flow with insights from the processing of 25,000 data points from customer intelligence. The first-ever autonomous shipping project of the world is making use of HPC computing, which involves processing a large amount of data collected from sensors. The data includes details of weather conditions, wave points, tidal data, and conditions of various systems installed.

High Performance Computing offers significant benefits over traditional computing for manufacturing enterprises. It can help an automobile unit vehicle maintenance. A wholesaler could optimize the supply chain as well as stock levels. HPC is also used in R&D. The innovative design of the 787 Dreamliner Aircraft by Boeing is a result of HPC-based modeling and simulation that helped the company conduct live tests on the aircraft to test the prototype.

HPC has become indispensable for enterprises to derive competitive advantage in the fast-growing business world and scaling technologies.

Sify Data Center and cloud services help you to centralize your IT infrastructure, operations, storage & management and enjoy tremendous scale and a lower cost of ownership. Our consultative solution approach helps you define a business technology strategy where delivery of services supports clear business outcomes.

Data Centers have become nerve centers of the organizations in the modern digital world. Their performance and efficiency are crucial levers of organizational success. The challenge for technology heads has been to arrive at Data Center solutions that are cost-effective, offering value without compromising on essential features. Utilizing available space for Data Center scaling is a cost-saving method often employed by organizations. This measure can only yield short-term value. Creating a cost-effective Data Center with optimum performance and decent ROI, in the long run, requires elaborate planning. Companies can plan a cost-effective Data Center transformation using the following measures.

Disaggregated architecture

We often visualize a Data Center like the one with racks of servers, blinking and humming away. What happens when one of the server modules get compromised or damaged? Will you replace the entire server? It will be wiser if you replace or upgrade only the equipment in question as this approach has lower downtime and is more cost-effective. A disaggregated architecture like High-Performance Compute (HPC) offers this benefit. An intelligent HPC architecture can transform your Data Center and give you significant savings in the long term.

The plan for a Data Center transformation or development relies on some key functions like design computing and power management. Making effective use of these functions would result in cost saving. Electronic design automation can shorten the design cycle for the Data Center architecture, which reduces the time to market. Further, with disaggregated architecture offered by HPC, physical operations become more cost-effective than workloads running on the cloud. A comparative cost saving upwards of 60% can be achieved.

Traditional Data Centers used power architectures with limitations and thus, incurred a major cost when scaling up, but modern Data Centers use low power consuming servers and efficient cooling systems and thus, have higher Power Usage Effectiveness and less cost. A desegregated architecture can further bring down the operating costs of a Data Center.

Micro-Data Centers

Scalability, speed and reliability. The modern business world relies on these elements for its operation, and they can be optimized by deploying the right technology. If the costs are not managed well, it can put a strain on the organization. The question is – Is the desired ROI achievable? The use of Micro-Data Centers that are located closer to the points of consumption can reduce latency and cost both for cloud and a remote Data Center. The micro-architecture can reduce up to 42% of the capital expenditure when compared to traditional Data Centers.

Data Centers designed with technologies like virtualization, compaction, and hyperconvergence result in significant cost savings. Virtualization allows a company to use the computing power across workloads in a Data Center, saving power; compaction consolidates multiple racks into one, saving space. On top of it, hyperconvergence integrates high performance compute, high performance storage, and networking, thereby increasing the speed of deployment.

Power provisioning

Around 3% of the total electricity in the world is utilized by global Data Centers, and that amounts to roughly 416 terawatts. By 2025, Data Centers would end up consuming a fifth of the global power. No wonder, power is a major concern when it comes to Data Centers, and power-saving measures bring cost benefits for an organization. Power provisioning can be used to optimize Data Center power consumption by understanding how power is consumed by equipment, servers, and workloads, and optimizing it.

Power supplies have a sweet spot, the level at which high operational efficiencies can be achieved. However, most Data Centers operate below this level. The reason why companies could be restrictive in using enough power is the cautious measure of average and peak power consumption given on the nameplate of a power supply equipment. However, it is important to understand that if the Data Center is operating below this sweet spot, it could be wasting energy, and this wasted energy would further increase the need for cooling. Using the right power supplies, amortizing power across servers, and running Data Centers at optimum loads can bring efficiency and cost-savings.

A cost-effective Data Center transformation can be achieved through effective architecture, fast deployment, and optimized power consumption. Organizations can beneficially employ these measures when scaling up their Data Centers for achieving long-term gains.

Sify Data Center and cloud services help you to centralize your IT infrastructure, operations, storage & management and enjoy tremendous scale and a lower cost of ownership. Our consultative solution approach helps you define a business technology strategy where delivery of services supports clear business outcomes.