5 Common AWS EC2 Challenges – and How to Tackle Them!

5 Common AWS EC2 Challenges – and How to Tackle Them!

The Cloud revolution is well and truly underway, as demonstrated by the 44,000 attendees at the recent Amazon Web Services (AWS) re:invent conference. Many businesses have adopted a Cloud provider like AWS in some form or another.

AWS’s Elastic Compute Cloud (EC2) has been a service offering since 2006. It allows users to launch virtual computing environments on demand. EC2 is one of over 100 services from AWS, and each provides incredible value in your business’ journey to the Cloud.

Many businesses will be in the early stages of their Cloud adoption journey. We have seen some really successful transitions and some…not so great transitions. This post will explore some of the challenges with AWS EC2, and how they can be solved.

Challenges with AWS EC2

Resource Utilisation

Challenge: AWS EC2 makes it easy for businesses to scale. EC2 gives you complete control over your instances, with a range of instance types at your disposal. The challenge in these cases is how you manage the number of instances you have, so that costs aren’t impacted by large, long-running instances.

Solution(s):

  • Limit the number of acceptable instances, using Infrastructure as Code tools such as AWS’ CloudFormation or Hashicorp’s Terraform as a provisioning strategy. This will help display resource graphs to gain a further insight into your infrastructure.
  • Understand the type of instances you require. AWS has four payment options for instances: on-demand, reserved, spot and dedicated. These will significantly reduce cost and help a business understand if EC2 is being used in the right way. For example, if all your instances are dedicated, you are most likely not leveraging the true benefits of the Cloud.
  • Use AWS CloudWatch to detect and shut down idle instances. This will remove any long running instances that are not used, and ensure the environment is not cluttered.

Security: 

Challenge: Whilst EC2 places importance on security, many organisations still face challenges when ensuring that instances are running securely. What happens when you have an instance that is public-facing? Who has access, and how is this monitored?

Solution(s):

  • Use AWS CloudTrail. This will track all user and API usage. This, as a minimum, will help toward auditing and begin to satisfy compliance controls.
  • Create rules that restrict misconfigured instances, such as allowing for Public IPs. These could be integrated into your CloudFormation or Terraform
  • Use Amazon GuardDuty to monitor your AWS accounts and workloads. This uses intelligent threat detection to determine any malicious activity, and can take action with automated remediation.

Deploying at Scale

Challenge: Running hundreds (or even thousands) of instances can result in unmanageable and cluttered environments. This can make it difficult to determine who owns which instance, which regions are using it and what it’s being used for.

Solution(s):

  • As your business scales, separate it into different AWS accounts, to maintain control. AWS Organisations will enable policy-based management for these separate accounts.
  • Use CloudFormation or Terraform to enforce a tagging strategy for the separation of environments, applications, business units and more.

Configuration Management

Challenge: Businesses will use some of the default Amazon Machine Images (AMIs) provided by AWS. However, as adoption matures, many find that custom configurations need – such as additional users and patching – need to be made.

Solution(s):

  • Create a process to manage the lifecycle of your AMI, using the default AMIs. Then, use Hashicorp’s Packer, to make further changes to the image.
  • Use Cloud Init to handle the early initialisation of an instance. This, along with other config management tools such as Puppet and Ansible, can be used to make custom changes.

Serverless

Challenge: Managing EC2 instances! What if we could deploy code without worrying about the instances it has to get deployed to?

Solution: Use AWS Lambda. AWS Lambda lets you run code without provisioning or managing servers. With Lambda, you can run code for virtually any type of application or back-end service. Just upload your code, and Lambda will take care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services, or call it directly from any web or mobile app. The learning curve for Lambda can be steep, but once you have passed that barrier, you will never look at code deployment in the same way again.

How to best adopt AWS Cloud

Many of the solutions we’ve mentioned in this post are tools available within AWS. When starting your Cloud journey, it’s important to understand these resources (and the many more) that are available in AWS, to ensure successful implementation.

Some additional general practices that should be considered when adopting any Cloud, include:

Cost Management

Using tools like AWS Cost explorer will enable you to see patterns and trends in your spend over time, that can help you understand Cloud costs. This data can then be used to forecast Cloud costs over the next quarter, which can be used to set budgets for your Cloud spend. Tools like AWS budgets can alert businesses when costs or usage are forecasted to exceed, and provide oversight over where overspend is occurring.

Build with fault tolerance in mind

Nowadays, companies who don’t achieve 99.99% uptime are in risk of grave loss of both business and client trust, simply because they’re not available (usually in a high-traffic periods). Tools such as AWS S3 guarantee 99.99% uptime for your static assets, while services like RDS and CloudFront are designed with failover in mind, to provide HA and data availability at any given time. AWS also publishes regular whitepapers that illustrate how to architect and build resilient applications, helping businesses to decrease infrastructure and ownership costs.

There will always be challenges and learnings involved in the adoption of new technology. To make sure your Cloud migration is as effective, efficient and valuable as possible, it’s important to consider potential challenges and solutions of Cloud configuration, migration and management, before you adopt. Being aware of the challenges your business might experience using AWS or any other Cloud platform will enable you to tackle any issues and recover much more quickly.

If you’re experiencing issues with your AWS implementation, or Cloud infrastructure in general, please get in touch. ECS Digital offers a Cloud Health Assessment to help businesses realise the potential of Cloud and ensure their applications are truly native in the Cloud.

Thivan Visvanathan5 Common AWS EC2 Challenges – and How to Tackle Them!
read more
ECS Digital sponsors DevOps Enterprise Summit 2017

ECS Digital sponsors DevOps Enterprise Summit 2017

I know what you’re thinking to yourself as soon as you read this title.

“An Enterprise Summit? That sounds like buzzwords, free food and a lack of innovative talks.”

But, you couldn’t be more wrong. A moment of honesty: DOES17 is the first large talk in the DevOps space that I’ve attended. So, I also had the same thoughts as above.

I was pleasantly surprised. The summit is aimed at large, complex organisations, who are either just beginning their digital transformation into an agile world – or assessing their current architecture to see how their processes align with the rest of the industry.

Tackling Digital Transformation

I’ve worked with both start-ups and large corporations, so I’m well versed in the practices that praise and haunt both sides. Many complain about ease of innovation vs. unnecessary regulatory hoops, but not many people understand why the walls blocking innovation are so hard to tear down.

DevOps Enterprise Summit 2017(DOES17) shared digital transformation stories from many dominant companies that have encountered roadblocks to innovation. Just a few notable talks included:

How Do You Fit a Core Banking System into a Few Containers?

Amine Boudali, Senior Project Manager, Nordea

Jose Quaresma, DevOps Lead DK, Accenture

DevOps at Capital One: Focusing on Pipeline and Measurement.

Tapabrata “Topo” Pal, Director, Engineering Fellow, Capital One.

Platform engineering at ASOS.

Ian Margetts, Platform Lead, ALM, ASOS.

But, DOES17 wasn’t just talks. There were also breakout sessions, workshops and Q&As with the speakers. So, it’s an experience that provides actionable insight into enterprise adoption of DevOps.

Digital Transformation at ASOS 

One of the best stories shared about a dominant industry undergoing a massive transformation, was from ASOS.

Their story of moving to a platform engineering team had all of the typical burdens associated with a sizeable corporate entity:

  1. A large monolithic codebase
  2. Non-scalable architecture
  3. Lack of upskilling and disillusioned teams.

Their resolution to these issues? To embrace the platform way of working – which subsequently lead to them adopting the DevOps approach.

The codebase was broken down into a micro-services based architecture. Platform teams were born and developers were forced to take on more responsibility, putting platform first, features second.

It took them only two iterations to achieve deployment to production 300 times a day. But their journey still wasn’t over. As they revealed, it wasn’t a smooth transition. And, they still faced numerous challenges: their security didn’t scale, and their business requirements needed to be balanced with IT.

Companies ASKING for help?  

It was here I realised something was different. The companies at DOES17 weren’t trying to sell themselves to the audience. They were genuinely asking for help.

This honesty was much needed. And, by asking for help, an atmosphere of collaboration was born. The trend was repeated throughout the day, with businesses dedicating numerous slides that simply stated what they were stuck on – and asked whether anyone in the crowd could help.

DevOps Enterprise Summit

Offering Solutions  

The majority of booths present at the event compromised of vendors. Their aim was simple: offer complete solutions for common problems that were being raised throughout the various workshops & talks.

ECS Digital was in a unique position compared to other booth exhibitors. We too were selling a product, but that product was us – our expertise.

We talked to many companies including Barclays, RBS & KPMG, and we found that they were already using many of the tools in our partnership portfolio. The problem they faced, however, was a lack of a maturity assessment.  They were unsure of how well they were doing in their DevOps journey – or if they were even doing DevOps at all.

Another scenario that seemed to crop up often was how to train internal employees to use all of their wonderful tools. Vendors offer the solutions to enterprise problems, but there is often no-one to implement and maintain the solutions in an organisation.

DevOps Enterprise Summit

Overall, DOES17 was a great experience. As well as highlighting major pain points of the industry in terms of challenges both solved and unsolved, it managed to create a positive atmosphere of collaboration.

I’m sure everyone’s looking forward to the next event in San Francisco!


Do get in touch if you’re interested in discussing what’s involved in our DevOps Maturity Assessment or if you have any questions for us. We’d love the chance to discuss how we can help you in your adoption of DevOps. 

Thivan VisvanathanECS Digital sponsors DevOps Enterprise Summit 2017
read more