DevOps Playground – Hands On with CloudBuild

DevOps Playground – Hands On with CloudBuild

Led by our very own Michel Lebeau, this Playground explores the fundamentals of a CI/CD pipeline using CloudBuild.

Over the session, we walk guests through how to create a basic build config file that defines the steps and parameters needed for CloudBuild to perform your tasks.

We also look at how to build and test a golang application, and then finish off the Playground by deploying the application using Google Cloud App Engine. And then we prepared this video so you can give it a go from the comfort of your home!

***

Interested in attending our next DevOps Playground in London? Follow us on Meetup to receive a notification about the next event.

Check out the Meetups we have at our other global locations:

You can also find all the information and resources you need about DevOps Playground sessions, upcoming events and past events on our website: https://devopsplayground.co.uk

Michel LebeauDevOps Playground – Hands On with CloudBuild
read more
CloudBees & Electric Cloud: the holy grail for CI/CD software?

CloudBees & Electric Cloud: the holy grail for CI/CD software?

As a specialist DevOps consultancy, ECS Digital often finds itself at the forefront of new and emerging technologies. We work with clients that aim to solve ever more complex problems and have established a history of working with industry-leading software vendors in response to the tools required to tackle these problems head-on. This has enabled ECS Digital to become intrinsically linked to the ever-evolving nature of the business software world.

What we’ve come to realise is that there is a natural lifecycle to the software vendors we work with. Some will grow quickly, establishing themselves as leaders in their market, and will eventually go public in an IPO. Some will fail, falling away as victims of the marketplace. And some will be acquired by another software vendor to be included in a wider portfolio of products. This is a common trend, as we have seen with GitHub joining Microsoft and Red Hat becoming part of IBM, both in multi-billion-dollar deals.

And this trend continues, with CloudBees and their recent acquisition of Electric Cloud.

Electric Cloud is the second business to be acquired by CloudBees – Codeship, a continuous integration and continuous delivery firm, being the first in 2018. These deals pair nicely with two of the end-of-life cycles outlined earlier. They also affirm CloudBees’ overall strategy of acquiring smaller, specialist software companies as a way of bringing onboard expertise missing from their current offerings.

In the words of Andy Cureton, Managing Director and Founder of ECS Digital:

“Combining CloudBees and Electric Cloud gives the combined entity the capability breadth to compete against the AWS CI/CD stack and the Microsoft CI/CD stack prevalent on Azure. Combining the feature depth of multiple tools in a seamless capability that is platform agnostic also gives a powerful alternative to those with a brown field site, as well as addressing concerns around vendor lock in (particularly on Cloud)”.

Phil Drouet, Head of Channel at ECS Digital, agreed with Andy, adding that today’s software landscape enables users to “build their own pipeline and pick their own tools. Whilst it may seem that choosing one ‘continuous delivery powerhouse’ limits your choice, this is offset by integrated systems and better experience. At the end of the day, you don’t want every development team to have their own tools. I have no doubt that enterprises will see this as a good thing, a credible alternative to having to buy everything from different places”.

Is CloudBees the holy grail for CI/CD software?

Electric Cloud is a known brand in its own right, with Gartner positioning them as a leader in its Magic Quadrant for Application Release Orchestration just last year. By acquiring Electric Cloud, CloudBees have strategically strengthened their position in the CI and CD space, as well as allowing them to enter the end-to-end solution market. This will help protect them in a marketplace that is increasingly offering these solutions when migrating to the Cloud.

Not only are they home to the enterprise version of Jenkins, they now have a compelling brand story within the CI/CD and release automation arena. What’s more, these products can now be combined into a single suite, offering the holy grail of product portfolios, without the complexity. In the words of Sacha Labourey, the CEO and co-founder of CloudBees:

“As of today, we provide customers with best-of-breed CI/CD software from a single vendor, establishing CloudBees as a continuous delivery powerhouse. By combining the strength of CloudBees, Electric Cloud, Jenkins and Jenkins X, CloudBees offers the best CI/CD solution for any application, from classic to Kubernetes, on-premise to Cloud, self-managed to self-service.”

The joining of CloudBees and Electric Cloud will unquestionably result in a stronger product set, and thus a stronger brand for those looking for a CI, CD and release platform partner. Electric Cloud evidently feel the same, as being a previously well-funded vendor meant that this acquisition did not come about as a result of them struggling in the marketplace. Much the opposite; “it will strengthen the market for them as a unit and give CloudBees (and now Electric Cloud) another revenue stream” (Phil Drouet).

And it benefits users too, as noted by Christina Noren, Chief Product Officer, CloudBees:

“Having the Electric Cloud offerings under the CloudBees umbrella gives companies a greater ability to manage the delivery of value to customers.

Having CI and CD solutions under one banner may mean customers come to rely on CloudBees. But where monopolistic powerhouses have spelt doom for innovation in other markets, in this case, Andy Cureton sees this as “giving back control” to the customer. It’s a holistic offer that means customers lessen the risk presented by multiple vendors and unintegrated systems.

What this acquisition means for partners

Being the Service Delivery Partner of the year for CloudBees, and with a partnership stretching back many years, we will inevitably see a shift in what we will need to provide following the integration of Electric Cloud.

Part of this shift will involve witnessing new challenges emerge, especially during the ‘settling in’ period where the merging vendors decide upon strategies, personnel and technical directions. We’ll also be keeping an eye out for any innovative products born from this acquisition and look forward to introducing these offerings in future projects. Whilst nothing has been confirmed, we imagine CloudBees will begin to release more information regarding their new direction towards the end of the year, timing it nicely with their annual DevOps World | Jenkins World | CloudBees conference – this year taking place in sunny San Francisco and Lisbon, Portugal.

Despite the turbulence that may occur, working closely with partners in the DevOps software world, and having a legacy of trust and reliable support, we are best placed to deliver the same high-quality service support to software vendors at times of change. And thanks to our existing relationship with CloudBees, we are able to upskill our team at pace. We can get ready to hit the ground running as new tools and technology emerge as a result of this deal.

Not only has their recent acquisition piqued industry interest, CloudBees have reaffirmed themselves as a technology vendor to watch. Not only are they bulking up their market presence, they are also providing customers with an extensive offer in the CI/CD and Release Automation space. And since this new option will be simpler and more robust, more customers will no doubt be drawn to this valued convenience. After all, complexity is the killer of progress.

If you’re yet to reap the benefits of CloudBees and Electric Cloud for your business, talk to a member of the ECS Digital team today.

 

****

Image Credit: <a href=”https://www.freepik.com/free-photos-vectors/background”>Background vector created by creative32965 – www.freepik.com</a>

Phil DrouetCloudBees & Electric Cloud: the holy grail for CI/CD software?
read more
Take your testing to the Cloud

Take your testing to the Cloud

There are lots of reasons why companies choose to make the transition to the Cloud, but it’s safe to say that improving the speed and accuracy of your testing is rarely one of them. In fact, the benefits to your testing after moving to the Cloud often go unrealised. This is not because getting to those benefits is hard (it isn’t), or because there are clear reasons for keeping testing on-premise (we would argue that in most circumstances there really aren’t any), but simply because the focus tends to be elsewhere.

In this piece we are going to play out some of the key benefits and also address some of the misconceptions there are around potential barriers.

The benefits:

  1. It’s easy to set up and provision testing environments

Traditionally, getting an environment up and running is a process that takes days, potentially weeks. This means it’s intensive, both in terms of time and resource (which equals money). It also means that in some instances, test environments may not be set up because the cost is seen as too high. Take for example testing your code changes once you have raised a pull request. Creating a test environment for your pull request can have significant benefits when it comes to speeding up delivery and feedback. On-premise it would be highly unlikely to spend the time setting up test environments for pull requests. Testing would be done locally, with issues often missed creating further problems down the line.

In the Cloud, Infrastructure as Code tools like CloudFormation or Terraform can go up in a matter of minutes. You can create a new test environment as you need it and then simply take it down when you are done.

  1. Consistency

By using tools like Docker, you can get a far greater level of consistency between environments. This means you can be more confident you are testing like for like. On-premise there will almost always be small, but potentially important, discrepancies between environments.

  1. Data creation and manipulation

Poor quality data is always an issue when it comes to testing. The worse the data, the fewer issues you will be able to uncover. It is also hard to know whether the test data you have is any good until the testing is underway – and by then it’s too late. Tools like Docker can again be very useful, because the quality of data will be far more consistent.

  1. Scaling

In the on-premise environment, running tests in parallel rather than sequentially would be almost impossible. This is where Cloud comes in.

Because creating and provisioning environments is so much easier in the Cloud, parallel testing is much more doable. You can run the same test across multiple scenarios or run multiple test cases at the same time. Running tests in parallel not only saves a considerable amount of time, it is also far easier to validate different permutations such as browser types and versions.

  1. Faster time to market with reduced risk

Not only does the Cloud enable you to move through test cycles faster, it also enables you to do it with less risk. Tools like Docker and Heroku enable you to release in much smaller chunks which means it is far easier, and faster, to deal with points of failure and then move forward. The fully automated release process also means less manual interference, which in itself reduces risk further.

The barriers:

While the benefits of testing in the Cloud are clear, there are still some concerns / perceived barriers that might hold people back:

  1. Cost

The cost of Cloud is something that is at the top of a lot of people’s minds. Some of those that have already made the transition are finding the cost is substantially higher than they imagined. A lot of this is down to how it is managed, and the same applies to the testing piece. Because environments can be spun up so quickly, there is a danger that they will just proliferate, and numbers will get out of hand. It is important that there is clear governance and process in place to ensure environments are taken down when they are no longer needed.

  1. Security

A few years ago, security was considered the biggest barrier to moving to the Cloud full stop. These days, most will admit that security is often better in the Cloud than on-premise, but that is not to say that security problems don’t exist. In essence, the same rules apply in both environments. Do a good job and follow the right processes and security shouldn’t be an issue. One clear benefit, however, is that in the Cloud environment, it is much easier to test just how good the security is.

  1. Disaster recovery

There have been a number of high-profile instances of Google and Amazon outages affecting customers. The most high-profile causing data to be rerouted to China – which is tricky as Google doesn’t do business there!

The fact is, although these are high profile, they are usually pretty low impact and actually far less likely than on-premise. However, in the case of security, one major benefit of the Cloud testing environment is that it gives you the ability to test your disaster recovery far more effectively.

It is pretty clear that for almost everyone, moving their testing into the Cloud will deliver significant benefits. Although it might not be the thing that is driving Cloud adoption, it is certainly a substantial value add.

To find out about how you can successfully move your testing to the Cloud, talk a member of the ECS Digital team to discuss how you can start reaping the benefits mentioned above.

_______________

Image credit

Kouros AliabadiTake your testing to the Cloud
read more
Out Scaling Peak Load

Out Scaling Peak Load

Not building your website for scale can be extremely detrimental to your service / product line and reputation – and can become an expensive mistake in the long term! Surges in genuine traffic are a rare opportunity, which makes it terrible timing for your website to be crashing.

In this short 25 minute talk, Morgan Atkins, DevOps and Continuous Delivery Consultant at ECS Digital, covers:

  • Why you want to engineer for scale
  • How you can build your services to scale
  • What the common success factors are
  • Where this technology is moving to next, and how this evolution will support scale beyond the Cloud

 

 

You can also watch the video for free on our YouTube channel here and learn how you can get yourself in the best position to react to unforeseen, performance-critical traffic spikes to your website.

If you want to talk to the team about any specific parts of the lecture, please reach out to hello@ecs-digital.co.uk and one of our consultants will be in touch to help answer your questions.

***

Banner Photo credit: Farzad Nazifi on Unsplash

Morgan AtkinsOut Scaling Peak Load
read more
ECS Digital at Cloud Expo Europe

ECS Digital at Cloud Expo Europe

This week saw thousands of IT professionals head to the Europe’s largest Cloud exhibition – including ECS Digital and our village of partners!

From blockchain and IoT vendors, to datacentre climate control systems manufacturers and cable suppliers, the crowds enticed anybody interested in Cloud-first, scaling up, refining, or just getting started with the tools and technology.

As has been echoed in previous events, Cloud Expo Europe is an unrivalled opportunity to meet with leading technology innovators and service providers, and network with your peers. Attendees also gain access to a wealth of knowledge and advice including emerging trends, tech deep dives, lessons learned and market forecasts.

ECS Digital once again have donned their conference apparel and deployed Europe’s only DevOps Partner Village, bringing together vendors from across the DevOps tool chain. This included Sonatype, New Relic and Storage OS – experts in application performance monitoring and artifact lifecycle management to dynamic container storage, all under one red, white and blue banner.

As you’ll probably know by now, ECS Digital is an agnostic DevOps and continuous delivery consultancy that provides professional and DevOps expertise to thousands of customers, and this year they have been showcasing the newest feather in their cap; Enablement Pods™.

Enablement PodsTM introduce a new way of working, processes and tooling. By remaining flexible in their resource profiling, our fixed priced outcome-based engagement model enables you to transform at scale by embedding – for short periods – in existing engineering teams to enable them to implement true change. What’s more, you decide the outcomes and objectives for each sprint, and we put our fees at risk to ensure we deliver.

It really is that simple.

During their talk on day one, Young DevOps Consultant of the Year 2018 Michel Lebeau and Pod Delivery Lead Tom Chapman described Enablement Pods™ as ECS Digital new weapon for making long-lasting and impactful changes in the Enterprise space. They also walked guests through how they are using Enablement Pods™, some of the challenge case studies and what Enablement Pods™ has achieved. If you missed the talk, you can learn more here.

ECS Digital also introduced a second, equally exciting initiative at Cloud Expo Europe. The DevOps Playground is supporting the tech community by giving enthusiasts a platform to explore new technology and tools. We’re also super chuffed that the Playground reached 4,000 members worldwide during the show – we’re giving credit to the talk our team delivered on day two!

I was fortunate enough to deliver the DevOps Playground story, describing how ECS Digital have used the DevOps Playground to build a tech community that encourages tech enthusiasts to not only learn about tools, but to gain hands-on experience under the supervision of engineers.

Each month, the DevOps Playground welcomes large numbers of tech-enthusiasts to one of our four worldwide locations – London, Singapore, Pune and Edinburgh. Each Playground lasts for around 2.5 hours, with a chunk of that time set aside for individuals to run and use the chosen tech / tools on their own laptop.

Attendees can expect to follow along with a structured and comprehensive exercise, designed to jumpstart new users with unfamiliar technologies and to highlight the best ways to use the technology. If this is the first you’ve heard of it, head over to the DevOps Playground website to find out more.

Other key takeaways from the show include:

  • Containers are driving the Cloud adoption of software development for 90% of the customers we spoke to
  • DevOps is the new standard for Cloud and agile working and is driving the need to modernise the more traditional methods
  • Cloud Expo is the one-stop-shop for all things Cloud – it was an eye opener seeing how much physical infrastructure goes into the ‘Cloud’

And that is just the tip of the iceberg. We will be revealing a mini-video series about Cloud Expo over the next couple of weeks to recapture some of the best moments from the show – watch this space!

Morgan AtkinsECS Digital at Cloud Expo Europe
read more
5 Common AWS EC2 Challenges – and How to Tackle Them!

5 Common AWS EC2 Challenges – and How to Tackle Them!

The Cloud revolution is well and truly underway, as demonstrated by the 44,000 attendees at the recent Amazon Web Services (AWS) re:invent conference. Many businesses have adopted a Cloud provider like AWS in some form or another.

AWS’s Elastic Compute Cloud (EC2) has been a service offering since 2006. It allows users to launch virtual computing environments on demand. EC2 is one of over 100 services from AWS, and each provides incredible value in your business’ journey to the Cloud.

Many businesses will be in the early stages of their Cloud adoption journey. We have seen some really successful transitions and some…not so great transitions. This post will explore some of the challenges with AWS EC2, and how they can be solved.

Challenges with AWS EC2

Resource Utilisation

Challenge: AWS EC2 makes it easy for businesses to scale. EC2 gives you complete control over your instances, with a range of instance types at your disposal. The challenge in these cases is how you manage the number of instances you have, so that costs aren’t impacted by large, long-running instances.

Solution(s):

  • Limit the number of acceptable instances, using Infrastructure as Code tools such as AWS’ CloudFormation or Hashicorp’s Terraform as a provisioning strategy. This will help display resource graphs to gain a further insight into your infrastructure.
  • Understand the type of instances you require. AWS has four payment options for instances: on-demand, reserved, spot and dedicated. These will significantly reduce cost and help a business understand if EC2 is being used in the right way. For example, if all your instances are dedicated, you are most likely not leveraging the true benefits of the Cloud.
  • Use AWS CloudWatch to detect and shut down idle instances. This will remove any long running instances that are not used, and ensure the environment is not cluttered.

Security: 

Challenge: Whilst EC2 places importance on security, many organisations still face challenges when ensuring that instances are running securely. What happens when you have an instance that is public-facing? Who has access, and how is this monitored?

Solution(s):

  • Use AWS CloudTrail. This will track all user and API usage. This, as a minimum, will help toward auditing and begin to satisfy compliance controls.
  • Create rules that restrict misconfigured instances, such as allowing for Public IPs. These could be integrated into your CloudFormation or Terraform
  • Use Amazon GuardDuty to monitor your AWS accounts and workloads. This uses intelligent threat detection to determine any malicious activity, and can take action with automated remediation.

Deploying at Scale

Challenge: Running hundreds (or even thousands) of instances can result in unmanageable and cluttered environments. This can make it difficult to determine who owns which instance, which regions are using it and what it’s being used for.

Solution(s):

  • As your business scales, separate it into different AWS accounts, to maintain control. AWS Organisations will enable policy-based management for these separate accounts.
  • Use CloudFormation or Terraform to enforce a tagging strategy for the separation of environments, applications, business units and more.

Configuration Management

Challenge: Businesses will use some of the default Amazon Machine Images (AMIs) provided by AWS. However, as adoption matures, many find that custom configurations need – such as additional users and patching – need to be made.

Solution(s):

  • Create a process to manage the lifecycle of your AMI, using the default AMIs. Then, use Hashicorp’s Packer, to make further changes to the image.
  • Use Cloud Init to handle the early initialisation of an instance. This, along with other config management tools such as Puppet and Ansible, can be used to make custom changes.

Serverless

Challenge: Managing EC2 instances! What if we could deploy code without worrying about the instances it has to get deployed to?

Solution: Use AWS Lambda. AWS Lambda lets you run code without provisioning or managing servers. With Lambda, you can run code for virtually any type of application or back-end service. Just upload your code, and Lambda will take care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services, or call it directly from any web or mobile app. The learning curve for Lambda can be steep, but once you have passed that barrier, you will never look at code deployment in the same way again.

How to best adopt AWS Cloud

Many of the solutions we’ve mentioned in this post are tools available within AWS. When starting your Cloud journey, it’s important to understand these resources (and the many more) that are available in AWS, to ensure successful implementation.

Some additional general practices that should be considered when adopting any Cloud, include:

Cost Management

Using tools like AWS Cost explorer will enable you to see patterns and trends in your spend over time, that can help you understand Cloud costs. This data can then be used to forecast Cloud costs over the next quarter, which can be used to set budgets for your Cloud spend. Tools like AWS budgets can alert businesses when costs or usage are forecasted to exceed, and provide oversight over where overspend is occurring.

Build with fault tolerance in mind

Nowadays, companies who don’t achieve 99.99% uptime are in risk of grave loss of both business and client trust, simply because they’re not available (usually in a high-traffic periods). Tools such as AWS S3 guarantee 99.99% uptime for your static assets, while services like RDS and CloudFront are designed with failover in mind, to provide HA and data availability at any given time. AWS also publishes regular whitepapers that illustrate how to architect and build resilient applications, helping businesses to decrease infrastructure and ownership costs.

There will always be challenges and learnings involved in the adoption of new technology. To make sure your Cloud migration is as effective, efficient and valuable as possible, it’s important to consider potential challenges and solutions of Cloud configuration, migration and management, before you adopt. Being aware of the challenges your business might experience using AWS or any other Cloud platform will enable you to tackle any issues and recover much more quickly.

If you’re experiencing issues with your AWS implementation, or Cloud infrastructure in general, please get in touch. ECS Digital offers a Cloud Health Assessment to help businesses realise the potential of Cloud and ensure their applications are truly native in the Cloud.

Thivan Visvanathan5 Common AWS EC2 Challenges – and How to Tackle Them!
read more