ECS Digital is proud to once again sponsor the DevOps Days Singapore event. This year’s theme of ‘feedback loop’ was reflected strongly in the keynote talks that took place, such as ‘Data Drive DevOps’ and discussions about technologies like Hashicorp’s Consul.
Alongside the various keynote talks, there were sharing sessions and open discussions, providing company experiences of implementing feedback loops, or the cost of lacking such tools. Topics ranged from more general ones like ‘Busting DevOps’ and ‘Automation of legacy tools’ to ‘Chaos Engineering in production and banking’.
You only have to change one thing to ‘do the DevOps’, everything
The opening talk was delivered by Ken Mugrage, establishing the theme of the conference and reminding everyone what DevOps was, and just as importantly what it wasn’t. On top of the usual encouragement of companies to change their mentality to embrace DevOps, Ken talked about reorganising teams into product teams to encourage developers to take ownership over features and products, as well as involving other teams such as compliance throughout the process
One interesting concern raised was that of product team’s unwillingness to iterate further for fear of taking on even more responsibility, since each product created would be another the team would need to manage. What team would be willing to continually innovate if it meant an ever-increasing amount of responsibility and workload?
Ken addressed this concern with the idea that product teams can “earn” the right to hand over a product to another team for maintenance. If a feature/product is stable and no longer a priority to iterate upon and improve, it can be passed on. If the situation changes, then the product team will take back the product and begin the iterative process anew. This prevents product teams from being burdened by their own innovation.
It’s Down! Simulating incidents in production
Kelsey Pedersen from Stitch Fix shared her company’s experience with implementing chaos engineering, i.e. deliberately injecting failure in a system in a bid to build more resilient systems. Sharing interesting observations on other industries having dry runs and practices for handling crises, she commented on the general lack of a similar rigour within many software companies.
Instead of preparation, programmers are expected to learn on the spot to deal with crises as they happen, and often under extremely stressful and thus non-conducive conditions. How could people learn from these experiences and be expected to perform better in the future?
To address this, it is beneficial to encourage incident simulation in production systems. This ensures system failures do not bring the entire application down, on top of better training your engineers to deal with similar future problems. This knowledge and experience is invaluable in crisis management as well as prevention of similar service outages.
Addressing the concern companies may have about limited resources and viability of such methods, companies should focus their efforts on simulating failures that have the highest frequency and impact. This enables the most benefit to be garnered from the simulations.
Our Hashicorp Partner Ned Shawa gave an overview of the Consul tool, a service discovery tool to help with scaling connectivity management for Dynamic Infrastructures, among other issues. As ever, Hashicorp is dedicated to keeping Consul as flexible as possible, supporting integration for the many tools in the market, such as Cloud services like Azure, AWS and GCP and tools like Kubernetes and Docker.
Ned describes several challenges that have arisen from the migration from static to dynamic systems, such as the securing of service-to-service communications. While conventionally handled with firewall rules, it is unfeasible with large systems that may necessitate 100s of firewall rules. Instead, Consul allows the implementation of a select few Intentions that serve the same purpose, easing the management of such large, dynamic systems.
As usual, DevOpsDays Singapore is a great event to attend, full of useful insight and like-minded individuals. We are proud to have been Gold sponsors for the second year in a row – thank you DevOpsDay for an excellent day!
If you are interested in learning more about ECS Digital and the services we can provide, please get in touch.