Cloud and Agile: why do they work so well together?
It sounds really geeky but over the last year, I have ‘fallen in love" with Agile.
I came from a traditional infrastructure IT background, and when I heard ‘Agile’ being banded around by my colleagues and peers, I half saw it as an excuse to not plan, or as a big buzz word just because the powers that be had heard about it at a conference that year.
I had never seen people successfully implementing it, so it had a bit of a stigma in my head. I am glad I can now be honest and call out that bias, because more recently I have really seen how successful Agile can be in cloud projects. This is what I mean by "falling in love" with Agile. I have seen the true measurable benefits of Agile and, after achieving such successes myself at Cloudreach, I cannot see myself going back. This blog hopes to describe why, and help enterprises embarking on their cloud journey to buy into trying to deliver in a more Agile way.
Agile & Cloud from an Organisational Perspective
Working solely on cloud projects in the two years , I have seen that cloud and Agile tend to go hand in hand.
Why is this? Well, cloud is something new and transformative for a lot of enterprise customers and, because it’s so new, these organisations lack the requirements or exact understanding of how they want to utilise the cloud. Naturally they struggle to articulate what they want and need, which as a problem lends itself nicely to the user story and demo methodologies in Agile.
‘What do I mean by this?’, I hear you say. I mean that Agile methodologies make it very easy for companies to articulate on a "high level" what they think they want users to be able to achieve with the cloud. They do this using the three pillars of Agile, which are transparency, inspection, and adaptation(http://www.scrumguides.org/scrum-guide.html for more details).
You can demo functionality and certain behaviours to your organisation early, and learn to adapt to their needs and help them gain an understanding of what they really want to achieve with cloud technologies. This is also critical for organisations as it makes the development transparent and the demo acts as an inspection element for the company to workout if the product is exactly what they wanted. At Cloudreach this has allowed me to present back to large organisations and assist them into getting to grips with how they would like to use the cloud, instead of a more traditional way of planning something which means spending months developing something that the business then might realise it doesn’t want.
I see the benefits to Agile planning as twofold:
- It allows the organisation (consultancy or internal IT arm) to be more responsive to demand
- It reduces the demand for change control (even more important from a consulting perspective) or delivering to a fixed scope, which would be highly administrative
Agile & Cloud from a Change Management Perspective
Cloud isn’t just about a change of technology, it’s also a massive organisational change. I was reading an interesting article on Forbes which I have quoted below, and I believe it rounds out nicely another key reason why Agile is super effective in the context of Cloud adoption:
"The key to success in this new climate isn’t about technology, it’s about management vision. The organization — and its leaders — need a forward-thinking, entrepreneurial culture that capitalizes on this computing power. Organizations that are restrictive, risk-averse and hierarchical are likely to find themselves overwhelmed and flying blind, forever fighting an endless battle to gain control over fast-changing technology forces." – source: https://cdn.ampproject.org/c/www.forbes.com/sites/joemckendrick/2017/01/22/cloud-computing-spins-out-of-control-but-maybe-thats-not-such-a-bad-thing/amp/
The mantra above is key to how the Agile mindset can really assist companies in avoiding their pre-existing controlling nature. Instead it can allow the more forward thinking entrepreneurial culture to develop. It’s this culture of entrepreneurship which I truly believe is required to deliver impactful change via Cloud Adoption. A lot of goals the "C Level" aim for when they decide to adopt the cloud are the same as those the Agile mindset helps to achieve:
- Enabling Innovation
- Organisations moving away from control mindset
- Learning to fail fast and gain organisational learning which is critical for the advancement of technology
- Increased Efficiencies
- Smaller teams with more autonomy and less management
- Quicker Speed of Delivery
- Change in processes
- Iterative development
- Value Realisation
- Delivering value earlier in the development chain
I could go on and on, but all of the above benefits are true with both cloud and Agile and, ultimately, in this context they clearly have strong symbiosis.
Agile & Cloud from a Technical Perspective
I have explained why Agile and the cloud work well together from both an organisational perspective and a change perspective and, lastly, I want to take you through a technical perspective. Cloud lends itself well to an Agile approach because of one major concept in the industry – "Infrastructure as Code" – which allows cloud infrastructure deployment to be done in a similar way to software delivery, a way that was impossible for traditional infrastructure.
What is infrastructure as code?
Definition: Infrastructure as Code (IaC) is the process of managing and provisioning computing infrastructure via code, rather than physical hardware configuration or the use of interactive configuration tools. This is around the automation of deployments of infrastructure which has been made possible with the growth of Cloud Computing. (https://www.thoughtworks.com/insights/blog/infrastructure-code-reason-smile)
As Infrastructure is now being managed more and more via code like software has in the past, moving and adopting Agile Frameworks makes sense for cloud infrastructure delivery. This is why in the industry we see Agile becoming more and more preferred. The main blockers to this at the moment are organisational rather than technical.
So, as highlighted above, cloud and software development have many similarities. They both:
- Use Programming Languages (python, ruby, json, etc.)
- Use Code Management
- Follow Continuous Integration and Deployment (CI/CD)
- Require Testing (Regression, UAT, Integration)
However there are some differences:
- In the Cloud you are responsible for the environments and the underlying infrastructure of the code. E.g. Having to use code to create and manage your staging environment that other code sits on. This is not the case with software which depends on infrastructure being there.
- Infrastructure tends to be very logical and less tangible than software (i.e. you can’t see it) for example Firewall config just ports and IP address or Access Controls with error messages.
These differences can create challenges in my experience when following an Agile methodology:
- Demos pose a challenge due to the less tangible nature of cloud services, which I have solved in the past by trying to keep them focused on the user and business benefit
- I have also had to split user stories into environments first then use code management to assist with the big picture, this effectively creates a matrix of user stories adding some complexity, but also helps with clarity if done correctly
- Although 95% of cloud has API access and can be done via code some elements are manual which cannot be automated or follow software development practices
So, in conclusion, if I was an enterprise working on any cloud project, I really would recommend an Agile delivery method. It is going to help team’s align with the need for quicker delivery and higher quality of the inputs into the cloud. Additionally it helps organisations get greater value from their project and truly iterate towards the goal they are achieving so that they can recognise some of that value sooner.
I hope that this first blog in the series will help you equally fall in love with Agile in large enterprises. And if you have any questions – reach out! I’d love to hear your thoughts or feedback and help you become a true believer like I have.