How we get dynamic, real-time visibility into your cloud estate
As a part of the Sunstone team at Cloudreach, I get to work with the latest cloud technologies to make our application scalable, performant, reliable and cost-effective. We are a small team of “owner/operator” engineers, which gives us a unique set of challenges.
- We do not have a dedicated operations team, meaning we want to offload as much operational responsibility as we can.
- We need to keep up with and understand new features from the Cloud Service Providers (CSPs) so Sunstone can make the best recommendations.
- Finally, we need to stay flexible to allow us to make use of those new cloud services ourselves!
To this end, we use serverless or fully managed services wherever possible.
Sunstone’s architecture can be split up into four steps:
- Data collection (getting the details of a customer’s cloud estate)
- Data storage (storing that data we’ve collected)
- Analysis (crunching the data we’ve got and making recommendations for modernization)
- And finally, presentation
In each of these steps we make use of different managed or serverless services.
Sunstone data collection
Our data collection and analysis components both make heavy use of Functions as a Service (FaaS) and managed message queues. Where in more traditional architectures you may have an application running on virtual machines in a group which can scale as demand increases, with FaaS we can give the CSP a bundle of our source code and have them manage the environment it runs in. We can then configure the service to allow for many parallel executions of this code, giving us massive scalability for very little effort. We don’t need to wait for a new virtual machine to boot up to scale – the CSP deals with that for us!
Queues make parallel execution easy
Queues are a big part of how this is orchestrated. We can publish messages to a queue, which will then trigger the functions to carry out work. The functions can even publish back on to the queue itself if need be, allowing for work to fan out easily so we can make the most of the massive amount of parallelism available to us. In a less cloud-native architecture you may run something like RabbitMQ to meet this requirement. For us, this would mean running virtual machines, managing networking and having to worry about standard operational issues such as updates and patching. Instead, we’ve made use of CSP-provided services that are tightly integrated with the underlying platform and allow us to quickly get on with the job of using the queues.
Sunstone data storage
From data collection and analysis we move on to data storage. Sunstone makes use of three types of data stores:
- Graph database
- Document database
- Object storage
All of these requirements can be fulfilled using managed CSP services. In each of these cases, the services are tightly integrated with the rest of the cloud provider’s ecosystem. In a less cloud-native system; we could run services such as Neo4j or MongoDB ourselves, worry about backups, patching and updates, and also manage authentication and authorization. By utilizing managed offerings, we offload backups, patching and updates to the cloud provider. Authentication and authorization is integrated with the rest of the CSP’s Identity and Access Management (IaM) system.
Sunstone presentation
To wrap up our architecture, we have our frontend and the API to which it communicates. We use a standard pattern of a single-page application written in javascript and an API backend written in python which is deployed as a container. To go back to a non cloud-native architecture, this could be split into a cluster of servers running Apache or Nginx as web servers for the javascript, then another cluster of VMs or even a Kubernetes cluster to run the backend. That’s a lot of moving parts!
In our architecture, the frontend can be handled by the object storage service provided by our CSP and their Content Delivery Network (CDN) service to ensure that users get the best experience. The backend container can be handed over to the CSP’s managed container offering. They’ll deal with the task of keeping it running and scaling it based on rules that we can define.
The cloud-native payoff
From an engineer’s point of view, using these services makes development really straightforward. We can focus on writing application code and even configure the deployments of our work with Infrastructure as Code (IaC). This close coupling means we can really focus on what matters to us: making effective features. Our monitoring dashboards can change focus from being a tool for operations teams to keep an eye on signals such as CPU or memory load to be more specific to our application. Engineers can have a greater sense of ownership over the whole application; it’s all in code the team has written, there is no scary operations stuff that some other team is looking after.
Our management are fans too! We’re able to quickly react to changes in what the business needs from us.
- New customers? Sure; we don’t need to spin up new infrastructure to support them – the system will just handle that!
- Cost management? When the system is quiet it scales to nothing, meaning the application is only costing money when it’s doing useful work.
- New features? We can quickly build and deploy proof of concept implementations of potential features without having to wait for infrastructure to be provisioned for us.
Identify modernization targets with Sunstone
Our cloud-native architecture really is integral to how Sunstone runs and how effective it is. As Sunstone develops we will recommend where similar improvements could be made to your cloud estate, allowing your organisation to get similar advantages and the best out of the cloud.
Learn more about Sunstone.’
