As an AWS Solutions Architect working for at an APN partner, I frequently review workloads and customer environments who are still very new to AWS, and one thing I’ve found frequently is a lack of CloudWatch alarms, with most infrastructure also being implemented directly in the console and not through Infrastructure-as-Code.
A more cloud mature organization would have these alarms defined in IAC from the beginning, but when dealing with customers with hundreds of EC2 instances and other resources that have been created in the console, I was looking for a quick way to create standard CloudWatch alarms. With my Windows admin background, PowerShell was a natural choice, also helped by how fantastic the AWS PowerShell tools are. My first iteration wrote directly to CloudWatch alarms by looping through a variable:
This worked well, but there are a few problems…
First, this only adds new alarms, it doesn’t have logic for removing unused alarms. Second, we should really be using Infrastructure-as-code to detect drift and make sure these alarms are not changed in the console. Third, we would have to run this frequently to keep alarms updated, which introduces another manual task (or using AWS Lambda, which has excellent PowerShell support!)
To resolve these issues, I decided to re-write the script to instead write CloudFormation templates, instead of writing alarms directly. The second iteration instead writes a code block through a loop to a CloudFormation Template
This is all tied together by a larger function that calls each individual function:
After a bit of debugging & fixing the template, I’m happy with the results! Deploying with CloudFormation has some nice benefits, including being able to detect drift, customize the template further for outliers (which my original script did not handle well at all) and check our generated template into source control.
I think there is potential in this method to much more! My immediate plan is to write more standard alarm generators for other services and custom metrics, but I think long term this could be an interesting solution to generate CloudFormation templates for existing resources within AWS (similar to Former2) without having to provide access keys to a 3rd party service.
So, what do you think? This code is all in my GitHub, feel free to fork it or contribute!