Creating Cloudwatch alarms over Metric Filters

A photo from freepik.com

CloudWatch collects monitoring and operational data in the form of logs, metrics, and events, providing you with a unified view of AWS resources, applications, and services that run on AWS and on-premises servers. You can use CloudWatch to detect anomalous behavior in your environments, set alarms, visualize logs and metrics side by side, take automated actions, troubleshoot issues, and discover insights to keep your applications
running smoothly.

There are a lot of metrics that can be used in Cloudwatch alarms by default. For example, I can get an insight into Lambda’s duration, errors, throttles, invocations, and concurrent execution by default. But I needed to track memory usage in case it reaches the limit. By default, it triggers invocation error when memory reaches the limit, but there are thousands of invocations happening in a short period of time, so it was really tough to find the exact error cause.

Metric Filters come in handy in my case. You can search and filter the log data coming to CloudWatch Logs by creating one or more metric filters.

To compare memory allocation and usage, I needed to create two metric filters by using my lambda’s logs. The first one is for allocated memory and the other one is for memory usage.

To create metrics filters;

1. Go Cloudwatch dashboard.
2. From the left menu, navigate Log groups and find your log group for your lambda function.
3. In the Log group, click the Metric Filters tab below and create a new metric filter.

Next, on the creation screen;

1. The Filter pattern refers to the search query of the metric filter. Basically, what you wrote there is used for filtering the logs. In my case, I needed to extract the Memory Usage string from many verbal logs. To achieve that I used […] filter to convert logs, into regex matches.
2. In the Test pattern section, I tested my filter pattern if it works or not. So the $18 regex group is exactly what I need. Noted down, because I will use it on the next page. After seeing it successfully working, click next.

And we reached the last page of metric filter creation where we put the details for our custom metric.

1. Filter name, at the top, is used for searching your metric when adding an alarm, etc. In my case, I’ve written memory-usage.

In the Metric details section;

1. Metric namespace; This is used for grouping up your metrics in Cloudwatch. So, when you track similar metrics, you can simply write the common name. So, I’ve written <lambda_name>MemoryMetrics. It’s again, will appear when you search for Cloudwatch metrics. Setting proper and unique naming will be helpful.
2. Metric name; A unique name specifically for this metric filter. I set MemoryUsage as a metric name. It must be unique in your Metric namespace. For another metric, you can’t use MemoryUsage naming again.
3. Metric value; is the most critical one because our metrics will be based on this value. In the section above, we’ve saved our filter pattern’s regex group match, which is $18 for memory usage. So, I’ve written $18 to this input.
4. Default value (optional); Is the value published when the filter pattern does not match anything. I’ve left it empty.
5. Unit (optional); is the unit that is gonna be used for your metric. I leave it empty since I will use the value of my filter match.
6. Dimensions: Identity of your metric. A name/value pair. A filter metric can have up to 3 dimensions. Used for searching and grouping purposes of your metric filter. I’ve left it empty since I search my filter by the name of it.
7. Click next, review and save your alarm.

Now we have created our metric filter successfully. To compare it with another metric, I followed the same steps for the Memory Allocation metric. The regex match value for Memory Allocation was $13.

Now we can add alarms by using this filter. So we go and create a new alarm on Cloudwatch by searching this filter;

Search result of our filter metric in Cloudwatch metrics section.

The comparison graph of MemoryAllocation and MemoryUsage metrics.

That's it! The rest of it is basically the alarm configurations. The metric filters can be used for filtering many log outputs. You can simply log anything you want and create metrics over it. There are other options like filtering JSON outputs or text matches, etc.

Hope you enjoyed it, stay healthy! ✊

Feel free to contact me on linkedin.