Metric alerts for Azure monitor logs

A common thing for traditional companies is to have one team responsible for monitoring. A few years ago, this team where close friends with the team provisioning infrastructure. Now, more and more companies are shifting to the “DevOps” world. Even Microsoft have killed SCOM and are only using Azure Monitor. Meaning that the one deployed the code (and the infrastructure) should be responsible for monitoring.  In essence, this is great. But this transition takes time, and one should not underestimate the knowledge of the team who have been responsible for monitoring your entire infrastructure for decades.

If you are familiar with SCOM, you know that rules and monitors is targeted against a class of objects. IE, Windows 2016 operating system. When we move our workloads to Azure, we want to use Azure Monitor to monitor our workloads and VMs.

Enter Log alerts

Log Alerts has been around for quite some time and is commonly used to alert on actual log data. IE custom application logs, Windows event log and so on. But Log alerts has a “hidden” feature, especially for your monitoring teams, not wanting to manage hundreds of duplicate rules.

By using Log alerts with metric measurements you can almost replicate the what discoveries in SCOM does- find resources of a specific type, and attach some kind of monitoring to them. For example, you can create a search query for all your IaaS VMs and alert on their CPU counter.

This will let your monitoring team recreate all their logic, and have control over the entire infrastructure, almost as they had on-permises. At the same time you can leverage more DevOps practices and at the end have every team responsible for their own work.

Kusto examlpe

Below is a simple example that will list all VMs and their processor time. You can create an alert straight from Azure Monitor logs (former Log Analytics) or start from a new alert.

Perf | where ObjectName == "Processor" and CounterName == "% Processor Time" | summarize AggregatedValue = avg(CounterValue) by bin(TimeGenerated, 5m), Computer

Summary

You have the option to monitor multiple VMs using one Alert Rule in Azure Monitor already. But one limitation is that this solution will not add new VMs to the alert rule. And for the time being, it only supports virtual machines
Log alerts are dependent on your query. So as long as your data is available, you can alert on it. Whether it is a web app, a SQL server or a custom log.

With Log Alerts, the transition to a public cloud-based infrastructure might be easier. Your operations teams can use their knowledge and re-create their on-premises monitoring logic as searches.
Application alerts could still be handled by the developers, and you can provision those using ARM templates or similar.

PS: I was going to write a longer post on how to manage and programmatically create log alerts, but with these great examples in Microsoft docs, there’s no need to re-invent the wheel.