monitoring Archives

Working with Azure Monitor Rest API

29/02/202003/09/2020
by Martin Ehrnst

It might be an edge case, or you are creating your own integration. But chances are, after fiddling around with Azure Monitor, you encounter a situation where you would have work with its API.
Personally, I have numerous situations over the last years, that required me to integrate directly with Azure Management APIs

In this blog post, I will help you get started using the APIs, hopefully making it less intimidating. All code examples will use PowerShell.

Create SPN and assign it to your subscription RBAC

I am using a service principal in order to connect programmatically to Azure. SPNs are created in Azure Active Directory as Application Registrations. You can choose whether you want to do this via the portal, or by using PowerShell.

After creating (or using an existing SPN) grant the application appropriate access. For simplicity, I am using contributor to my subscription. That might be best for you as well, but one should always use the least privileged assignment needed.

Consider the above as a prerequisite for you to continue.

At this point, you should have an application registration, a secret, and a role assignment on your subscription. We can now use this to acquire an access token and connect to Azure Monitor’s REST API.

Connect to Azure Monitor API using PowerShell

Azure Monitor APIs are a part of the Azure Management APIs. I will, therefore, use these names interchangeably. Also keep in mind, that all other APIs under Azure Management will follow the same methods I demonstrate for Azure Monitor.

To query data we need to authenticate. In the example below, I am using client credentials to acquire the access token. Microsoft’s official example is using the ADAL method, connecting with your identity. I have never had the use for this, as I am usually writing integrations service-to-service.
If you are creating an interactive portal and want to leverage the user’s authorization, ADAL (or MSAL) are probably better.

Retrieve Azure Monitor alert rules

I have no idea why you are exploring Azure Monitors API. Providing an integration solution is therefore not possible. But my gut feeling is that alerts and metrics is a good place to start.

When working with alerts, we need to work with multiple endpoints. Depending on what you are working on, these are the most common;

Alert rules*
Alert incidents
Metric Alerts
Metric Alert status

We can start with one of the basics, retrieve the current configured alert rules. To do that, we need to know what kind it is. The classic alert rules (old type) use a single endpoint, while the current use it’s own.

Below I have included three endpoints and a screenshot. As you can see, all the information that you expect is to be found in the output. From here we can start to explore the alert rule by accessing its properties.

Azure monitor rest api metric alert rule output

Get resource metrics from Azure Monitors API

Metrics is another fundamental in monitoring. When we work with the API in the context of metrics. You can explore the available metrics for each resource type by using the Metric Definitions endpoint.

Actual metrics values require a bit more when it comes to the actual query. The official documentation describes everything pretty well, but I have provided an example for a VM below. This example shows the basics of how you get data from one metric and one VM. You can add multiple metrics to one query, and do additional filtering using the OData filter.

Manage alerts, updating status, etc.

Viewing configured alert rules, looking at disk metrics for a VM. What about alerts. The actual things that send you emails- can we work with them using this API? Yes, you can.

Like I said. Providing an integration solution in this blog post isn’t possible. but most integrations I have seen with Azure Monitor or other monitoring solutions have had some kind of functionality to handle active alerts. Personally, I have created one for SquaredUp earlier, where we could acknowledge alerts in Azure Monitor as well as our on-premises SCOM installation.

Before we wrap up. Let’s take a look at how we can interact with an active alert. I have configured a very naggy alert rule, creating a lot of noise, and I want to change the status of those alerts. Armed with PowerShell and the alerts management endpoints everything is possible.

Summary

This blog post has covered the basics regarding the Azure Monitor REST API and PowerShell. With the examples above and the official documentation, you can start creating your own solutions and integrations.

While we have only covered how to get data out of Azure Monitor, you should know it’s also possible to inject data. By using the HTTP data collector API and the Metric store possibilities are ‘endless’.

Integrations ideas

Alert remediation/handling from a ticketing system
Dashboarding with third-party or custom web integration
Teams/Slack/IM connector
Custom application metrics or logs

In my examples, I have purposely not included how new alert rules are created, as I believe this should be done through ARM. If that is your use case, you should know it is possible and fully supported.

This blog post was originally published in November 2017. Rewritten for Azure Spring Clean 2020 and to reflect changes to Azure Monitor API

Azure

Resource health through Azure Rest API

18/09/201725/11/2019
by Martin Ehrnst

As a part of a large monitoring project involving on prem, Azure and Office 365 I have started to explore the different methods on how to acquire all relevent data. Previously, I have written a post on how you authenticate against Microsoft’s partner center API which is a part of the same feasibility project.
Later down the road i will try to write a larger blog series on how we can monitor Microsoft Cloud and on-Prem resources. Hopefully it will be joyful…

This blog post will describe how you can use Powershell to authenticate and get availability status from all resource groups and their resources. If you’re not that interested in monitoring data, use this post as a guide on how to get started with the API and the rest is documented on the Azure API documentation pages.

High level overview:

Set up an Azure Active Directory Application to authenticate (not covered)
Build an authentication header with a token from Azure AD
Get all resource groups within a subscription
Get the availability of all resources within a resource group

Get Azure AD application token

After setting up/registering the application in Azure AD you will have to use the application ID and secret in order to generate an authentication token to use against Azure management Rest API’s. I have created a basic Powershell function you can use, including an example authentication header.

$result = Get-AADAppoAuthToken –ClientID <AzureAD APPLICATION ID> –ClientSecret <ClientSecret> –TenantId "test.no"
$AuthKey = "Bearer " + ($result.access_token)
$authHeader = @{
 'Content-Type' = 'application/json'
 'Accept' = 'application/json'
 'Authorization' = $AuthKey
}

view raw
createAuthHeaderExample.ps1
hosted with ❤ by GitHub

 <#
 .SYNOPSIS
 Function to connect to the Microsoft login OAuth endpoint and return an OAuth token.
 .DESCRIPTION
 Generate Azure AD oauth token.
 You can specify the resource you want in the paramenter. Default is management.core.windows.net
 Parts of this function is created from these examples: https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-rest-api-walkthrough
 .PARAMETER ClientID
 Azure AD application ID
 .PARAMETER ClientSecret
 Your application secret.
 .PARAMETER TenantId
 Your tenant domain name. test.onmicrosoft.com
 .PARAMETER ResourceName
 Specify if you are accessing other resources than https://management.core.windows.net
 For example microsoft partner center would have https://api.partnercenter.microsoft.com
 .EXAMPLE
 Get-AADAppoAuthToken -ClientID 'xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx' -ClientSecret <application secret> -TenantId "test.no" will return
 token_type     : Bearer
 expires_in     : 3600
 ext_expires_in : 0
 expires_on     : 1505133623
 not_before     : 1505129723
 resource       : https://management.core.windows.net/
 access_token   : eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkhIQnlLVS0wRHFBcU1aaDZaRlBkMlZXYU90ZyIsImtpZCI6IkhIQnlLVS0wRHFBcU1aaDZaRlB
 kMlZXYU90ZyJ9.eyJhdWQiOiJodHRwczovL21hbmFnZW1lbnQuY29yZS53aW5kb3dzLm5ldC8iLCJpc3MiOiJodHRwczovL3N0cy
 
 .NOTES
 v1.0
 Martin Ehrnst 2017
 #>
 [Cmdletbinding()]
 Param(
 [Parameter(Mandatory = $true)]
 [string]$ClientID,
 [Parameter(Mandatory = $true)]
 [string]$ClientSecret,
 [Parameter(Mandatory = $true)]
 [string]$TenantId,
 [Parameter(Mandatory = $false)]
 [string]$ResourceName = "https://management.core.windows.net/"
 )
 $LoginURL = 'https://login.windows.net'
 #Get application access token
 $Body = @{
 grant_type = "client_credentials";
 resource = $ResourceName;
 client_id = $ClientID;
 client_secret = $ClientSecret
 }
 Return Invoke-RestMethod –Method Post –Uri $LoginURL/$TenantId/oauth2/token –Body $Body
}

view raw
Get-AADAppoAuthToken.ps1
hosted with ❤ by GitHub

Get all resource groups and resources

Next we will grab all resource groups within a subscription, before looping through each resource group to get the individual resources.

#Loop through each reasource group and get all resources.
#Add everything to a hash table
$Groups = @()
foreach ($rg in $ResourceGroups) {
 $ResourceGroupUri = "https://management.azure.com/subscriptions/$subscriptionID/resourceGroups/$rg/resources?api-version=$APIVersion"
 $res = (Invoke-RestMethod –Uri $ResourceGroupUri –Method GET –Headers $authHeader).value
 #Create array of all resources
 $resources = @{}
 $resources.Add($rg, $res)
 #Add all resource groups and their resources to a hash table
 $Groups += $resources
}

view raw
GetAllResorurcesWithinRG.ps1
hosted with ❤ by GitHub

#get all resource groups within a subscription
$APIVersion = "2017-05-10"
$subscriptionID = "xxxxxxxxxxxx-xxx-xxx-xxxxxxxxxx"
$RGURI = "https://management.azure.com/subscriptions/$subscriptionID/resourcegroups?api-version=$APIVersion"
$ResourceGroups = (Invoke-RestMethod –Uri $RGuri –Method GET –Headers $authHeader).value.name

view raw
GetAzureRG.ps1
hosted with ❤ by GitHub

Now that we have all resource group and all resources within we can use this to further get the resource health and availability. To be honest we don’t need to get RG before checking the resources it self, but as a starting point i find it very useful to have the resource groups and their resources available within my scripts.

Get the resource availability status

Now we have authenticated, grabbed all resource groups and their resources and it’s time to find our resource health. There are multiple ways on how to get this data, and we have to consider the methods we use. One limitation i hit pretty fast working with this is the number of subscription resource get requests, specified in the response header “x-ms-ratelimit-remaining-subscription-resource-requests: ” All limitations documentet, here

When you hit this limit every substantial requests is dropped. I have to rethink my whole monitoring scenario due to these limitations.

Get Resource health based on resource group

This let’s us get health state on all resources scoped to a resource group. If we reuse the data from previously (we have all resource groups within our subscription) we will get all resource group health this way.

availabilityState : Unavailable
summary           : Your virtual machine is unavailable
detailedStatus    : We're working to automatically recover your virtual machine and to determine the source of the problem. No additional action is required from you at this time.
reasonType        : Unplanned
occuredTime       : 2017-07-30T01:13:56Z
reasonChronicity  : Persistent
reportedTime      : 2017-09-12T11:27:42.3921293Z
resolutionETA     : 2017-07-30T01:38:56Z

view raw
example-output
hosted with ❤ by GitHub

#get the health of the whole resource group
# Add each health status to a hashtable before output a complete table with all resource groups and their resource health
$resourceGroupHealth = @{}
foreach ($ResourceGroup in $ResourceGroups) {
 #Set resource group name and use it in our url
 $health = Invoke-RestMethod –Uri "https://management.azure.com/subscriptions/$subscriptionID/resourceGroups/$ResourceGroup/Providers/Microsoft.ResourceHealth/availabilityStatuses?api-version=2015-01-01" –Method GET –Headers $authHeader
 $currentHealth = @{}
 $currentHealth = @{
 [string]"$ResourceGroup" = [object]$health
 }
 $resourceGroupHealth += $currentHealth
}
$resourceGroupHealth
#Explore the results
$resourceGroupHealth.item('ResourceGroup').Value.Properties

view raw
GetResourceGroupHealth.ps1
hosted with ❤ by GitHub

Other examples are to get availability by Resource and the entire subscription