Automation Archives

Serverless application with PowerShell: Azure Event Grid

30/05/201829/05/2018
by Martin Ehrnst

This is part one of my blog series describing how you can create a serverless application infrastructure using PowerShell. The scope and history behind, covered in the introduction post. If you haven’t read that I recommend doing so.

Azure Event Grid

In my serverless infrastructure design, I have chosen to use Azure Event Grid as a hub for the other components in play. The other components will write and subscribe to events/messages in Event Grid and perform their given task as ‘their’ event arrives. Event grid fit’s the concept perfectly as you act on state changes. You can publish as many events you like, but you don’t need to subscribe to every one. A lot of Azures PaaS offerings are already fully integrated which makes it easy to get started.

Other event based offerings in Azure are Service Bus and Event Hub. You can read a little on the differences here

Set up Event Grid custom topic

I am sure you are able to create resources in Azure so here’s a quick one. Feel free to use what ever you want for deployment, but for the purpose of screenshots, I’m using the portal. Just remember that the name you chose will be the url of your endpoint as well.

After the custom topic is successfully deployed, you are given the option to subscribe to events. Before doing that we are taking a break. Besides a graph showing incoming events you have no good way to see your events as they arrive. This makes it way more difficult to debug why stuff isn’t working. While writing this post I researched for a good way to solve this and stumbled across this blog post by David Barkol. He created a web app that lets you view events as they arrive. You can deploy it directly from GitHub using ARM. Do this while your coffee is brewing. And come back here in 10 minutes 🙂

Subscribe to Event Grid topic

Welcome back… If you have deployed the web app you can follow and create a subscription subscribing to all events. In a production scenario, it’s likely to have scoped your subscribers, but in order for Event grid viewer to function i’ll subscribe to everything.

Once deployed your overview page will look something like this. Showing all subscriptions within your topic and their corresponding metrics. Now lets try to send a custom event using powershell.

Send event to event grid using PowerShell

Basically, event grid accepts post request to an API endpoint, which is the name of your topic plus a suffix and the API version. your full URI looks something like this; https://adatumblogdemo.northeurope-1.eventgrid.azure.net/api/events?api-version=2018-01-01

To authenticate we will pass an ‘aeg-sas-key’ within our request header. You will find the keys for authentication in the keys pane. Further event grid events has a schema you willl need to follow, meaning that all post requests will need to have some properties defined and your custom event data will be in a data property. The basics of it all looks like this

[ { "topic": string, "subject": string, "id": string, "eventType": string, "eventTime": string, "data":{ "custom": "data" }, "dataVersion": string, "metadataVersion": string } ]

More details on the schema can be found on Msft Docs

If we combine everything we know this far, we can use the script below to post messages to event grid. Please add your own url and access key. Everything else should work.

Assuming no error messages, jump in to your event grid viewer to view the raw content like this;

A corresponding graph in the Azure portal could look something like this. Here you will see incoming events and how many matches your subscriptions in the same graph. You can also see individual metrics per subscription.

Summary

In this part of my blog series on how to create a serverless application infrastructure using PowerShell. We have walked through how to set up a custom Event Grid topic and how to post messages or event to it using standard PowerShell commandlets. We also deployed a custom web application capable of reading our incoming events in real time.

In the next part: For part two we are going to create an Azure Function running PowertShell that subscribes and send Event Grid messages. This is key to build up our infrastructure wich consists of multiple Azure Functions in order to work.

Automation

Creating a serverless application with Powershell: Introduction

08/05/201807/08/2018
by Martin Ehrnst

Welcome, fellow IT pro. You have found the blog series describing how you can create a serverless application infrastructure with Powershell.

If you consider your self being a ‘modern it pro’, you frequently solve complex tasks and automate your work using code. You know what Git is and have heard people talk about continuous integration. If you can relate, let’s label you a ‘modern it pro’.

When I develop solutions that challenge my knowledge, I often seek advice from our developers. (You know, those who are employed to ‘GIT commit’?) Usually they can share some light on the issue and point me in the right direction.
By developing our own solutions, we have a working proof-of-concept to handover to our developers when things grow too big or complex.

The original challenge

The challenge or scope for my POC was to streamline creation of Azure tenants for our customers. We required to create a tenant, assign subscriptions and do some configuration within Azure AD. Many of these steps was already solved with various scripts, but it is time to consolidate and automate the automation. We decided that we wanted to learn more around the integration and ‘serverless’ PAAS offerings available in Azure.
For the purpose of this blog series we are going to build a new application using the same techniques, in a smaller scale. To create our serverless application infrastructure, we make use of the following Azure offerings:

Pizza ordering

I bet you have worked a fair amount of overtime in your career, therefore eaten a lot of pizza as well. Let’s try to build a simple pizza order and delivery process using Event Grid as integration layer and multiple Azure Functions to process the order.

We will break the pizza order and delivery process in to the following steps:

Customer creates/orders a pizza online
Send orderdata to event grid
Azure function subscribe to the “new order” event
Create a new event when pizza is in oven
Post new event when pizza is cooked and ready for delivery
Create a new event when pizza is sent to customer.

Parts in this blog series

Automation

Creating Azure AD Application using Powershell

10/04/201809/04/2018
by Martin Ehrnst

Azure AD Application

Azure AD have something called Application registrations. These are often used to integrate with external services and can provide functionality like Single Sign On to your companies Twitter account. There’s a large selection of applications you can chose from in the Azure Portal, but this post will cover how to create your own application registration using Powershell.

In this scenario we are creating an app that can access Azure Activity Logs, used by our on premise Splunk environment. Since I am doing this across 300 tenants the manual approach isn’t feasible.

High level overview

Create the app using Powershell
Assign the required API access to the new app
Create access key
Create new Azure AD Service Principal for our app (SPN)
Assign ‘Reader’ role to subscription

Create the app using Powershell

This is the easiest part. Azure Powershell have a pretty simple Cmdlet that let’s you create a new application, New-AzureADApplication. The required steps is to Import AzureRM modules and AzureAD modules. After that, connect to Azure AD using

Connect-AzureAD -Credential -TenantId "domain.onmicrosoft.com"

Now you can run New-AzureAdApplication to create a new app, this example shows the required fields.

New-AzureADApplication -DisplayName "Adatum App Demo" -IdentifierUris "https://localhost/AdatumAppDemo" -HomePage "https://localhost/Adatum"

and in returnObjectId AppId DisplayName -------- ----- ----------- 2cd0a284-7b9e-4 34ecfd2a-8f78-38c4a8b0 Adatum App Demo

In the Azure portal we can see our new app registration, but it does not have a service principal, and no API access. If you would have gone through the steps creating the app in the portal it self SPN and a “read basic profile” API permission would be added to your app by default.

Assign required API access

This is where I spent the most time. In order to assign permissions to our Azure AD Application we will need to write a bit of code. Applications can be assigned Application Permissions and Delegated permissions. According to the documentation New-AzureAdApplication Cmdlet takes a parameter “RequiredResourceAccess”. This again wan’t something of ‘Microsoft.Open.AzureAD.Model.RequiredResourceAccess’ as a generic list. Using my favourite search engine I found that I also needed to construct this object and add multiple objects (the permissions) to it.

Luckily my friend Jan Vidar Elven (MVP) had an example. A sligthly different approach, but here’s what I did. My app needed access to Azure Service Management API, manually I added this via the portal and explored the JSON manifest to see what GUIDs was added. Later I found a better solution, exploring the Service Principal (API’s) using Get-AzureADServicePrincipal.

Get-AzureADServicePrincipal -All $true | Where-Object {$_.DisplayName -eq "Windows Azure Service Management API"}

which will return something like this.

ObjectId AppId DisplayName -------- ----- ----------- 28a1249a-c5bb-4c7b-b94d-064ca5bb1952 797f4846-ba00-4fd7-ba43-dac1f8f63013 Windows Azure Service Management API

Then I could start to build my permission object

## Azure Management API $AzureMgmtPrincipal = Get-AzureADServicePrincipal -All $true | Where-Object {$_.DisplayName -eq "Windows Azure Service Management API"} $AzureMgmtAccess = New-Object -TypeName "Microsoft.Open.AzureAD.Model.RequiredResourceAccess" $AzureMgmtAccess.ResourceAppId = $AzureMgmtPrincipal.AppId

If you had used this object with the ‘RequiredResourceAccess’ parameter now, the app would have assigned Windows Azure Service Management API, but no permissions. Just to be clear, this is the permissions I am talking about

You can explore the same permissions using
$AzureMgmtPrincipal.AppRoles for application permissions and $AzureMgmtPrincipal.Oauth2Permissions for delegated permissions. Azure Management only have one delegated permission

PS C:\> $AzureMgmtPrincipal.Oauth2Permissions
AdminConsentDescription : Allows the application to access the Azure Management Service API acting as users in the organization. AdminConsentDisplayName : Access Azure Service Management as organization users (preview) Id : 41094075-9dad-400e-a0bd-54e686782033 IsEnabled : True Type : User UserConsentDescription : Allows the application to access Azure Service Management as you. UserConsentDisplayName : Access Azure Service Management as you (preview) Value : user_impersonation

Who doesen’t love GUIDs?

Now add this permission to a new ‘microsoft.open.azuread.model.resourceAccess’ object

$AzureSvcMgmt = New-Object -TypeName "microsoft.open.azuread.model.resourceAccess" -ArgumentList "41094075-9dad-400e-a0bd-54e686782033", "Scope" before you add that permission object to the ‘resource access object’ created earlier.

$AzureMgmtAccess.ResourceAccess = $AzureSvcMgmt

If you where adding multiple permissions, repeat the above. After you completed this step, our first object $AzureMgmtAccess have an app ID assigned (Azure management api) with the corresponding Resource access object(s)
ResourceAppId ResourceAccess ------------- -------------- 797f4846-ba00-4fd7-ba43-dac1f8f63013 {class ResourceAccess {...

Now you can re-run our first command but add the permission parameter (delete the app created earlier or change the identifier)

New-AzureADApplication -DisplayName "Adatum App Demo" -IdentifierUris "https://localhost/AdatumAppDemo" -HomePage "https://localhost/Adatum" RequiredResourceAccess @($AzureMgmtAccess)

Check your portal now 🙂

Application Access Key

We have to create an access key for us to programmatically logging in via the application. Depending on your scenario, you might want to create multiple keys, but in this example I am sticking with one. Just like in the Azure portal you will specify the validity time for your key. The example below will add a key wich expire in one year.

New-AzureADApplicationPasswordCredential -ObjectId $AdatumDemoApp.ObjectId -CustomKeyIdentifier "Access Key" -EndDate (get-date).AddYears(1)

If you want the key to not be activated immediately, set the appropriate date time by using StartDate parameter.

Create new Azure AD service principal

For our application to access within our tenant, we need to assign a new service principal. The security principle will allow us to access the subscription (or other resources for that matter.) You can read more about security principals for users and services here, Application and service principal objects in Azure Active Directory (Azure AD)

To create the service principal in via Powershell, run the following: New-AzureADServicePrincipal -AppId -Tags @("WindowsAzureActiveDirectoryIntegratedApp")
Pay attention to that tag. It is required if you want the app to show under app integrations in the Azure AD portal view. Documented here

RBAC / IAM subscription access

The app I am creating is used to read Azure activity logs. To be allowed to read these logs, the app must be assigned ‘Read’ permission on the subscription level. To assign a new role, use the New-AzureRmRoleAssignment cmdlet, using ObjectID, RoleDefinitionName and Scope as input parameters.

New-AzureRmRoleAssignment -ObjectId 'SPN objectId' -RoleDefinitionName "Reader" -Scope "/subscriptions/$SubScriptionId"

PS: When you put the complete script together, you will find that the SPN is create async in Azure Resource Manager. Meaning that you will get the output, but the principal isn’t created. To get around this in our provisioning I created a while loop. Not the prettiest you will find, but works for now.

while ($addRole.DisplayName -ne $AzureSplunkAdApp.DisplayName ) { Start-Sleep -Seconds 5 Write-Verbose "Waiting for SPN to create" $addRole = New-AzureRmRoleAssignment -ObjectId $AzureSplunkAdAppAppSPN.ObjectId -RoleDefinitionName "Reader" -Scope "/subscriptions/$SubScriptionId" }

Final script

I have put together a complete script which will create this demo application. No error handling and your risk 🙂

Automation

Activate Webasto diesel heater using OMS and Azure functions

03/10/2017
by Martin Ehrnst

Looking at the above picture snow and winter is pretty awesome, but when you have to fire up your car outside your cabin in -15c (5f) you wish for summer. Not only is it extremely hard for your engine, but it’s also ridiculously cold to get inside the car! That’s why many cars in Norway are equipped with preheaters. A unit you can activate before starting or entering the car, heating up the engine (and in many cases the interior as well).
Just to put some numbers down, average temperature in Oslo January 2017 was -4.3, and coldest -12.6.

My child carrier, a Range Rover Sport, (one of Britains fines engineering that never fails) is equipped with a Webasto pre heater and controlled remotely by SMS command or an app. Communication is using WBUS, Webasto’s own language.

My goal in this post is to use data from the OMS Weather Data solution and Azure functions to automatically start my Webasto heater based on the observed temperature. I have to say, using the app directly way easier, but this is a very good opportunity to explore and play with cars, azure and automation.

Starting the heater

Many of these heaters and apps only communicate using SMS commands, but my heater controller can use internet connection as well, meaning there is an API of some sort it’s communicating with.

Using fiddler I found out how the app was communicating, replicating it using Powershell I ended up with this as a simple function to send commands to my heater. Typical commands are “cmd_heater_status” which asks for current status, and “cmd_time1:30,start” that starts the heater for 30 minutes.

To be honest, I wasn’t very pleased with the security here, but that’s something I will notify the company who make these controllers about.

Below is a typical output after requesting heater status.

heater : 1 gsm : 7 voltage : 12.0 h_temp : 25 flame : 0 date : 2017-10-1 19:13:9 valid : 1 status_date : 01/10 status_fdate : 01.10.2017 status_time : 21:13:09

Creating the Azure Function

After we have the basic functionality in place, we can add some logic to our script and create an Azure Function out of it. I have created a basic Powershell function below that accepts two input parameters. TempLow the temperature you have to be below to fire the heater, and heatingMinutes which is the number of minutes the heater will run.

The rest of the script is using our function above to request statuses and act upon our input and the heater response. I had to add sleep time within the script so that the Webasto controller had time process my commands.
Parameters like email-address and heaterId are defined as function environment variables

OMS / Log Analytics setup – query and alert.

Using data from the weather solution I created together with Cameron Fuller i created the following query to alert when observed temperature in Oslo is below a certain degree. This search is based on the new query language, Kusto. I reccommend everyone (including my self) to take a look at this Kusto cheet sheet.
YRno_CL | where LocationName_s == "Oslo" and TimeGenerated > ago(2h) | summarize AggregatedValue = avg(ObservedTemp_d) by bin(TimeGenerated, 15m), LocationName_s

Create an metric based alert based on the above query.

To kick off our Azure Function we set the alert to send a webhook with a custom json payload looking like this: (yes testing values)
{ "tempLow": "59", "heatingMinutes": "40" }

The test says webhook is sent successfully, and our function log confirms 🙂

Calm down John Snow, let winter come.

Automation

Using OMS and Azure Functions to restart Azure Web…

16/06/201714/06/2017
by Martin Ehrnst

From time to time I have had problems with this blog and other sites running in Azure. Occasionally they throw HTTP 5XX errors and when it doesnt fix it self I will have to restart the web app. By using Operations Management Suite (OMS) with the web app analytics solution added, I created an alert calling an Azure Function to restart the affected web app. It’s keeping the sites up until I can get to the bottom of the problem or change provider.. Anyway, this is how I set it up.

Before we create the actual function we will we have a few dependencies.

OMS Workspace with the Web App Analytics enabled (currently in public preview) –Not Covered
Azure Function account –Not Covered
Azure AD application used for authentication

Create an application for authentication

In order to access and manage Azure resources from Azure Functions we need to create an application in Azure Active Directory and assign it the proper permissions. I Used Powershell to do this, but it’s perfectly fine to use the online console.

The script below will create a new application and assign it “web site contributor” role. If you need other security rights chose a different role. You can read more about the built-in roles here

Please edit the script to your needs. You will need the App Id, app password and TenantId to use as variables in our function.

Login-AzureRmAccount

#Add azure application to use for authenication from azure function
$application = New-AzureRmADApplication -DisplayName "AUTH: Azure Function" -HomePage "http://yourwebsites/functions" -IdentifierUris "https://adatum.no/AzureFunctions" -Password "***********************************"
#Grab the application ID
$appid = $application.ApplicationId.Guid 

#create a service principal for the AAD app
New-AzureRmADServicePrincipal -ApplicationId $appid

#add a role to the newly created principal.
#I have chosen Web Site Contributor, but if you want to use anything else. please see https://docs.microsoft.com/en-us/azure/active-directory/role-based-access-built-in-roles
New-AzureRmRoleAssignment -RoleDefinitionName 'Website Contributor' -ServicePrincipalName $appid

#Get the tenant ID
$TenantId = (Get-AzureRmSubscription -SubscriptionName "NAME").TenantId

Finished:

Setting up the Function app

First, create three environment variables in your function app containing your application ID, password and tenant id.

FunctionApp>settings>Manage Application Settings

I had appreciated examples of how to encrypt this password

Next, the actual code.

#Get input
$input = Get-Content $req -Raw | ConvertFrom-Json
$AppName = $input.appName
$ResourceGroup = $input.resourceGroupName

#Get user and password, create credential object
$User = $env:AzureFunctionAppID
$Pass = ConvertTo-SecureString -String $env:AzureFunctionAppPwd -AsPlainText -Force

$Credential = New-Object System.Management.Automation.PSCredential $user,$pass

#Login
Add-AzureRmAccount -Credential $Credential -TenantId $env:TennantId -ServicePrincipal

#Restart the web app and output state
Restart-AzureRmWebApp -name $AppName -ResourceGroupname $ResourceGroup
$Output = (Get-AzureRmWebApp -name $AppName).State
Out-File -Encoding Ascii -FilePath $res -inputObject $output

This Powershell function will use the environment variables/AAD application to login to your Azure environment and restart the WebApp provided in the input. I have set it up to require AppName and ResourceGroup (name) in the json post.

Send the following in the test pane to verify that the function works.

{
    "appName": "mywebApp",
    "resourceGroupName": "MyWebbAppResourceGroup"
}

There isn’t much response other than the (hopefully) “Running” status, but you can confirm everything in you activity log. The user initiated will be your application we created in the first step.

OMS Alert with Json payload

To automate the process, OMS need to trigger the function when too many 500-errors occur. Azure Web App Analytics (preview) in OMS already have views enabled for different error codes, response time etc, so you can use this for what ever you need. I wanted to restart a specific web app based on error 500’s. This is the search string I ended up with

Type:AzureMetrics ResourceId=*"/MICROSOFT.WEB/SITES/"* (MetricName=Requests OR MetricName=Http*) MetricName=Http5xx Resource=WEBAPPNAME

Based on that search result i created an alert sending a webhook with a custom Json payload

{ 
"appName": "mywebApp", 
"resourceGroupName": "MyWebbAppResourceGroup"
 }

OMS should now kick off your Azure Function and restart the website without your interaction. To verify you can use the acitivity log and OMS’ alert log.

Serverless application with PowerShell: Azure Event Grid

Azure Event Grid

Set up Event Grid custom topic

Subscribe to Event Grid topic

Send event to event grid using PowerShell

Summary

Share this:

Creating a serverless application with Powershell: Introduction

The original challenge

Pizza ordering

Parts in this blog series

Share this:

Creating Azure AD Application using Powershell

Azure AD Application

High level overview

Create the app using Powershell

Assign required API access

Application Access Key

Create new Azure AD service principal

RBAC / IAM subscription access

Final script

Share this:

Activate Webasto diesel heater using OMS and Azure functions

Starting the heater

Creating the Azure Function

OMS / Log Analytics setup – query and alert.

Share this:

Using OMS and Azure Functions to restart Azure Web…

Create an application for authentication

Setting up the Function app

OMS Alert with Json payload

Share this: