Automation Archives - Page 2 of 7

Using Azure pipelines to deploy ARM templates

27/02/201907/01/2025
by Martin Ehrnst

There are many reasons that you would use an Azure Resource Manager (ARM) template to deploy your Azure resources. And there’s equally many ways to deploy this kind of template.

For simple one time deployments of Azure resources I tend to use PowerShell or AZ CLI, but lately I have ben busy trying to create a web api based on Azure Functions using C# and .NET core. For those who know me, that’s not straight foreward.
To make things even more diffucult, I wanted to use Azure DevOps and Pipelines to build and push my code. In order to do that, I had to have some infrastructure in place, so why wouldn’t I use Azure Pipelines to deploy the ARM template as well?

Create ARM template

In order to deploy Azure Resource Manager templates, you will need to create one. ARM templates are based on Json and follows a schema. This schema or rule set if you like, defines how you must structure your template.

There are two forms of deployment options. At a resource group level (most common) and subscription level deployment. Both uses the same kind of Json language but have two different schemas. You’ll find the current resource group deployment schema here (no need to read :))

If you are unfamiliar with Json and ARM templates I recommend you to read Microsofts official documentation and getting started guide which will show how you can find the template from an existing deployment or create a new.

For reference, this is a simple template to deploy a web app, that I copied from Microsofts GitHub repo with a bunch of templates to get you started.

Adding ARM template along with your code

As I already mentioned deploy ARM template in a pipeline makes sense in many ways, especially if you have developed an application. This way you can control the infrastructure along side the code.

My Azure Function uses Cosmos DB and Key Vault as parts of it’s infrastructure. It is all developed in Visual Studio Code, and the whole thing is cutting edge for an old operations guy like me.

In the picture below you’ll see that I have structured my project in to two subfolders. One is holding the ARM templates, and the other one is for the acutal functions code. I did this so i would be able to chose artifacts from the build process in Azure DevOps pipelines. There might be a better way, but when i had everything in the same folder the templates was packed with the code and not available in deployment pipeline.

vscode function project with arm template folder

Set up the build pipeline

Not long ago I learned that there are two types of pipelines. Build pipelines and release pipelines. Build is used to pack up or actually build the application (same as pressing f5 in Visual Studio). Where release are used for pushing the application on to your infrastructure (and deploy the infra).

For reference. In Azure DevOps, my folder structure looks exactly the same

Next move over to pipelines and create a new release pipeline. From here chose your repository (I use Azure repos).

After chosing your repo, ou can chose to start from a template. My project is a .NET core application and I had no idea what was needed to build that, so a template worked nicely. Here you can chose what’s best for you’re project. The important pieces from an ARM template perspective comes in the next steps

Start out by filling in the obvious. A name and a server to do the build. In my case i run with Azure hosted 2017. In an on premises environment I would use private servers.

Add your templates as build artifacts

What you will need to do next is to take those Json files in your arm template catalog, and make them available as part of the build. You will have to add in a “Copy files” step and fill in a few properties. Pay attention to where I chose the source folder and the destination folder which have a variable reference, $(build.artifactstagingdirectory)/arm

Now. Hit save and queue. This will save your new (or edited pipeline) and start a build of your project.

By now you should also have a build running, and if you’re like me, exploring new stuff, you get used to this

In this particular error. The publish task for .NET core is missing www root folder. By trial and error I found that I had to make a few changes to my build pipeline. This is not a prerequisite for running ARM templates with Azure pipelines, but i figured I should add in everything I had to do for this project, both for my reference and for yours if needed.

I removed “publish web projects” as i don’t have an ordinary web project, but an Azure function based API.

When you remove this tick, the task will refere to your project file (.CSPROJ).

Here i should give a shout out to my trusty colleague Emil Kjelsrud for helping out, once again.

Hit save and queue once again. Voila!

Configure the release pipeline with ARM templates

When your build is successful, we’re close to launch our application on to Azure. We do that by configuring a release pipeline. It is possible to combine the two, but I like them separated.

Under Pipelines > Releases hit new. A new wizard will pop up. Either chose one that fit your purpose, or start from scratch. In my case I chose to start with app service deployment. One of the first things you notice is “some settings need attention.”

Required settings in this context are the connection to your Azure environment and subscription. You will need to configure this in order to continue. There are several methods available for connecting your pipelines to Azure. I have my pipeline deploying resources in a tenant and subscription that my user does not have access to. That configuration requires a SPN/App registration in Azure AD with the correct permissions.

After configuring your connection. Modify your pipeline by adding in your required Azure Resource Group Deployment tasks. Here comes a few important bits. Prior to setting up my pipeline, I have a resource group created already. If you want to have the resource group created during deployment. You can use the Azure CLI or Azure powershell tasks (or a template) to do so.

Continue to fill in the required parameters. Remember that I separated my application code and the templates in to separate folders? This is why. I can now chose from the same artifact what i want to deploy, as the build pipeline have them available in their respective folders. Chose the template and your parameters file.

In this Azure DevOps environment we have an extension available that parses all outputs from your ARM templates. This way I can use variables from the previous step in my app service deployment. If you dont have this extension available, you can achieve the same result using PowerShell

Create the release

You’re pipeline is now complete. Save and create a release, the pipeline should now pick up the latest build and deploy your code to Azure. You can follow the whole process in Azure, under resource group > deployment and in Azure DevOps

After a few minutes (or seconds) depending on your configuration, you should have both your infrastructure and the actual application available.

Final thoughts

While I understand many of the concepts around infrastructure as code, continious integration and so on, I don’t always use the correct terms, so thank you for reading.

I hope you also have a greater understanding around the concepts, and also how you can utilize the possibilities available. My thoughts working with projects like this is always bound to the operational side. I know I’m going to use more ARM templates in Azure Pipelines, but probably not along side the actual code, as that’s not my day-to-day work. Again, I see my self working more and more with these Dev tools, and thats why I like to say that this is how we do modern Ops.

I leared a few more tips and tricks during this small project, so expect a few more posts soon.

Automation

Serverless application with PowerShell: Azure Event Grid

30/05/201807/01/2025
by Martin Ehrnst

This is part one of my blog series describing how you can create a serverless application infrastructure using PowerShell. The scope and history behind, covered in the introduction post. If you haven’t read that I recommend doing so.

Azure Event Grid

In my serverless infrastructure design, I have chosen to use Azure Event Grid as a hub for the other components in play. The other components will write and subscribe to events/messages in Event Grid and perform their given task as ‘their’ event arrives. Event grid fit’s the concept perfectly as you act on state changes. You can publish as many events you like, but you don’t need to subscribe to every one. A lot of Azures PaaS offerings are already fully integrated which makes it easy to get started.

Other event based offerings in Azure are Service Bus and Event Hub. You can read a little on the differences here

Set up Event Grid custom topic

I am sure you are able to create resources in Azure so here’s a quick one. Feel free to use what ever you want for deployment, but for the purpose of screenshots, I’m using the portal. Just remember that the name you chose will be the url of your endpoint as well.

After the custom topic is successfully deployed, you are given the option to subscribe to events. Before doing that we are taking a break. Besides a graph showing incoming events you have no good way to see your events as they arrive. This makes it way more difficult to debug why stuff isn’t working. While writing this post I researched for a good way to solve this and stumbled across this blog post by David Barkol. He created a web app that lets you view events as they arrive. You can deploy it directly from GitHub using ARM. Do this while your coffee is brewing. And come back here in 10 minutes 🙂

Subscribe to Event Grid topic

Welcome back… If you have deployed the web app you can follow and create a subscription subscribing to all events. In a production scenario, it’s likely to have scoped your subscribers, but in order for Event grid viewer to function i’ll subscribe to everything.

Once deployed your overview page will look something like this. Showing all subscriptions within your topic and their corresponding metrics. Now lets try to send a custom event using powershell.

Send event to event grid using PowerShell

Basically, event grid accepts post request to an API endpoint, which is the name of your topic plus a suffix and the API version. your full URI looks something like this; https://adatumblogdemo.northeurope-1.eventgrid.azure.net/api/events?api-version=2018-01-01

To authenticate we will pass an ‘aeg-sas-key’ within our request header. You will find the keys for authentication in the keys pane. Further event grid events has a schema you willl need to follow, meaning that all post requests will need to have some properties defined and your custom event data will be in a data property. The basics of it all looks like this

[ { "topic": string, "subject": string, "id": string, "eventType": string, "eventTime": string, "data":{ "custom": "data" }, "dataVersion": string, "metadataVersion": string } ]

More details on the schema can be found on Msft Docs

If we combine everything we know this far, we can use the script below to post messages to event grid. Please add your own url and access key. Everything else should work.

Assuming no error messages, jump in to your event grid viewer to view the raw content like this;

A corresponding graph in the Azure portal could look something like this. Here you will see incoming events and how many matches your subscriptions in the same graph. You can also see individual metrics per subscription.

Summary

In this part of my blog series on how to create a serverless application infrastructure using PowerShell. We have walked through how to set up a custom Event Grid topic and how to post messages or event to it using standard PowerShell commandlets. We also deployed a custom web application capable of reading our incoming events in real time.

In the next part: For part two we are going to create an Azure Function running PowertShell that subscribes and send Event Grid messages. This is key to build up our infrastructure wich consists of multiple Azure Functions in order to work.

Automation

Creating a serverless application with Powershell: Introduction

08/05/201807/01/2025
by Martin Ehrnst

Welcome, fellow IT pro. You have found the blog series describing how you can create a serverless application infrastructure with Powershell.

If you consider your self being a ‘modern it pro’, you frequently solve complex tasks and automate your work using code. You know what Git is and have heard people talk about continuous integration. If you can relate, let’s label you a ‘modern it pro’.

When I develop solutions that challenge my knowledge, I often seek advice from our developers. (You know, those who are employed to ‘GIT commit’?) Usually they can share some light on the issue and point me in the right direction.
By developing our own solutions, we have a working proof-of-concept to handover to our developers when things grow too big or complex.

The original challenge

The challenge or scope for my POC was to streamline creation of Azure tenants for our customers. We required to create a tenant, assign subscriptions and do some configuration within Azure AD. Many of these steps was already solved with various scripts, but it is time to consolidate and automate the automation. We decided that we wanted to learn more around the integration and ‘serverless’ PAAS offerings available in Azure.
For the purpose of this blog series we are going to build a new application using the same techniques, in a smaller scale. To create our serverless application infrastructure, we make use of the following Azure offerings:

Pizza ordering

I bet you have worked a fair amount of overtime in your career, therefore eaten a lot of pizza as well. Let’s try to build a simple pizza order and delivery process using Event Grid as integration layer and multiple Azure Functions to process the order.

We will break the pizza order and delivery process in to the following steps:

Customer creates/orders a pizza online
Send orderdata to event grid
Azure function subscribe to the “new order” event
Create a new event when pizza is in oven
Post new event when pizza is cooked and ready for delivery
Create a new event when pizza is sent to customer.

Using Azure pipelines to deploy ARM templates

Create ARM template

Adding ARM template along with your code

Set up the build pipeline

Add your templates as build artifacts

Configure the release pipeline with ARM templates

Create the release

Final thoughts

Share this:

Serverless application with PowerShell: Azure Event Grid

Azure Event Grid

Set up Event Grid custom topic

Subscribe to Event Grid topic

Send event to event grid using PowerShell

Summary

Share this:

Creating a serverless application with Powershell: Introduction

The original challenge

Pizza ordering

Parts in this blog series

Share this: