Skip to content
adatum
  • Home
  •  About adatum
  •  Learn Azure Bicep
  •  SCOM Web API
Azure DevOps

Automate Azure DevOps like a boss

  • 23/06/202123/06/2021
  • by Martin Ehrnst

Over the past weeks, I have worked on automation within Azure DevOps. One of the tasks was to create a self-service tool for our developers to create new projects where all pre-requisites were taken care of. That means the project itself, service connections (ARM) to access our Azure Kubernetes Service (AKS), Azure AD groups for accessing the project, and so on.

For the Azure AD part, I had mostly everything in place already. Service connections for Azure AD are essentially Azure application registrations, and Azure AD groups are easily created using New-AzADGroup or Microsoft Graph API. However, my experience with Azure DevOps API was and still is limited.

Creating the project

Creating the project itself is pretty straightforward. Below is a PowerShell snippet to do so. I used a service account and a Personal Access Token (PAT) with the following access permissions to do everything in my code.

  • Graph, read/manage
  • Identity, read/manage
  • Member Entitlement Management, read/write
  • Project and team, read/write/manage
  • Security, manage
  • Service connections, read/manage/query

Assigning permissions using Azure DevOps API

You don’t get far with just a project. Someone needs to have access to it as well. Since I try to create a paved road for the DevOps users I needed to add users to the DevOps groups inside the project. Here I spent some significant time, trying to understand how. In my routine, I already create three AAD groups for Project Administrators, contributors, and readers. We want to have one single pane for managing access to company resources, and Azure AD is the place.

In my opinion, access control in Azure DevOps is a nightmare, but lets not focus on that. I want to add each AAD group in to it’s corresponding Azure DevOps group.

To do that you use Azure DevOps Graph and the Groups endpoints. What you can see in the Microsoft example is that you need to perform a post operation against https://vssps.dev.azure.com/{organization}/_apis/graph/groups?groupDescriptors=vssgp.Uy0xLTktMTU1MTM3NDI0NS0yMTc2MDU2ODQ4LTMyODAwNzczODUtMjM4NDQzMDA4Mi0yNTc3Njk3NTA4LTEtNzM0NDQ1NzM2LTQwNzkyNzIyNjgtMzA0NzY5MjIyMy0yMjg2NTY0ODM0&api-version=6.1-preview.1

Azure DevOps Descriptor

From the documentation, you can see the request to add AAD groups into a project group like Project Administrators you need to specify a groupDescriptor. I could not find any good information on how to get this information, and as you can see, the request does not specify the project, only at the organization level.

Now, after some digging, I figured I could fetch this information with some additional calls to the API.

  1. Get your project descriptor, by using the projectId as your descriptor…
  2. Get all groups within the project, using the actual project descriptor
  3. Filter the groups you want from the above request
  4. Pass the Azure AD group objectId in the body and POST against /graph/groups?groupDescriptors=

Modify Azure DevOps project features

In our case, we do not use all features in Azure DevOps. I, therefore, wanted to turn off some features like boards. Making it a bit more difficult for our developers to start using features that we have other tools to support. I couldn’t find any official endpoint documented, so I reached out on Twitter to get inputs. Of course fellow MVP Rob Bos had some time to inspect the portal calls and came up with the solution. Basically, after the project is created, you need to make additional calls to an old API to flip the switch for each feature you do not use.

Adding ARM service connections to project using DevOps API

This part is quite well documented. So after creating the app registrations and service principals in Azure AD. You need to create the service connection(s) in the project. In my case I had more than one service connection, so I had to run this snippet multiple times, as I could not find a batch operation for it.

I think that pre-provisioning of the service connections are a good idea. This feature is quite complex to grasp for many, as the built in wizard cause more confusion than it helps. Especially when the users try to create a new SC, and they do not have access to create App registrations in Azure, or to set permissions on the subscription. Then the entire wizard will fail, but you might end up with ghost SPNs in Azure…

I said it was properly documented, but if you take a close look. The endpoint is called servicendpoint and not serviceconnection which is what it is called in the portal.

Automation solutions for Azure DevOps

In my examples I use the native Azure DevOps APIs and PowerShell. This worked best in our solution. However, you can also use Azure CLI or Azure DevOps provider for Terraform.

Since we do not use Terraform for any IAC, I weren’t keen to start using it just for Azure DevOps. Azure CLI have some limitations since it only allow interactive login, or a semi-solution for the PAT. I suggest you see what solution that works for you. But my examples above do at least work.

Summary

In this post I showed how I solved Azure DevOps project creation. The code above only show half the solution as it is very domain specific. However, I hope it helps you along the way. Creating the actual project is only one part of it. You need to create a paved road for the users. Therefore, make sure the users are able to access their projects and deploy resources. Automating the setup will ensure that every project is set up equally and supporting future requests from your developers will be much easier.

Share this:

  • LinkedIn
  • Twitter
  • Reddit
gray laptop computer showing html codes in shallow focus photography Azure

Azure Infrastructure As Code video series

  • 28/10/202028/10/2020
  • by Martin Ehrnst

For weeks Marcel Zehner and I have held four live streams. Covering ‘everything’ related to Infrastructure as code on Azure.

Recording available

In the series, we covered the following topics, and everything is now available on YouTube

  • Advanced ARM templates
  • Deployment scripts
  • Linked and nested ARM templates
  • ARM template deployment with Pipelines

Share this:

  • LinkedIn
  • Twitter
  • Reddit
Automation

Multi subscription deployment with DevOps and Azure Lighthouse

  • 11/03/202003/09/2020
  • by Martin Ehrnst

Companies today are rapidly adopting new technology. Adding more pressure on the companies IT-department or the service provider. No matter where the workload runs, governance, security, and deployments are fundamental parts.
Azure Lighthouse came last year, and while it solves a lot of our trouble. Multi subscription deployments aren’t one of them – but it sure makes it more possible!

Azure deployments with Azure DevOps

One of the things that make service providers great at what they do is standardization. For example; making sure all subscriptions (customers in many cases) have the same set of baseline Azure policies.

Azure DevOps has multiple ways to deploy resources in Azure or other places. How do we deploy the same Azure template to multiple subscriptions using the same pipeline? In our case, I solved this with some existing features, forward-thinking and PowerShell magic.

Azure DevOps service connection with Azure Lighthouse

Multi subscription deployments with Azure DevOps is not a built-in feature. With Azure Lighthouse it became a little bit easier but will require some work.

First, you must set up a service connection and allow that to access one of your internal subscriptions. In Azure DevOps service connections are bound to one subscription.

For this service connection to be capable of multi subscription deployments, it will need access to your customer’s subscriptions. This can be solved through delegated resource management and Azure Lighthouse.
In my case, I had a group with contributor access. And I could add the SPN to that group. Otherwise, you will have to update your current delegation.

Repository structure

Everything you need, including a YAML pipeline is available on GitHub, but I will walk you through how and why I set it up.

I needed to create not only a solution to deploy one resource to multiple subscriptions. I also needed to deploy multiple ARM templates.

For this purpose, I had the option to create one large PowerShell script with complex logic. Or, I could reuse the same script for every deployment. I chose option two. My code repository now looks something like this

  • ARM-Templates (folder)
    • storage-Account (folder)
      • azuredeploy.json
      • azuredeploy.parameters.json
      • deploy.ps1
    • another-resource (folder)
      • azuredeploy.json
      • azuredeploy.parameters.json
      • deploy.ps1
    • […]
  • azure-pipelines.yaml

PowerShell magic?

In regular deployments, we can use built-in tasks in the pipeline and deploy directly. For multi subscription deployment, PowerShell is my weapon of choice.

To people with PowerShell competency. The script used is fairly simple;

  1. Connect to Azure
  2. Retrieve the subscriptions
  3. Iterate and deploy the ARM template(s) to each customer subscriptions.

Below is a short example. Showing the core in my deployment script (deploy.ps1)

$deploymentName = "Multi-sub-deployment"
$deploymentLocation = "westeurope"
$templateFile = ".ARM-Templatesstorage-Accountazuredeploy.json"
$templateParameterFile = ".ARM-Templatesstorage-Accountazuredeploy.parameters.json"

# getting all subscriptions
$subscriptions = Get-AzSubscription | Where-Object { $_.Id -NotIn $excludedSubs }

foreach ($subscription in $subscriptions) {
        
    # set context to the current subscription
    $subscriptionId = $subscription.id
    Set-AzContext -SubscriptionId $subscriptionId

    # deploy the arm template
    New-AzSubscriptionDeployment -Name $deploymentName -Location $deploymentLocation `
        -TemplateParameterFile $templateParameterFile -TemplateFile $templateFile
}

Multi subscription deployment, build pipeline

Although my repository contains a YAML pipeline, you don’t have to use it. To be honest, I’m not sure I like them. I used too much time trying to wrap my head around it. And at this point, it seems unfinished from the Azure DevOps side. Therefore, I will show you how to set up your pipeline to support multi subscription deployments using the classic method.

azure pipeline template selector

kick off with the classic mode, and chose the empty job on the template page. We could probably discuss if we even need multiple pipelines for this. But it doesn’t hurt, and it will be easier for the next person if we do this by the book.

For the build pipeline, I am renaming my job to something meaningful, and add one single task. Publish pipeline artifact

After you save and run the build. An artifact should be produced. The result should look something like this

Azure pipeline artifact

Multi subscription deployment, release pipeline

After a successful build (or in this case copy files), it is time to create our release pipeline. When using YAML, the two pipelines are combined. Not at all confusing for someone like me, who not that many months ago, didn’t know anything at all about this stuff.

Once again, I start off with an empty job, before I add my artifact from my build pipeline. I also renamed the first stage to “resource deployment”.

Now it’s the matter of adding tasks to our job, and it’s here you will need the service connection that you added earlier. The task we are working with is the Azure PowerShell task. And for multi subscription deployment, it is the only task you’ll need. Below is my task configuration

For some reason, it takes a few tries before pipelines want to work. It might be because of lat hours, or a law created by some guy named Murphy. Anyway, once it’s up and running you shole be able to see something similar to my output below;

2020-03-10T17:13:12.2503976Z ## Az module initialization Complete
2020-03-10T17:13:12.2517976Z ## Beginning Script Execution
2020-03-10T17:13:12.2532801Z Generating script.
2020-03-10T17:13:12.3293381Z ========================== Starting Command Output ===========================
2020-03-10T17:13:12.3416275Z ##[command]"C:Program FilesPowerShell6pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'd:a_temp112fe6df-8ee0-4a10-b53e-5694a4d34b0f.ps1'"
2020-03-10T17:13:25.0914923Z No subscription specified. Deploying to all subscriptions
2020-03-10T17:13:25.2151812Z 
2020-03-10T17:13:25.2211374Z Name                                     Account             SubscriptionName    Environment         TenantId
2020-03-10T17:13:25.2292766Z ----                                     -------             ----------------    -----------         --------
2020-03-10T17:13:25.2300155Z MVP-Sponsorship (6dca9329-fb22-46cb-826…  MVP-Sponsorship     AzureCloud          22046864-98a9-4a9…
2020-03-10T17:14:01.7100820Z 
2020-03-10T17:14:01.7151741Z Id                      : /subscriptions/6dca9329-fb22-46cb-826c-/providers/Microsoft.Resources/deployments
2020-03-10T17:14:01.7152947Z                           /Multi-sub-deployment
2020-03-10T17:14:01.7154721Z Location                : westeurope
2020-03-10T17:14:01.7159918Z ManagementGroupId       : 
2020-03-10T17:14:01.7161790Z ResourceGroupName       : 
2020-03-10T17:14:01.7162557Z OnErrorDeployment       : 
2020-03-10T17:14:01.7163149Z DeploymentName          : Multi-sub-deployment
2020-03-10T17:14:01.7163805Z CorrelationId           : d49c10f8-0260-49d5-aa8d-08a41591d1a7
2020-03-10T17:14:01.7164463Z ProvisioningState       : Succeeded
2020-03-10T17:14:01.7165107Z Timestamp               : 3/10/2020 5:14:00 PM
2020-03-10T17:14:01.7166059Z Mode                    : Incremental
2020-03-10T17:14:01.7166610Z TemplateLink            : 
2020-03-10T17:14:01.7167216Z TemplateLinkString      : 
2020-03-10T17:14:01.7167793Z DeploymentDebugLogLevel : 
2020-03-10T17:14:01.7168836Z Parameters              : {[rgName, Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable], 
2020-03-10T17:14:01.7169800Z                           [location, Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable], 
2020-03-10T17:14:01.7173723Z                           [storagePrefix, 
2020-03-10T17:14:01.7174379Z                           Microsoft.Azure.Commands.ResourceManager.Cmdlets.SdkModels.DeploymentVariable]}
2020-03-10T17:14:01.7175014Z ParametersString        : 
2020-03-10T17:14:01.7177870Z                           Name             Type                       Value     
2020-03-10T17:14:01.7178513Z                           ===============  =========================  ==========
2020-03-10T17:14:01.7179228Z                           rgName           String                     adatum    
2020-03-10T17:14:01.7179831Z                           location         String                     westeurope
2020-03-10T17:14:01.7180452Z                           storagePrefix    String                     str       
2020-03-10T17:14:01.7181006Z                           
2020-03-10T17:14:01.7181454Z Outputs                 : {}
2020-03-10T17:14:01.7181916Z OutputsString           : 
2020-03-10T17:14:01.7182291Z 
2020-03-10T17:14:02.0220641Z 
2020-03-10T17:14:02.0937716Z ##[command]Disconnect-AzAccount -Scope Process -ErrorAction Stop

If you look at the output, you can see that the script set’s context to a subscription, and that there is no subscription specified in the pipeline.

Multi subscription deployment summary

Multi subscription deployments with Azure DevOps is not available as a default. But with a little bit of PowerShell trickery, you got a great solution. For service providers and large enterprises, Azure Lighthouse is now a preferred way to manage resources.

By granting a service principal access to your customer subscriptions (or internal for that matter), and use this SPN as a service connection in your Azure Pipeline. You can use PowerShell to iterate through each subscription and deploy the resources needed.

The beauty with this is that it will work regardless of where your DevOps environment is hosted. You can have separate tenants for Lighthouse, Azure DevOps and workplace.

This post described the following, which is required for multi subscription deployment to work.

  • Created an SPN in the management tenant
  • Authorized the service principal through Azure Lighthouse
  • Created a repository and added our scripts and templates to it
  • Created pipelines and used the SPN as our service connection
  • Used PowerShell and the built-in task to iterate through each subscription and perform deployments.

Share this:

  • LinkedIn
  • Twitter
  • Reddit

Popular blog posts

  • Azure Application registrations, Enterprise Apps, and managed identities
  • SCOM 1801 REST API Interfaces
  • Creating Azure AD Application using Powershell
  • Automate Azure DevOps like a boss
  • Multi subscription deployment with DevOps and Azure Lighthouse

Categories

Automation Azure Azure Active Directory Azure Bicep Azure DevOps Azure Functions Azure Lighthouse Azure Logic Apps Azure Monitor Azure Policy Community Conferences CSP Monitoring DevOps Guest blogs Infrastructure As Code Microsoft CSP MPAuthoring OMS Operations Manager Podcast Powershell Uncategorised Windows Admin Center Windows Server

Follow Martin Ehrnst

  • Twitter
  • LinkedIn

RSS feed RSS - Posts

RSS feed RSS - Comments

Microsoft Azure MVP

Martin Ehrnst Microsoft Azure MVP
Adatum.no use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it. Cookie Policy
Theme by Colorlib Powered by WordPress