Azure Infrastructure as code – Pulumi

Infrastructure As Code is here to stay. And all companies work with this in a variety of ways. Recently I changed job, and with that comes new challenges. The team I joined I highly skilled and is responsible for a very complex, and large infrastructure in Azure. A great part of this infrastructure is deployed and maintained using a tool called Pulumi.

My new role does not require me to become a developer creating Apps. But I have been advocating and teaching fellow IT pro’s the importance of embracing developer tools and processes for our infrastructure management tasks. My knowledge around infrastructure as code is with PowerShell, Terraform, and ARM. My C# skills are very limited, although I have some experience.
Pulumi is definitely putting developers first, and I need to step up my game.

What is Pulumi

Azure Resource Manager and Azure Bicep are both domain-specific languages, meaning they only work with Azure. Terraform, is another popular tool (almost a standard), which also has it’s own language (HCL). HCL differs from ARM as it works with more than Azure.

Create, deploy, and manage infrastructure on any cloud using familiar programming languages and tools.

Pulumi

Pulumi on the other hand, use general-purpose programming languages. This means you can deploy and maintain your infrastructure with ‘real programming languages’, like C#, Java, TypeScript, and Go.

How does Pulumi work

Pulumi is a declarative infrastructure as code tool. And it’s core engine will ‘build’ your desired infrastructure, and keep track of its state.

Projects and stacks

You start with something called a Project. The project folder is controlled via a Pulumi.yml file looking something like this, where name and runtime are mandatory.

name: core-infra
runtime: dotnet
description: my very first pulumi project

After creating the project you will need to create a stack. The stack is an instance of your project. For example, staging and production of project core-infra would be two separate stacks.

State management

You might be familiar with this concept already, but if not here’s what’s what;
Pulumi keeps a snapshot of your infrastructure, referred to as ‘state’. This allows Pulumi to delete, create, and change your infrastructure components. But it also means you have to think about where you perform edits (only within the Pulumi stack/project), and where to store your state files.

By default Pulumi will store and manage state with their online service, Pulumi Console.

Getting started with Pulumi for Azure

My short goal for self learning Pulumi is to replicate what I demoed in me and Marcel Zehner’s Live streams on Azure resource manager and infrastructure as code.
for Pulumi I am using this repository

For some reason, I assume you run Windows and CSharp, but if you fancy any of the other options, they are documented as well.

To run Pulumi on Azure you will need to install Pulumi, log in/sign up, install .NET 3.1, and Azure CLI (if you don’t have it already). The process is documented on the getting started page.
I tried to run with .NET 5.0, without any luck, but that might be solved soon.

Your next task is to create your project. In all essence, you run a few commands against an empty folder. This will generate the Pulumi program files and your project metadata files. Below is my configuration

cd C:users\MartinEhrnst\repos\Pulumi\
mkdir 1.ResourceGroup-storageAccount
cd 1.ResourceGroup-storageAccount
pulumi new azure-csharp

After filling in your mandatory project parameters, a getting started code will be generated for you. This will create an Azure resource group and a storage account.
In the above picture, I have changed this slightly to include a storage container, and change some of the default parameters. You can find my latest Pulumi code in this GitHub repo

For those experienced with C#, you can see that Pulumi has classes for the Azure resources. But since this is C#, we can use common coding techniques, like iterations (for-each) to deploy our infrastructure.

Pulumi deployments

If I now want to deploy my infrastructure. I will need to run Pulumi, which translates this code into something Azure Resource Manager can understand. To my knowledge, Pulumi uses the Azure Resource Manager REST APIs to run the deployment.

To deploy the resources, you can follow this guide. In my environment above, this is the code and output from my review.

PS C:\Users\MartinEhrnst\repos\Pulumi\1.ResourceGroup-storageAccount> pulumi up
Previewing update (dev)

View Live:

     Type                         Name                Plan
 +   pulumi:pulumi:Stack          rg-and-storage-dev  create
 +   ├─ azure:core:ResourceGroup  resourceGroup       create
 +   ├─ azure:storage:Account     storage             create
 +   └─ azure:storage:Container   container           create
 
Resources:
    + 4 to create

Do you want to perform this update? details
+ pulumi:pulumi:Stack: (create)
    [urn=urn:pulumi:dev::rg-and-storage::pulumi:pulumi:Stack::rg-and-storage-dev]
    + azure:core/resourceGroup:ResourceGroup: (create)
        [urn=urn:pulumi:dev::rg-and-storage::azure:core/resourceGroup:ResourceGroup::resourceGroup]
        [provider=urn:pulumi:dev::rg-and-storage::pulumi:providers:azure::default_3_33_2::04da6b54-80e4-46f7-96ec-]
        location  : "norwayeast"
        name      : "rg-PulumiStorage"
    + azure:storage/account:Account: (create)
        [urn=urn:pulumi:dev::rg-and-storage::azure:storage/account:Account::storage]
        [provider=urn:pulumi:dev::rg-and-storage::pulumi:providers:azure::default_3_33_2::04da6b54-80e4-46f7-96ec-b56ff0331ba9]
        accountKind           : "StorageV2"
        accountReplicationType: "LRS"
        accountTier           : "Standard"
        allowBlobPublicAccess : false
        enableHttpsTrafficOnly: true
        isHnsEnabled          : false
        location              : output<string>
        minTlsVersion         : "TLS1_0"
        name                  : "storage2966fa9"
        resourceGroupName     : "rg-PulumiStorage"
    + azure:storage/container:Container: (create)
        [urn=urn:pulumi:dev::rg-and-storage::azure:storage/container:Container::container]
        [provider=urn:pulumi:dev::rg-and-storage::pulumi:providers:azure::default_3_33_2::04da6b54-80e4-46f7-96ec-b56ff0331ba9]
        containerAccessType: "private"
        name               : "images"
        storageAccountName : "storageab46f04"

In Azure, I can now see that the storage account and resource group are created. But I cannot find this as deployments. I suspect this has to do with how Pulumi interacts with Azure resource manager. This might not be an issue for you, but if you rely on the deployment plane, you should have given this a thought.

Should you use Pulumi for Azure?

Given my very limited knowledge of the product that is hard for me to answer. But there are things you should consider.
As I said, I have advocated for a few years about the ‘Modern IT pro’. Meaning we need to adopt and use more developer-oriented software and processes, like Git for example.

By using Pulumi you are not only adopting processes, but you also assume your team knows CSharp or any of the other supported languages. If your team consists of IT Pro’s who are beginning to explore the Dev side of the DevOps circle. Pulumi will give you some rough weeks ahead.

On the other hand, if your team is developer heavy, looking into the operations side. Pulumi might be your best choice. As a developer, it must seem alluring to be able to provision infrastructure together with your application code.
However, the responsibility for correct configuration, governance, and security is still the most important for your infrastructure. Can this be done with the same team and codebase, you can definitely consider using Pulumi.

Pulumi ARM template converter

A tool to convert ARM templates to Pulumi already exists. During my initial testing, I had success converting less complex templates, but when I tried to convert a nested template with a Copy loop the tool failed.

I suggest you try it out with your own templates, and since it’s open-sourced, you could always try to improve it your self. If not, the community will at some point.