Skip to content
adatum
  •  SCOM Web API
  • About adatum
Automation

Using webhook in scom subscription (POC)

  • 09/09/201611/09/2016
  • by Martin Ehrnst

Recently, I was at System Center Universe (Now, Experts Live) in Berlin. During one side session i got talk to Cameron Fuller (Microsoft MVP) and I asked if he knew if anybody had used azure automation and it’s webhook capabilites in a SCOM subscription.

At Intility (my employer) we have developed an alert central which handles alerts from Solarwinds and other systems we use daily. OpsMgr is one of few systems that still tap directly in to our ticketing system, but were working on a change here. Webhooks and azure automation is one thing were looking at.

My attempt (if possible), using webhooks and Azure automation is to be able to send alerts from SCOM, and not have something query for new alerts when there nothing to handle. Cameron quickly reached out to his fellow MVPs and a few ours later i got a reply. Turns out that Jakob Gottlieb Svendsen (MVP) from Coretech had used webhooks in a recovery task. He basicly done all the work now :).

http://blog.coretech.dk/jgs/azure-automation-scom-triggering-a-runbook-as-a-recovery-task-using-webhooks/ (read this :))

The rest of this post will quickly show how to.

 

  • Create azure automation account
  • Create a new runbook
  • Add webhook
  • Runbook parameter
  • SCOM Command Channel and subscription

 

Azure Automation Account

 

Search for automation accounts in services and add a new Automation account. Use an existing resource group or create a new. After wizard is finished we should have an account assigned to a subscription.

automationaccount

 

By now you have probably read Jakobs blog and you have a new runbook in place, but i will guide you through anyway 🙂

Create a new runbook and use “Powershell workflow”. Once finished you will be sent to the online editor which is fine for now, but i suggest you download azure automation addon for ISE if you plan to do a lot of scripting here 🙂

Azure Automation PS worflow will start like this. If you add script parameter and publish where able to create a webhook. Place your normal PS script between the two curly brackets.

workflow 'your runbook name'

{

Param [object]($WebhookData)

}

I have adde the parameter already, that way we are able to create webhook an parameter at the next step.

Next task is to create a webhook for your runbook. With a webhook you are able to call the runbook with a URL and also pass parameters with JSON data inside the url. (Heres where the parameter for your webhook and script comes in)

Remember, copy your webhook URL – it’s not available after it is created

webhook1

 

After youre ‘hooked’ add paramters and define where to run. For this test i just fire it off in azure, but you can run your script on prem with the hybrid worker. Use your script parameter in the webhook data input.

webhook

You have probably added your own script inside the Azure workflow already, but here is my OpsMgrAlertHandlerTEST script which is based on Jakobs example and will output some key alert datato the console. Instead of just output it to the console i can now pass this information to our alert central by triggering a new runbook or adding it directly within this script.

 

workflow OpsMgrAlertHandlerTEST

{

param (

[object]$WebhookData

)



    $WebhookName    =   $WebhookData.WebhookName

    $WebhookHeaders =   $WebhookData.RequestHeader

    $WebhookBody    =   $WebhookData.RequestBody



    $Inputs = ConvertFrom-JSON $webhookdata.RequestBody

    $ComputerName = $Inputs.ComputerName

$AlertName = $Inputs.Alert

$ResolutionState = $Inputs.State

$AlertID = $Inputs.AlertID



    Write-Output "Computername: $ComputerName"

Write-Output "Alert: $AlertName"

Write-Output "State: $ResolutionState"

Write-Output "ID: $AlertID"

Write-Output "$Inputs"

}

 

From scom we will use powershell to POST alert data with invoke-restmethod and the URL you copied from the webhook configuration, right?

Again Jakob is apparently our JSON webhook guy http://blog.coretech.dk/jgs/azure-automation-using-webhooks-part-1-input-data/

SCOM Command Channel and subscription

 

Our final step is to create a command channel, subscriber and a subscription to trigger the runbook. From the admin panel in scom create a new command notification channel

channel1

In settings we will add path to Powershell and a command to run. Remember. Webhook url and the parameters to POST

channel2

Here is my full commandline (i have removed my webhook uri here)

-executionpolicy Unrestricted -Command " &{Invoke-RestMethod -Method Post -Uri 'YOUR WEBHOOK URL' -Body (ConvertTo-Json -InputObject @{'ComputerName'='$Data[Default='Not Present']/Context/DataItem/ManagedEntityPath$\$Data[Default='Not Present']/Context/DataItem/ManagedEntityDisplayName$';'Alert'='$Data[Default='Not Present']/Context/DataItem/AlertName$';'State'='$Data[Default='Not Present']/Context/DataItem/ResolutionStateName$';'AlertID'='$Data/Context/DataItem/AlertId$'}) -ErrorAction Stop}"

Inside the JSON string we will pass Computer Name, Alert name and the resolution state. Modify this by using the picker on the right side. After you have created the channel, create a new subscriber to use with this channel and finally create the actual subscription

For this test/POC i have chosen to send all information alerts to this subscription. These are mostly rules i can close (and generate) during testing.

Subscriber and channel are the two we just created.

subscription

 

At this point you should have:

  • Azure automation account
  • Automation runbook, my example or your own
  • Web hook enabled
  • Command channel
  • Subscriber and subscription

 

Finally, lets test our solution. I have added all informational messages to this subscription. I will close one of those and cross my fingers. We can follow the process in azure portal

closealert


As you see the job is queued and in the output console we see the output from our webhook data
automationoutputwait

 

It worked!

automationoutput

 

 

As this is totally in  a proof of concept state for us i would greatly apriciate inputs on how we can accomplish the task using other methods.

As we see, using webhooks in a subscription works quite well. But i havent implemented it in our production environment or sent it to our alert central yet. I see some issues, one being limited by how many powershell scripts we can run at once. Jakob suggested maybe we will look in to creating a connector and let us query a subscription for new alerts but i haven’t gotten around to try it out.

 

Share this:

  • LinkedIn
  • Twitter
Operations Manager

Free E-book: Inside the Microsoft Operations Management Suite

  • 05/01/2016
  • by Martin Ehrnst

Stanislav Zhelyazkov, Pete Zerger, Tao Yang and Anders Bengtsson just released their free OMS eBook, which is available for download on TechNet. Although, this is a preview release, i know there will be interesting to read  this thing “back to back”

  • Chapter 1: Introduction and Onboarding
  • Chapter 2: Searching and Presenting OMS Data
  • Chapter 3: Alert Management
  • Chapter 4: Configuration Assessment and Change Tracking
  • Chapter 5: Working with Performance Data
  • Chapter 6: Process Automation and Desired State Configuration
  • Chapter 7: Backup and Disaster Recovery
  • Chapter 8: Security Configuration and Event Analysis
  • Chapter 9: Analyzing Network Data
  • Chapter 10: Accessing OMS Data Programmatically
  • Chapter 11: Custom MP Authoring
  • Chapter 12: Cross Platform Management and Automation

Read more on Stanislavs blog

Share this:

  • LinkedIn
  • Twitter
Operations Manager

SCU2016 Dallas Texas

  • 05/01/2016
  • by Martin Ehrnst
SCU 2016 dallas texas
SCU 2016 dallas texas

 

System Center Universe is only a few days away, and there is a lot to look forward to. Take a look at the sessions. All the system center speakers you wan’t to see is on SCU2016

As for me, i will have to wait until  SCUEurope, with this video giving us the highlights for 2016

Share this:

  • LinkedIn
  • Twitter
MPAuthoring

Maintenance Mode based on Event

  • 28/09/201528/09/2015
  • by Martin Ehrnst

Automating Maintenance Mode in OpsMgr is probably one of the most blogged topics, but every environment has different needs and solution to this problem. This is just another one.
I suggest you gather your information and browse the internet to find the solution that best fit your needs.

Earlier, i have posted a solution to maintenance mode a SCOM group using Powershell and Windows task scheduler. By doing this, we sorted out a problem with computers that was scheduled to reboot once per day. This post is about putting computer objects in to maintenance mode based on a windows event id. What we are trying to solve is when people forget to MM before boot or Windows Update automatically, randomly reboot computers during it’s patch window. Both scenarios will add an event id to system event log, and since we’re working with Events, this solution can be adapted to MM for any event id you’d like.

What we need to do:

  • Create a new rule to look for the events we want (1074 in this case)
  • Create the Powershell script to actually do the work
  • Create a command channel to call the script
  • Create a command channel subscription and subscriber

Creating the Alert Rule

There are several events that you could use to trigger this rule. In this post i will be using 1074 for both user initiated reboot and Windows update.

I assume you already know how to create a rule and a new management pack, so we will skip that part. As for the rule configuration it will look like this:

The highlighted areas is the difference between the two groups. Explorer.exe is triggered in parameter 1 when a user clicks start>reboot etc. EventID 22 is logged when Windows Update is finished installing new updates. After this event is triggered, the computer will boot within 15min (default). If you want to enable MM for updates in an earlier stage, i suggest looking for events triggered when patches are downloaded, installed or similar.

You will find the parameter # in the xml of the event. Once you have the formula in place, complete your rule with alert message, severity etc.  For reference, this is how my formula looks like.

( ( ( Event Source Equals Microsoft-Windows-WindowsUpdateClient ) AND ( Event ID Equals 22 ) ) OR ( ( Parameter 5 Equals restart ) AND ( Parameter 1 Equals Explorer.EXE ) AND ( Event Source Equals User32 ) AND ( Event ID Equals 1074 ) ) ) 



Now, if you don’t want to include maintenance mode for servers being shut down. You can use Parameter 5, which includes the shutdown type. Putting “restart” in as a parameter will only trigger the rule when a computer is rebooted and not shut down.

Using event parameters, and not just wildcard search the entire event description is to reduce the performance impact on the agent computer. Wan’t to know why, check Kevin Holman’s post about event detection.

Creating the Powershell script

During my research for maintenance mode automation, i have come across many PS script, some of them was written for OpsMgr 2007, which means that they won’t work ‘out of the box’. operatingquadrant was one of my resources, which had written something similar back in 2009.

This is how my script look now, not the most complex script out ther, but it does it job for now.

param($sHost) #Parameter for computer to MM. This is passed to the script from the Command Channel: $Data/Context/DataItem/ManagedEntityDisplayName$
$ServerName = "ScomMS.fqdn" #Your desired MS
Import-Module OperationsManager #Load opsmgr ps module
New-SCOMManagementGroupConnection -ComputerName $ServerName #connect to managment group
$Time = ((Get-Date)).AddMinutes(20) #Minutes maintenance should be active
$class = Get-SCOMClass -name "Microsoft.Windows.Computer" #query class
$computerObj = $class | get-scommonitoringobject | where {$_.name -like "$sHost*"} #Find object (computer) based on class
#======Event log config=====#
$logname = "Application"
$logsrc = "Maintenance Mode Script"
$eventid = "1010"
$eventlvl = "Information"
#======Event log config=====#

#====Run this to write new event source=====#
#New-EventLog -LogName $logname -Source $logsrc
#===========================================#
Write-EventLog -LogName $logname -Source $logsrc -EntryType $eventlvl -EventId $eventid -Message "The follwing objects where put in to MM $computerObj" #write the event log
Start-SCOMMaintenanceMode -Instance $computerObj -EndTime $Time -Reason PlannedOperatingSystemReconfiguration -Comment "MM started automaticly." #maintenance mode the object

 

This script also writes an event to the management servers application log, each time the script runs. To be able to write the events, you will have to add a new event source. If you don’t want to write these events, just comment out the whole part.

To register a new source run the following line in the script:

New-EventLog -LogName $logname -Source $logsrc

Without event log you should have a script like this

param($sHost) #Parameter for computer to MM. This is passed to the script from the Command Channel: $Data/Context/DataItem/ManagedEntityDisplayName$
$ServerName = "ScomMS.fqdn" #Your desired MS
Import-Module OperationsManager #Load opsmgr ps module
New-SCOMManagementGroupConnection -ComputerName $ServerName #connect to managment group
$Time = ((Get-Date)).AddMinutes(15) #Minutes maintenance should be active
$class = Get-SCOMClass -name "Microsoft.Windows.Computer" #query class
$computerObj = $class | get-scommonitoringobject | where {$_.name -like "$sHost*"} #Find object (computer) based on class

Start-SCOMMaintenanceMode -Instance $computerObj -EndTime $Time -Reason PlannedOperatingSystemReconfiguration -Comment "MM started automaticly." #maintenance mode the object

 

Command Channel

Operations Manager is perfectly capable of triggering a command/script when an alert is logged. I will show how a command channel for triggering the above PS script is set up. Be aware of command channel’s limitations regarding async. proc.

Command Channel setup
Command Channel setup

Our “AutoMaintenanceMode” command channel is set up as in this picture. the one thing you need to pay attention to, is how we pass the computer name to our PS script. In command line parameters put this after your script path $Data/Context/DataItem/ManagedEntityDisplayName$

Next is to set up our subscriber and subscription. 

Setting up a subscriber and a subscription for the subscriber is pretty straight forward. The first thing we will need is a Maintenance Mode subscriber. I suggest you name it so it’s easy to see that this is an automated process.

In the channel tab, choose command and then select your command channel.

Command Subscriber

Complete the wizard, and continue to set up a subscription. Again, naming your subscription makes it easy to understand.

subscription

 

Quickly, set op your criteria, here’s what we use. The only thing you actually need is “created by specific rule […]”

subcriteria

For your subscription and channel, you simply select the ones you created earlier. A summary page will look something like this

Name
Auto: Put Computers in MM when booted

Description

Criteria
Notify on all alerts where
created by Reboot initiated rules or monitors (e.g., sources)
and of a Information severity
and of a Low priority

Subscribers
Maintenance Mode

Channels
AutoMaintenanceMode

 

 

That’s all there is to it! If everything is correctly set up. Next time one of your colleagues reboot a computer, SCOM wil automaticly place that object in to Maintanance Mode.

And as always – always test before implementation

 

Footnote

When using a command channel to perform this operation you will be limited to “maximum number of asynchronous process” which defaults to 5. You will get an alert saying something with “script dropped” or Operations Manager failed to start a process due to lack of resources
What is actually happening, is that when the limit of 5 is reached OpsMgr is protecting it self from starvation in case of an alert storm etc. This limitation can be raised, but i suggest to wait and see if you actually have this problem.

Another way, is to drop the command channel entirely and use SMA, orchestrator etc. to catch the alert and run the powershell script.

Share this:

  • LinkedIn
  • Twitter
Operations Manager

SquaredUp v2.2 and community dashboards

  • 21/09/2015
  • by Martin Ehrnst

SquaredUp just released a new version which includes some great new features.

Page Scope – to speed up the creation of custom dashboards when displaying information for a single dataset (eg. SQL Computers, or a specific Distributed Application), you’re now able to set the scope for dashboards at the level of the entire page, rather than having to scope sections individually.

 

Clone Dashboards – to make it even easier for you to build custom dashboards, we’ve introduced the ability to clone an existing dashboard. In conjunction with Page Level Scope, this now means it’s easier than ever for you to create a template dashboard and then quickly replicate that for multiple different applications or infrastructure components.

 

Group Maintenance Mode – under the hood of 2.2, we’ve made some significant changes to the way that Squared Up works and that’s allowed us to introduce a cool new feature, Group Maintenance Mode, which allows you to place an entire Group into Maintenance Mode with a single click.

 

Revamped Status Plugin – we’ve updated the Status plugin to provide much better options for sorting and grouping objects, as well as significantly enhanced labelling options

Along with version 2.2 of SquaredUp, they released their new community dashboard site where you can download already configured dashboards and contribute to the rest of our community.

Share this:

  • LinkedIn
  • Twitter

Posts navigation

1 … 4 5 6 7 8

Top Posts & Pages

  • Azure AD authentication in Azure Functions
  • Creating Azure AD Application using Powershell
  • Access to Blob storage using Managed Identity in Logic Apps - by Nadeem Ahamed
  • Multi subscription deployment with DevOps and Azure Lighthouse
  • Using Azure pipelines to deploy ARM templates
  • Working with Azure Monitor Rest API
  • SCOMpercentageCPUTimeCounter cause CPU Spike
  • Resource health through Azure Rest API
  • SCOM Alerts to Microsoft Teams and Mattermost
  • Web API for System Center Operations Manager

Tags

agent announcements api ARM authoring Automation Azure AzureAD AzureCloudShell AzureFunctions AzureLighthouse AzureMonitor AzureSpringClean Bicep Community CSP database EventGrid ExpertsLive ExpertsLiveEU IaC Infrastructure as code Integrations LogAnalytics management pack monitoring MSIgnite MSIgnite2017 MSOMS MSP nicconf OperationsManager OpsMgr Powershell QUickPublish rest SCDPM SCOM SCOM2016 SCVMM Serverless SquaredUP SysCtr system center Webasto

Follow Martin Ehrnst

  • Twitter
  • LinkedIn

RSS Feed RSS - Posts

RSS Feed RSS - Comments

Microsoft Azure MVP

Martin Ehrnst Microsoft Azure MVP

NiCE Active 365 Monitor for Azure

NiCE active 365 monitor for Azure
Adatum.no use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it. Cookie Policy
Theme by Colorlib Powered by WordPress
adatum
Proudly powered by WordPress Theme: Shapely.