SCOM 2016 - Deployed Agent on Domain Controller Grayed Out

So I have been going through the process of setting up SCOM 2016 and came across an interesting issue. I deployed the SCOM agent to my Domain Controller using the install as other account method which, in my opinion is the preferred method. Agent installed successfully and reported back to SCOM as healthy. A short time later it went gray on me.

OK, this happens sometimes. I ran the repair using the same account (maybe I fat fingered it). Repair was successful but agent was still gray. I did an uninstall and re-install, watching it the whole time. Came back as green then a short time later it was gray again. Checked on the domain controller. Agent was running and pointing to the correct management group.

After some digging I came across this KB article which explains what is happening. In short when you use the install as other account method the service itself does not actually run using the Domain Level account specified. It only uses that account to do the physical installation and configuration. Once installed the service runs as Local System or NT Authority\System. At some time between 2012 and 2016 the security approach to agent accounts changed an according to the article "only the NT AUTHORITY\Authenticated Users security principal is allowed access to the Health Service. But when the Active Directory is hardened, or the agent is misconfigured, the Local System account cannot authenticate through the Authenticated Users security principal, therefore the agent cannot process Health Service configuration information."

So at this point there are two ways to fix this. Create a Run-As Account and deploy that to all your domain controllers or use the HSLockdown tool. I only have one DC in my lab so I opted to do the latter.

On your Domain Controller navigate to C:\Program Files\Microsoft Monitoring Agent\ Shift-Right Click on the Agent folder and select Open Command Window Here. Type hslockdown.exe /l this will list all users authenticated to access the management group.

Voila! Access has been denied to NT Authority\System. So we need to grant access to it in order to fix this. Type hslockdown.exe /a "NT Authority\System" 

You will be prompted to restart the Health Service in order for the change to take effect.

Give it a few minutes to start reporting in again, and there you go!

It's strange that I have installed the agent on literally hundreds of Domain Controllers and never had to do this before... Odd...

More to come!

If you like this blog, give it a g+1