SCOM 2012 - Agent Causing High CPU Utilization

So I ran into an interesting situation not too long ago. I was just wrapping up a deployment when one of the people I was working with came to me with a performance issue. He had noticed that there were a few of the servers being monitored by SCOM that were experiencing an issue with high CPU utilization. The CPU was flapping about every five minutes or so as shown in the perf-mon in figure 1. I was a bit surprised to hear this because everything I had read up to this point about SCOM 2012 had indicated that Microsoft had slimmed down the agent profile to run thinner and lighter then its predecessors.

Fig. 1

After some discussion I asked him to disable the SCOM agent and do a follow up perf-mon and the results are below in figure 2. The CPU went from a flapping state to almost null.

Fig. 2

This was quite strange. I had to do some digging to see if anyone else had come across the same thing. As it turns out there have been some an update to the BaseOS MP to run utilization scripts. One of the things that changed in this new management pack was the addition of a script which runs to provide an output on network utilization. There are three monitors and three rules for 2003 and 2008 that use this new script.

2003

Monitors

•  Microsoft.Windows.Server.2003.NetworkAdapter.PercentBandwidthUsedReads (Percent Bandwidth Used Read)
•  Microsoft.Windows.Server.2003.NetworkAdapter.PercentBandwidthUsedWrites (Percent Bandwidth Used Write)
• Microsoft.Windows.Server.2003.NetworkAdapter.PercentBandwidthUsedTotal (Percent Bandwidth Used Total)

Rules

• Microsoft.Windows.Server.2003.NetworkAdapter.PercentBandwidthUsedReads.Collection (Percent Bandwidth Used Read)
•  Microsoft.Windows.Server.2003.NetworkAdapter.PercentBandwidthUsedWrites.Collection (Percent Bandwidth Used Write)
• Microsoft.Windows.Server.2003.NetworkAdapter.PercentBandwidthUsedTotal.Collection (Percent Bandwidth Used Total)

2008

Monitors

•  Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedReads (Percent Bandwidth Used Read) 
•  Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedWrites (Percent Bandwidth Used Write) 
• Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedTotal (Percent Bandwidth Used Total)

 Rules

•  Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedReads.Collection (Percent Bandwidth Used Read)
•  Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedWrites.Collection (Percent Bandwidth Used Write)
•  Microsoft.Windows.Server.2008.NetworkAdapter.PercentBandwidthUsedTotal.Collection (Percent Bandwidth Used Total)

The advantage to having this run is it allows you insight into what the network bandwidth is, but the downside is when the script runs, every five minutes by default, it consumes a great amount of CPU. The best approach is to disable the monitor and rule for Percent Bandwidth Used Total in each 2003 and 2008. Read and write are disabled by default. For more information on how to disable these rules / monitors review SCOM 2012 - Creating Overrides.


More to come!

If you like this blog give it a g+1