Session "" failed to start with the following error: 0xC0000022

Problem

The event log fills up with Event ID 2 from Kernel-EventTracing stating Session “” failed to start with the following error: 0xC0000022.

image

Analysis

If you look into the system data for one of the events, you will find the associated ProcessID and ThreadID:

image

If the event is relatively current, the Process ID  should still be registered by the offending process. Open Process Explorer and list processes by PID:

image

We can clearly see that the culprit is one of those pesky WMI-processes. The ThreadID is a lot more fluctuating than the ProcessID, but we can always take a chance and se if it will reveal more data. I spent a few minutes writing this, and in that time it had already disappeared. I waited for another event, and immediately went to process explorer to look for thread 18932. Sadly though, this didn’t do me any good. For someone more versed in kernel API calls the data might make some sense, but not to me.

image

I had more luck rummaging around in the ad-profile generator (google search). It pointed me in the direction of KB3087042. It talks about WMI calls to the LBFO teaming (Windows 2012 native network teaming) and conflicts with third-party WMI providers. Some more digging around indicated that the third-party WMI provider in question is HP WBEM. HP WBEM is a piece of software used on HP servers to facilitate centralized server management (HP Insight). As KB3087042 states the third-party provider is not the culprit. That implies a fault in Windows itself, but one must not admit such things publicly of course.

In their infinite wisdom (or as an attempt to compensate for their lack thereof), the good people of Microsoft has also provided a manual workaround for the issue. It is a bit difficult to understand, so I will provide my own version below.

Solution

As usual, if the following looks to you as something that belongs in a Harry Potter charms class, please seek assistance before you implement this in production. You will be messing with central operating system files, and a slip of the hand may very well end up with a defective server. You have been warned.

The fix

But let us get on with the fix. First, you have to get yourself an administrative command prompt. The good old fashioned black cmd.exe (or any of the 16 available colors). There is no reason why this would not work in one of those fancy new blue PowerShell thingy’s as well, but why take unnecessary risks?

Then, we have a list of four incantations – uh.., commands to run through. Be aware that if for some reason your system drive is not C:, you will have to take that into account. And then spend five hours repenting and trying to come up with a good excuse for why you did it in the first place. Or perhaps spend the time looking for the person who did it and give them a good talking to. But I digress. The commands to run from the administrative command prompt are as follows:

Takeown /f c:\windows\inf
icacls c:\windows\inf /grant “NT AUTHORITY\NETWORK SERVICE”:”(OI)(CI)(F)”
icacls c:\windows\inf\netcfgx.0.etl /grant “NT AUTHORITY\NETWORK SERVICE”:F
icacls c:\windows\inf\netcfgx.1.etl /grant “NT AUTHORITY\NETWORK SERVICE”:F

The first command takes ownership of the Windows\Inf folder. This is done to make sure that you are able to make the changes. The three icacls-commands grants permissions to the NETWORK SERVICE system account on the INF-folder and two ETL-files. The result should look something like this:

SNAGHTML46207857

To test if you were successful, run this command:

icacls c:\windows\inf

And look for the highlighted result:

image

Should you want to learn more about the icacls command, this is a good starting point.

The cleanup

This point is very important. If you do not hand over ownership of Windows\Inf back to the system, bad things will happen in your life.

This time, you only need a normal file explorer window. Open it, and navigate to C:\Windows. Then open the advanced security dialog for the folder.

Next to the name of the current owner (should be your account) click the change button/link.

SNAGHTML4628dae2

Then, select the Local Computer as location and NT SERVICE\TrustedInstaller as object name. Click Check Names to make sure you entered everything correctly. If you did, the object name changes to TrustedInstaller (underlined).

image

Click OK twice to get back to the file explorer window. If you did not get any error messages, you are done.

It IS possible to script the ownership transfer as well, but in my experience the failure rate is way to high. I guess the writers of the KB agrees, as they have only given a manual approach.

The system event log is bloated with WMI Performance Adapter messages

Problem

A couple of times each minute, the WMI Performance Adapter Service is started and stopped, resulting in an informational message in the system event log (event 7036 from System Control Manager to be exact). This not only fills the log, but also causes pressure on the system due to the constant starting and stopping of the service. I have yet to see this issue on Win2008R2, but I have read reports from others: http://serverfault.com/questions/108829/why-is-my-system-event-log-full-of-wmi-performance-adapter-messages. Most of my Win 2012 servers exhibit the issue, but for some reason my 2008R2 servers have been spared.

image

Analysis

The root cause of this is usually SCOM, Splunk or similar agents who are collecting performance data from the server. The issue is not a problem per se, it is just a result of the fact that the monitoring agents are running a WMI query now and then. The problem is with log readability, it can mask other errors and let them slide out of the event log “window”, that is the amount of data the event log is allowed to contain at any point in time. I had a 20MiB max log size on one server, and it was only able to hold log data for about four days.

image

Solution

The solution is quite simple, you just have to set the startup type for the WMI Performance Adapter Service to Automatic:

image

Thus, you ensure that the service is kept running instead of restarting every 5 seconds. I have yet to see any adverse effects of this so far, but all the servers I have tested this on are physical database servers with tons of resources. The Wmi Performance Adapter service (wmiapsrv.exe) is only using about 7MB of ram on my servers. The Wmi provider host, that is also heavily utilized by SCOM/SPLUNK, is much more of a resource hog:

image