Why does my Avg Disk Write / Sec counter keep climbing?

There have been a number of issues with time in W2k3 guests, the first of which was covered in https://blogs.msdn.com/tvoellm/archive/2008/06/05/negative-ping-times-in-windows-vm-s-whats-up.aspx .  The first issue involved negative ping times.

A recent issue has come up around performance counters reporting huge disk write times that did not correspond with how the Virtual Machines was actually behaving.  it took quite a while to repro the issue because it did not always happen.  It turned that that the key to reproducing the strange increasing of times in the performance counters was due to the Virtual Machine (VM) having two virtual processors.

In Hyper-V there is guarantee that on a given VP the TSC register will never go backwards.  The TSC is basically a CPU cycle counter used for precise timing.  New operating system like WS08 are aware of Hyper-V and make better timing calculation and use different sources of time.

So why does the TSC matter?  The problem is the performance counter can read the start value of an operation on one VP and the end value on a different VP.  Since the counter collection also includes collecting the time it might actually go negative causing for example the Avg Disk Write / Sec to continue to increase until the timing delta used actually rolls over.  You can see the behavior in the graph below;

image

The solution is the same as the problem with negative ping time and that is to use the “/usepmtimer” in the boot.ini file.  This causes the performance counters to use the PM timer rather than the TSC.  the PM timer is consistent across all processor albeit slower to read that TSC.  The TSC does not cause a switch into the Hypervisor where the PM timer does.

Comments

  • Anonymous
    February 19, 2009
    If you're running Windows Server 2003 in a Hyper-V VM and have the VM configured to use multiple virtual

  • Anonymous
    November 06, 2009
    I don not agree with you, using the “/usepmtimer” in the boot.ini file would reduce the unexpected counters but still have Avg Disk Write / Sec counter keep climbing. When you monitor performance from inside the virtual machine, you only have access to performance counters that are provided by the guest machine. Therefore, utilization counters are only relative to the virtual object that is being monitored. In this case, counters will sometimes report more physical resource usage than actually exists. Check this KB http://support.microsoft.com/kb/967124/