SCVMM service may consume high memory or CPU utilization

  • Article

Symptoms

In System Center Virtual Machine Manager 2008, if the Job Count exceeds the recommended upper limit then the Virtual Machine Manager service may consume a large amount of memory or exhibit high CPU utilization.

Cause

This issue may occur if the Job Count exceeds the recommended limits.

The maximum number of running jobs that an SCVMM system should be able to handle is around 10,000 and hitting this many running jobs at one time would be an exceptionally rare occurrence. Should it occur, however, it can have a severe impact on the performance of the SCVMM engine.

The number of retained Jobs count refers to the number of running tasks tracked in the TaskTrail database table in SQL. The size of this database can have as much of an impact on performance as the actual running jobs. If the number of running jobs nowhere reaches 10,000 then it may be that the tombstone limit on entries tracked in the TaskTrail database is too conservative

By default the TaskGC entry in HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Microsoft System Center Virtual Machine Manager Server/Settings/SQL is set to tombstone entries at 90 day intervals. If a user is running a number of jobs on a daily basis that would cause the TaskTrail database to exceed this tolerance then this number can be reduced to tombstone entries more often.

By default, a job runs every 20 hours that will read this value and start the process to remove the jobs above the upper recommended limit, however it may take several cycles to bring the total job count down to acceptable limits.

Resolution

  Add the TaskGC registry value to limit the number of days that VMM retains job historical information.

  1. Navigate to the following registry entry on your VMM server

                  HKEY_LOCAL_MACHINE/SOFTWARE/Microsoft/Microsoft System Center Virtual Machine Manager Server/Settings/Sql

    2.    Create a DWORD value named "TaskGC" and set its value to a number less than 90.

                 (TaskGC value controls the number of days that the events are stored in the DB)

    3.    Restart the VMM server several times to remove jobs until the total drops below 10,000

 

Alternativly, you may run SQL Manager directly and run the following commands multiple times, until the execution does not find any more objects.

DECLARE @CutoffDateTime datetime
SELECT @= '4/1/2009'
EXECUTE [VirtualManagerDB].[dbo].prc_TR_GarbageCollect @CutoffDateTime

Note: you need to adjust CutoffDateTime to a date for your desired history length

For immediate cleanup, run the following SQL commands.

delete from [VirtualManagerDB].[dbo].[tbl_TR_DeletedObject]

delete from [VirtualManagerDB].[dbo].[tbl_TR_AuditTrail]

delete from [VirtualManagerDB].[dbo].[tbl_TR_SubtaskTrail]

delete from [VirtualManagerDB].[dbo].[tbl_TR_TaskTrail]

Remember to perform a full backup of the VMM database before performing any of these actions!

Comments

  • Anonymous
    March 25, 2011
    Hi, thanks for the article. Can you please specify if the number of days (whitin the DWORD value) should be in Hex or Decimal? Thanks in advance