Threadpool Work item priority

Download source code from MSDN Code Gallery.

In the previous post, I talked about how to queue work items to the process-wide system threadpool using TrySubmitThreadpoolCallback. In this post, I will talk about work item priority and how to queue low priority work items. 

The thread that your work item executes is not owned by your application; it’s owned and managed by the system. That implies the application should not modify the state because it has no visibility into other work items that might be scheduled to run on this thread. Depending on how the state was modified, other work items may not react well.

One of the most common ways developers modify thread state is by lowering the thread priority in the work item. This is a no-no and has unintended consequences, even if you reset the thread’s priority before returning from your work item.

Here’s why: In Windows 7, the threadpool uses something called a concurrency control to control the number of concurrently running threads. This number is typically equal to the number of cores on the machine; if a machine has two cores, then the threadpool will have a concurrency control number of 2. This seems quite logical, if you have two cores, only two threads can execute code. Note: If a work item blocks for any reason, then a new thread is created by the threadpool to execute other work items so all cores stay busy.

However, if the work item lowers the priority of the thread below normal and there is another normal priority thread in the system which is ready to run, then the Windows scheduler schedules the normal priority thread and preempts the lower priority threadpool work item thread. This is bad because the threadpool sees that the thread whose priority has been lowered is still in a ready state. A thread in ready state counts towards your concurrently running threads, because technically it could run as soon as it is scheduled. Therefore, the threadpool thinks it has two running threads in the system. However, only one thread is actually running, so work items in the threadpool will get starved. This is especially egregious when it happens in the shared process-wide threadpool which has system work items, because it could impact system-wide performance.

Note: if you increase the thread priority in the callback, then it may not starve the other work items the same way as theoretically the callback should now finish quicker. However, this is still not recommended especially if you are in the process-wide system threadpool as the thread is not yours to modify. Plus if everyone decides to boost their priority this way then work items will be starved and the system performance will be impacted.

This is not a theoretical concern and is quite real based on our experience. If you have any work items that need to execute at lower priority, there are two alternatives:

1. Create your own private threadpool which services only your work items and set their priorities as needed.  If these work items are starved or executed slowly the Windows system is not affected. Later in the series I will show how to use the wrapper class windowsthreadpool::PrivateThreadPool to do exactly this.
2. Use the threadpool function SetThreadpoolCallbackPriority to set the priority on a work item. This function does not modify thread priorities; instead it uses separate queues for low, high and normal priority work items. The MSDN documentation hints at this:

https://msdn.microsoft.com/en-us/library/dd405519(VS.85).aspx

Remarks

Higher priority callbacks are guaranteed to be run first by the first available worker thread, but they are not guaranteed to finish before lower priority callbacks.

The QueueUserWorkItemWithHiPri and QueueUserWorkItemWithLowPri methods in the SimpleThreadPool class use the SetThreadpoolCallbackPriority function to change work item priority. We create a new environment structure, initialize the structure, set the appropriate priority and associate it with our cleanup group so that we can wait on the work items using the WaitForAll method.

template <class Function>

bool QueueUserWorkItemWithLowPri(Function cb)

{

       InitializeInfra();

       TP_CALLBACK_ENVIRON LowPriCallbackEnvironment;

       InitializeThreadpoolEnvironment(&LowPriCallbackEnvironment);

       SetThreadpoolCallbackPriority(&LowPriCallbackEnvironment, TP_CALLBACK_PRIORITY_LOW);

       SetThreadpoolCallbackCleanupGroup(&LowPriCallbackEnvironment, CleanupGroup, NULL);

       return QueueUserWorkItemInternal(cb, NULL, &LowPriCallbackEnvironment);

}

template <class Function>

bool QueueUserWorkItemWithHiPri(Function cb)

{

       InitializeInfra();

       TP_CALLBACK_ENVIRON HiPriCallbackEnvironment;

       InitializeThreadpoolEnvironment(&HiPriCallbackEnvironment);

       SetThreadpoolCallbackPriority(&HiPriCallbackEnvironment, TP_CALLBACK_PRIORITY_HIGH);

       SetThreadpoolCallbackCleanupGroup(&HiPriCallbackEnvironment, CleanupGroup, NULL);

       return QueueUserWorkItemInternal(cb, NULL, &HiPriCallbackEnvironment);

}

Next up, how to create your own private threadpool…