Throttling, DSCP, and 802.1p with Traffic Control

  • Article

In his introductory post about the legacy Traffic Control (TC) API, Gabe discussed the host-based model that TC provides. In this post, we will see how Traffic Control APIs can be used to achieve the following for TCP/IPv4 and UDP/IPv4 traffic sent from a host:

  • Throttle (rate-limit) outgoing traffic
  • Add DSCP value in layer-3 (IPv4) header
  • Indicate that an 802.1p tag value should be added in layer-2 header

At the bottom of the post, we’ve provided a link to the Networking Connect site to download full source and binaries to a tool which implements all of the above functionality.

The following are the steps involved:

1. The first step in the process is to obtain a handle to the Traffic Control subsystem through a call to TcRegisterClient() .

2. Next, make a call to TcEnumerateInterfaces() using the registration handle obtained in step #1. This call returns a list of all TC enabled interfaces on the system. Iterate through the list to find the interface(s) on which you want to prioritize and/or throttle outgoing traffic.

3. For each interface of interest, issue a TcOpenInterface() using the pInterfaceName from the corresponding TC_IFC_DESCRIPTOR for that interface from the list returned in step#2. Store the handle returned by TcOpenInterface() ; let’s call it the ifchandle.

4. At this point, create a TC Flow and add it to the interface(s) of interest.

a. Create the TC Flow:
A TC flow is a way of describing various QoS characteristics to be applied to a set of packets and is represented by a TC_GEN_FLOW structure. The following code snippet shows how to create a TC Flow given the DSCP value, 802.1p value and the throttle rate.

BOOL CreateFlow(PTC_GEN_FLOW * _ppTcFlowObj, USHORT DSCPValue, USHORT OnePValue, ULONG ThrottleRate)
{
BOOL status = FALSE;

  //
// Flow Parameters
//
ULONG TokenRate = QOS_NOT_SPECIFIED;
ULONG TokenBucketSize = QOS_NOT_SPECIFIED;
ULONG PeakBandwidth = QOS_NOT_SPECIFIED;
ULONG Latency = QOS_NOT_SPECIFIED;
ULONG DelayVariation = QOS_NOT_SPECIFIED;
SERVICETYPE ServiceType = SERVICETYPE_BESTEFFORT;
ULONG MaxSduSize=QOS_NOT_SPECIFIED;
ULONG MinimumPolicedSize=QOS_NOT_SPECIFIED;
PVOID pCurrentObject;
PTC_GEN_FLOW _pTcFlowObj = NULL;
int Length = 0;

  //
// Calculate the memory size required for the optional TC objects
//

  Length += (OnePValue == NOT_SPECIFIED ? 0:sizeof(QOS_TRAFFIC_CLASS)) + (DSCPValue == NOT_SPECIFIED ? 0:sizeof(QOS_DS_CLASS));

  //
// Print the Flow parameters
//

  printf("Flow Parameters:\n");
DSCPValue == NOT_SPECIFIED ? printf("\tDSCP: *\n"):printf("\tDSCP: %u\n", DSCPValue);
OnePValue == NOT_SPECIFIED ? printf("\t802.1p: *\n"):printf("\t802.1p: %u\n", OnePValue);
ThrottleRate == QOS_NOT_SPECIFIED ? printf("\tThrottleRate: *\n"):printf("\tThrottleRate: %u\n", ThrottleRate);
TokenRate = TokenBucketSize = ThrottleRate;

  //
// Allocate the flow descriptor
//
_pTcFlowObj = (PTC_GEN_FLOW)malloc(FIELD_OFFSET(TC_GEN_FLOW, TcObjects) + Length);

  if (!_pTcFlowObj)
{
printf("Flow Allocation Failed\n");
goto Exit;
}

  _pTcFlowObj->SendingFlowspec.TokenRate = TokenRate;
_pTcFlowObj->SendingFlowspec.TokenBucketSize = TokenBucketSize;
_pTcFlowObj->SendingFlowspec.PeakBandwidth = PeakBandwidth;
_pTcFlowObj->SendingFlowspec.Latency = Latency;
_pTcFlowObj->SendingFlowspec.DelayVariation = DelayVariation;
_pTcFlowObj->SendingFlowspec.ServiceType = ServiceType;
_pTcFlowObj->SendingFlowspec.MaxSduSize = MaxSduSize;
_pTcFlowObj->SendingFlowspec.MinimumPolicedSize = MinimumPolicedSize;

  //
// Currently TC only supports QoS on the send path
// ReceivingFlowSpec is legacy and ignored
//

  memcpy(&(_pTcFlowObj->ReceivingFlowspec), &(_pTcFlowObj->SendingFlowspec), sizeof(_pTcFlowObj->ReceivingFlowspec));
_pTcFlowObj->TcObjectsLength = Length;

  //
// Add any requested objects
//
pCurrentObject = (PVOID)_pTcFlowObj->TcObjects;
if(OnePValue != NOT_SPECIFIED)
{
QOS_TRAFFIC_CLASS *pTClassObject = (QOS_TRAFFIC_CLASS*)pCurrentObject;
pTClassObject->ObjectHdr.ObjectType = QOS_OBJECT_TRAFFIC_CLASS;
pTClassObject->ObjectHdr.ObjectLength = sizeof(QOS_TRAFFIC_CLASS);
pTClassObject->TrafficClass = OnePValue; //802.1p tag to be used
pCurrentObject = (PVOID)(pTClassObject + 1);
}

  if(DSCPValue != NOT_SPECIFIED)
{
QOS_DS_CLASS *pDSClassObject = (QOS_DS_CLASS*)pCurrentObject;
pDSClassObject->ObjectHdr.ObjectType = QOS_OBJECT_DS_CLASS;
pDSClassObject->ObjectHdr.ObjectLength = sizeof(QOS_DS_CLASS);
pDSClassObject->DSField = DSCPValue; //Services Type
}

  DeleteFlow(_ppTcFlowObj);
*_ppTcFlowObj = _pTcFlowObj;
status = TRUE;
Exit:

  if(!status)
{
printf("Flow Creation Failed\n");
DeleteFlow(&_pTcFlowObj);
}
else
printf("Flow Creation Succeeded\n");

  return status;
}

b. Add the Flow on the interface:
After obtaining a TC_GEN_FLOW structure with the desired characteristics using a function similar to the one above, issue a call to TcAddFlow() with the ifchandle (obtained in step #3) and a pointer to the TC_GEN_FLOW object (obtained in step #4a). Store the handle returned by TcAddFlow() ;let’s call it the flowhandle.

5. The next step is to create a TC Filter and add it to the TC Flow created above.

a. Create the TC Filter:
A TC Filter is a way of describing which packets to apply the QoS characteristics to. The QoS characteristics defined in the TC_GEN_FLOW will only apply to the packets matching the filter(s) associated with the Flow.
The following code snippet describes how to create a TC Filter given the destination address, the destination port, and the protocol (TCP,UDP or IP).

BOOL CreateFilter(PTC_GEN_FILTER * ppFilter, SOCKADDR_STORAGE Address, USHORT Port, UCHAR ProtocolId)
{

  BOOL status = FALSE;
USHORT AddressFamily = Address.ss_family;
PTC_GEN_FILTER pFilter = NULL;
PIP_PATTERN pPattern = NULL;
PIP_PATTERN pMask = NULL;

  if(AddressFamily != AF_INET)
goto Exit;

  //
// Allocate memory for the filter
//
pFilter = (PTC_GEN_FILTER)malloc(sizeof (TC_GEN_FILTER));
if(!pFilter)
goto Exit;

  ZeroMemory(pFilter, sizeof(TC_GEN_FILTER));

  //
// Allocate memory for the pattern and mask
//
pPattern = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
pMask = (PIP_PATTERN)malloc( sizeof(IP_PATTERN));
if(!pPattern || !pMask)
goto Exit;

  memset ( pPattern, 0, sizeof(IP_PATTERN) );
pPattern->DstAddr = ((SOCKADDR_IN *)&Address)->sin_addr.s_addr;
pPattern->tcDstPort = htons(Port);
pPattern->ProtocolId = ProtocolId;
memset ( pMask, (ULONG) -1, sizeof(IP_PATTERN) );

  //
// Set the source address and port to wildcard
// 0 -> wildcard, 0xFF-> exact match
//
pMask->SrcAddr = 0;
pMask->tcSrcPort = 0;

  //
// if the user specified 0 for dest port, dest address or protocol
// set the appropriate mask as wildcard
// 0 -> wildcard, 0xFF-> exact match
//
if(pPattern->tcDstPort == 0)
pMask->tcDstPort = 0;

  if(pPattern->ProtocolId == 0)
pMask->ProtocolId = 0;

  if(pPattern->DstAddr == 0)
pMask->DstAddr = 0;

  pFilter->AddressType = NDIS_PROTOCOL_ID_TCP_IP;
pFilter->PatternSize = sizeof(IP_PATTERN);
pFilter->Pattern = pPattern;
pFilter->Mask = pMask;

  //
// Delete any previous instances of the Filter
//
DeleteFilter(ppFilter);
*ppFilter = pFilter;
status = TRUE;

  Exit:
if(!status)
{
printf("Filter Creation Failed\n");
DeleteFilter(&pFilter);
}
else
printf("Filter Creation Succeeded\n");

  return status;
}

b. Adding the Filter to the TC Flow:
Once a TC Filter structure is obtained using a function similar to the one above, issue a call to TcAddFilter() passing the flowhandle obtained in step #4b and a pointer to the TC_GEN_FILTER structure obtained in step #5a. Store the filter handle returned by TcAddFilter() ;let’s call it filterhandle.
You can add multiple filters on the same flow causing different sets of packets matching each filter to get the same QoS characteristics applied to them.

6. At this point, your application is applying QoS on all matching outgoing packets as specified in the TC Filter and TC Flow. Finally, once your purpose is served, make sure you call the respective close calls on all the open TC handles – TcDeletefilter() , TcDeleteFlow() , TcCloseInterface() and TcDeregisterClient() .

You can download the full source and binaries of a simple command line tool – tcmonlite, which takes the filter and flow parameters as input, creates TC Flow and Filter, and configures the QoS subsystem with them using the Traffic Control API. All the outgoing traffic on the system matching the filter gets the desired QoS characteristics as long as tcmonlite is running. Go to the Microsoft Connect website, choose Available Connections on the left-hand side of the page, and select Windows Networking from the available connections (bottom half of the page). On the left-hand side of the Windows Networking page, choose Downloads, and select TCMonLite.

This tool can be used in conjunction with the NDIS LWF driver to detect 802.1p tags in the Ethernet header and DSCP in the IP header of packets. Let us know what you think!

-- Hemant Banavar

Comments

  • Anonymous
    November 14, 2007
    This is great, but how about network connections with intrinsic QoS -  Will you support technologies that "offload" both the classification/tagging and throttling functionality from the OS?

  • Anonymous
    November 15, 2007
    I'm not certain what "intrinsic" QoS means? Perhaps you are refering to a fabric that intermediate elements already support strict priorities (e.g. switches or routers)? If you're more specific, I'm happy to comment. As for offloading classification/tagging and throttling functionality from the OS; I'd love to hear your scenarios for why offload would be valuable. We recently explored supporting this very thing, but found the savings (CPU cycles) were negledgeable even for 1Gbps link saturation on average server class machines. The QoS platform on Windows is extremely efficient for very large workloads. It does sound like a blog post is in order that describes, architecturally, how the QoS subystem works. -- Gabe Frost

  • Anonymous
    November 22, 2007
    The comment has been removed

  • Anonymous
    November 27, 2007
    The comment has been removed

  • Anonymous
    December 05, 2007
    The comment has been removed

  • Anonymous
    March 04, 2008
    Hi Gabe There are two things that I'm trying to confirm and would very much appreciate your comment on:

  1. It is my understanding that we can use group policy (GPO) to set RTA (audio payload) to expedited forwarding (EF) and RTA (signaling) to assured forwarding (AF31) and make this happen from a centralized location. Is this correct?
  2. Assuming that (1) above is correct, is it also true that in Windows Vista we can BLOCK any Windows Vista compliant AND non-compliant programs from changing these markings? Basically I'm trying to understand if I have an enterprise with users logged into the domain, do I have complete control over the DSCP marking of packets leaving the PC? Thanks, Dan

  • Anonymous
    July 23, 2008
    Hi Gabe There is just one things that I'm trying to confirm and would very much appreciate your comment on: Can the TC api work on a router and mark the packets  forwarded by the router? I'm trying to use the TC api to mark the packets forwarded by the router,but i doesn't work! Can somebody give me some advice? Thanks

  • Anonymous
    August 15, 2008
    The comment has been removed

  • Anonymous
    October 28, 2008
    The comment has been removed

  • Anonymous
    October 30, 2008
    Hi Martin, Thanks for your question. Yes, you'll be able to set an arbitrary DSCP value for your application using QOSSetFlow with a new QOS_SET_FLOW type we've introduced in Windows 7. The requirements are

  1. the flow your application creates must be non-adaptive; and
  2. your application is a member of the Administrators group or the Network Configuration Operators group. Please let us know if this answers your question. We're also interested in the use cases of your application as you said your application would adapt to 11 different DSCP values. Would you mind sharing a bit more details of those scenarios? thanks

  • Anonymous
    October 31, 2008
    As the "net" guy for a video based developing application, mobile videocon, we'll say, I am wondering if a windows based application could incorporate this type of DSCP marking and if only data from that application would do such marking - or if the PC interface will always mark packets once installed.   I do not want to effect the way other applications transmit if at all possible. I am also looking for a way to change the tags if needed by an appliance before interfacing with the network.  Any input would be greatly appreciated. Thank you in advance!

  • Anonymous
    November 03, 2008
    The comment has been removed

  • Anonymous
    November 04, 2008
    The comment has been removed

  • Anonymous
    November 12, 2008
    Hi Martin, Thanks for the info. Because your application has separate sockets for signaling, media transmission and so on, you'll be able to set a different DSCP value for each of these socket connections using "QOSAddSocketToFlow" and "QOSSetFlow". Only the packets sent to those sockets will be marked. However, the requirement that your application must have the admin or network configuration operator privilege still holds. One possible way to work around it is to write your own miniport driver and mark your application packets there. thanks

  • Anonymous
    February 10, 2009
    Hi and Thanks, but i don't know well C++, can anyone have a sample (TC API) for Visual Basic 6... I want to create a small utility for managing traffic (upload/download) for my local network. Thanks in advance :-)

  • Anonymous
    April 12, 2009
    Hi Sasha, I am using wince 5.0 with wifi card. I want to set QOS enabled for WMM ie MULTIMEDIA Data, I used setsockopt() but Microsoft says an issue http://support.microsoft.com/kb/248611 after this I switched to QOS structure I fulfilled the structure ie defined it. Now what's next to do for enabling QOS / DSCP for WMM?

  • Anonymous
    April 14, 2009
    The comment has been removed

  • Anonymous
    April 16, 2009
    Hi Sarforsh, I never tried it on Win CE 5.0, but here is a though. Is the dscp variable defined as an int? -- Martin

  • Anonymous
    November 19, 2009
    From a comment above: "Yes, you'll be able to set an arbitrary DSCP value for your application using QOSSetFlow with a new QOS_SET_FLOW type we've introduced in Windows 7. The requirements are" did this really happen? The QOS_SET_FLOW seems to indicate that I have to pass in an enum for the traffic type and not a hex value. The enum does not provide a method for setting the EF DSCP markings. This will need to be done without have a domain. Any suggestions? David

  • Anonymous
    December 22, 2009
    Can't we use SetSocketOption ??

  • Anonymous
    January 12, 2010
    hello, i'm computer cience student, i wanted to develop a project on Qos implementation on windows.i mean i want apply Qos on differnt kind of data ie.multimedia data for a health information system. i want to use C++.please can you guide me,for  from where  should i start? i wanted to implement Diffserv. i have seen the TC API at your site,QOS at MSDN libary. can i get ready made code for the implementation of Qos techniques in C++?     so that i can use them as refernce for the development of my project. regards vishal

  • Anonymous
    March 31, 2010
    Hello, I'm exploring QOS capabilties in Windows networking. After reading the inital msdn documentation, I came across that with Winsock2, programmers are provided access to advanced Microsoft® Windows® networking capabilities such as multicast and Quality of Service (QOS). I've query whether it is possible to have QOS capabilities along with "WinHttp" ?? I'm sorry if my query is vague !! Regards Deepu

  • Anonymous
    May 27, 2010
    The comment has been removed

  • Anonymous
    June 17, 2010
    Hello,            Are there ways to call the qWAVE APIs from the kernel in a transport driver based on WinSock kernel? Thanks for your help!

  • Anonymous
    April 01, 2011
    Got this working on Windows XP after following instructions here: technet.microsoft.com/.../2007.02.cableguy.aspx I wasn't able to set the DSCP value until I made the registry change to HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesTcpipParametersDisableUserTOSSetting=0 Thanks for such a great tool!!!! Adam :)

  • Anonymous
    May 31, 2011
    Thank you for the article. In the meantime, in order to run this application, I think I need to create a policy-based QOS rule from Group Policy Editor. As a network application tester, I'm looking for a script or application to create a new policy. Is any one who can share the script or application? Thank you. ;-)

  • Anonymous
    June 03, 2011
    What I want to do is testing DSCP value is changing when use tcmonlite for transmission of UDP packet. Host A: W2K8 R2 Ent. (10.0.100.200) Host B: W2K8 R2 Ent. (10.0.100.4) They are connected back-to-back While sending an UDP packet from A to B's port 9999, Host B confirms that it receives the UDP packet correctly. Then, run tcmonlite.exe from Host A tcmonlite -proto udp -destip 10.0.100.4 -destport 9999 -dscp 40 -onep 4 -throttle 100000 Wireshark from Host A capture the udp packet and still DSCP value is x00 not x28. Could you explain why? What I did miss? Thank you, spark.