Discovering the Underlying Office Protocols Using Microsoft Message Analyzer

Back in September of 2014, Microsoft announced the release of a new network analysis tool, Message Analyzer (MA). According to the release announcement, MA offers developers and IT professionals a powerful new way to capture and analyze packets from an application layer perspective. This is accomplished through the use of Open Protocol Notation (OPN)—a text-based protocol description language that models the protocol architecture, behavior, and data. The OPN protocol descriptions have a one-to-one mapping to the Protocol Object Model (POM). To learn more about about MA, I recommend visiting the Message Analyzer blog at Technet.com. You may also find it helpful to review the Message Analyzer Feature Summary to familiarize yourself with the application UI.

The focus of this article is to provide an introduction to MA and demonstrate how it can be used to discover the underlying Microsoft Office protocols involved in client-server communications. The scenario I use here is intentionally simple, but it should be enough to provide the basic mechanics of using MA. Let's get started!

To follow along, you'll need to do the following:

• Install the latest version of Microsoft Message Analyzer from the Microsoft Download Center.
• Download the latest version of the Office and SharePoint parsers from the Downloads section of MA's Start Page. Note that you must close and restart MA after installing parsers before the parsers can be used. If you don't see the parser listed under Downloads, check Settings to see if it's already been installed.
• Obtain write-access to a SharePoint site. For this exercise, I set up a SharePoint 2013 server test environment that uses non-SSL transport (http rather than https). MA can also capture HTTPS client-side unencrypted traffic, but the process is slightly more involved, requiring the installation of a third-party proxy server API. This isn't difficult, but we'll stick to the basics for now.
• Upload a Word document to a document library of your SharePoint site.

Trace Scenario

In this example, I'll be capturing the traffic generated by opening a Word document from a SharePoint document library, where it is currently saved. Document versioning and required check out are enabled for this library. To enable versioning and required check out in a SharePoint 2013 library, go to Site Settings > Site libraries and lists > Customize "LibraryName" > Versioning Settings. I will then make a change to the document and check it back in.

Scenario Steps

Here are the steps in this trace scenario:

1. Open a web browser to the SharePoint document library where you have saved a test Word document saved. Close all other applications, including Microsoft Word.
2. Start Message Analyzer as an administrator, and then run the Local Network Interfaces (Win 8.1 and Later) tracescenario from the Quick Trace menu. Alternatively, you can run the trace by clicking File > New Session > Live Trace and thenselecting Local Network Interfaces (Win 8.1 and Later) from the Trace Scenario list and clicking Start. The trace starts.
   Note: If you are accessing the SharePoint server using a VPN connection, you will need to use the "Network Tunnel Traffic and Unencrypted IPSEC" trace scenario.
3. Return to your web browser and click a Word document to open the document in Word 2013.
   Note: If your SharePoint site is set up with Office Online, the document will first open in Word Online. In this happens, select the Edit Document menu and select Edit in Word.
4. If the document library is correctly set up to use "Require Check Out" as mentioned above, the document will open with a message indicating that check out is required. Click Check Out to continue.
5. Make a change to the document and then click File > Check In. Type a version comment if you wish and then click Ok.

--End

About the MA analysis grid

Before going into the details about the data captured during the scenario, I'd like to spend a little time on the way the MA can present the data. Here is a screenshot of the first few operations and messages in the MA analysis grid:

Since the data can be grouped and filtered in various ways, your capture may look somewhat different. In this view, each node consists of either a message stack (green) or an operation (blue). Unlike Network Monitor which displays the entire message stack, MA coalesces messages so that only the top layer is displayed. And whenever possible, MA groups request messages with their corresponding server response messages. One benefit of combining the messages together in this way is for analyzing server response times. It should be noted, however, that this behavior depends on how the parsers are implemented. For instance, at the time of this writing, the SOAP parser has not implemented this behavior. Consequently, the Office protocols that sit atop SOAP in this scenario are not grouped into operations.

To expose the associated request and response messages, expand the operation nodes (under the MessageNumber column). You can also hide operations by clicking the Hide Operations button. This displays all the individual messages nodes without the operation nodes. To reveal the entire message stack, expand the message nodes or select Message Stack from the Tool Windows drop-down.

Customizing the Grid View

MA allows you to customize the way that capture data is displayed. For instance, depending on the traffic being captured and the View Layout used, some columns in the analysis grid may be devoid of data. If you know that this will always be true for certain trace scenarios, you might want to remove the unnecessary field columns by right-clicking the column heading and choosing Remove. You can then save this customized layout by clicking the View Layouts button and then selecting Save Current Layout As.

Data Grouping

For larger sets of capture data, the protocols of interest can be difficult to find. Another way to scrutinize particular protocols is to group the messages by field column. You can do this by right-clicking the column heading and choosing Group. The following shows how the grid looks when the data is grouped by protocol (Module):

Whenever the data is grouped, a box appears above the grid that contains the name of the field used to group the data (in this case, "Module"). To ungroup the data, click the x within that box. The data can also have groups within groups.

Filtering

Another way to view the data is to apply filters. Create filters by clicking View Filter and then adding filter expressions to the filter pane. For example, to view the Office protocol message groupings from this scenario trace, type each Office-related module in the View Filter box separated by the 'or' operator. The Office protocols from this list are the ones that start with "MS". By adding those to the filter and clicking Apply, we see only the protocols that we're interested in:

Overview of the captured Office protocols

The capture from this example consists of hundreds of messages. Of the 27 protocols captured in this trace, five are Office protocols. Note, however, that you may see a few more than this, depending on your SharePoint configuration. The following is a list of the Office protocols captured in this trace include the following:

MSVERSS—the Versions Web Service Protocol. This protocol uses the SOAP message protocol for formatting request and response messages. It is described in the MS-VERSS specification as follows:

This Versions Web Service Protocol enables a protocol client to view, delete, and restore a specified version of an existing file on a protocol server.

MSWEBSS—the Versions Web Service Protocol. This protocol uses the SOAP message protocol for formatting request and response messages. It is described in the MS-WEBSS specification as follows:

The Webs Web Service Protocol specifies a SOAP protocol that provides methods for modifying SharePoint sites in a site collection. In the context of the Webs Web Service Protocol, "Webs" refer to sites in a site collection. This protocol provides functions to get and modify content types, pages and files, list templates, columns, cascading style sheets (CSS), and Webs.

MSLISTSWS—the Lists Web Service Protocol. This protocol uses the SOAP message protocol for formatting request and response messages. It is described in the MS-LISTSWS specification as follows:

The List Web Service Protocol is used for the manipulation of list schemas and list data.

MSFSSHTTP—the File Synchronization via SOAP over HTTP Protocol. This protocol uses the SOAP message protocol for formatting request and response messages. It is described in the MS-FSSHTTP specification as follows:

The File Synchronization via SOAP over HTTP Protocol enables one or more protocol clients to synchronize changes done on shared files stored on a server.

MSWWSP—the Workflow Web Service Protocol. This protocol uses the SOAP message protocol for formatting request and response messages. It is described in the MS-WWSP specification as follows:

The Workflow Web Service Protocol specifies the communication sequences used to query, start, and manipulate workflows on a document.

Note that the names of the protocols as they appear in the MA analysis grid don't include a hyphen as they do in the official protocol specifications.

Message Stack and Details

All of the Office protocol messages sent in this scenario rely on the SOAP protocol to format the requests and responses. To view the details of any the messages, right-click the message row and select Show Details (or press CTRL+Enter). There are two tabs available in the Details view—the Stack tab and the Fields tab. The Stack tab is a convenient way to view the full message stack. For example, right-clicking the first MSWEBSS message and selecting the Details tab reveals the following message stack information:

This messaging and transport stack is consistent with the description provided in the MS-WEBSS specification (SOAP over HTTP|HTTPS).

The Fields tab provides a view of the message details. So for the WebUrlFromPageUrl Request, you'll see the following:

In this screenshot, you can see the page URL being requested in the Value column (truncated), as well as the value type, bit offset, and bit length. If any part of this message does not comply with the WSDL definition, an error would be flagged.

You can also see the parser definition for any message by right-clicking the row and selecting Go to 'ThisModule' definition . Doing this opens the OPN file for the given protocol parser. If you're interested in learning about OPN source code, the PEF Open Protocol Notation Programming Guide is available for download from the Microsoft Download Center.

Description of scenario messages

The following table describes the Office protocol messages sent during this trace scenario:

Conclusion

I hope that this simple demonstration has provided you with a good starting point in researching Office protocols, how they are implemented in Office products, and how they might be used in developing you own applications.