Migrating from SQL Trace to Extended Events

In SQL Server codenamed “Denali” we are moving our diagnostic tracing capabilities forward by building a system on top of Extended Events. With every new system you face the specter of migration which is always a bit of a hassle. I’m obviously motivated to see everyone move their diagnostic tracing systems over to the new extended events based system, so I wanted to make sure we lowered the bar for the migration process to help ease your trials.

In my initial post on Denali CTP 1 I described a couple tables that we created that will help map the existing SQL Trace Event Classes to the equivalent Extended Events events. In this post I’ll describe the tables in a bit more details, explain the relationship between the SQL Trace objects (Event Class & Column) and Extended Event objects (Events & Actions) and at the end provide some sample code for a managed stored procedure that will take an existing SQL Trace session (eg. a trace that you can see in sys.Traces) and converts it into event session DDL.

Can you relate?

In some ways, SQL Trace and Extended Events is kind of like the Standard and Metric measuring systems in the United States. If you spend too much time trying to figure out how to convert between the two it will probably make your head hurt. It’s often better to just use the new system without trying to translate between the two. That said, people like to relate new things to the things they’re comfortable with, so, with some trepidation, I will now explain how these two systems are related to each other. First, some terms…

SQL Trace is made up of Event Classes and Columns. The Event Class occurs as the result of some activity in the database engine, for example, SQL:Batch Completed fires when a batch has completed executing on the server. Each Event Class can have any number of Columns associated with it and those Columns contain the data that is interesting about the Event Class, such as the duration or database name.

In Extended Events we have objects named Events, EventData field and Actions. The Event (some people call this an xEvent but I’ll stick with Event) is equivalent to the Event Class in SQL Trace since it is the thing that occurs as the result of some activity taking place in the server. An  EventData field (from now on I’ll just refer to these as fields) is a piece of information that is highly correlated with the event and is always included as part of the schema of an Event. An Action is something that can be associated with any Event and it will cause some additional “action” to occur when ever the parent Event occurs. Actions can do a number of different things for example, there are Actions that collect additional data and, take memory dumps.

When mapping SQL Trace onto Extended Events, Columns are covered by a combination of both fields and Actions. Knowing exactly where a Column is covered by a field and where it is covered by an Action is a bit of an art, so we created the mapping tables to make you an Artist without the years of practice.

Let me draw you a map.

Event Mapping

The table dbo.trace_xe_event_map exists in the master database with the following structure:

By joining this table sys.trace_events using trace_event_id and to the sys.dm_xe_objects using xe_event_name you can get a fair amount of information about how Event Classes are related to Events. The most basic query this lends itself to is to match an Event Class with the corresponding Event.

SELECT
t.trace_event_id,
t.name [event_class],
e.package_name,
e.xe_event_name
FROM sys.trace_events t INNER JOIN dbo.trace_xe_event_map e
ON t.trace_event_id = e.trace_event_id

There are a couple things you’ll notice as you peruse the output of this query:

• For the most part, the names of Events are fairly close to the original Event Class; eg. SP:CacheMiss == sp_cache_miss, and so on.
• We’ve mostly stuck to a one to one mapping between Event Classes and Events, but there are a few cases where we have combined when it made sense. For example, Data File Auto Grow, Log File Auto Grow, Data File Auto Shrink & Log File Auto Shrink are now all covered by a single event named database_file_size_change. This just seemed like a “smarter” implementation for this type of event, you can get all the same information from this single event (grow/shrink, Data/Log, Auto/Manual growth) without having multiple different events. You can use Predicates if you want to limit the output to just one of the original Event Class measures.
• There are some Event Classes that did not make the cut and were not migrated. These fall into two categories; there were a few Event Classes that had been deprecated, or that just did not make sense, so we didn’t migrate them. (You won’t find an Event related to mounting a tape – sorry.) The second class is bigger; with rare exception, we did not migrate any of the Event Classes that were related to Security Auditing using SQL Trace. We introduced the SQL Audit feature in SQL Server 2008 and that will be the compliance and auditing feature going forward. Doing this is a very deliberate decision to support separation of duties for DBAs. There are separate permissions required for SQL Audit and Extended Events tracing so you can assign these tasks to different people if you choose. (If you’re wondering, the permission for Extended Events is ALTER ANY EVENT SESSION, which is covered by CONTROL SERVER.)

Action Mapping

The table dbo.trace_xe_action_map exists in the master database with the following structure:

You can find more details by joining this to sys.trace_columns on the trace_column_id field.

SELECT
c.trace_column_id,
c.name [column_name],
a.package_name,
a.xe_action_name
FROM sys.trace_columns c INNER JOIN dbo.trace_xe_action_map a
ON c.trace_column_id = a.trace_column_id

If you examine this list, you’ll notice that there are relatively few Actions that map to SQL Trace Columns given the number of Columns that exist. This is not because we forgot to migrate all the Columns, but because much of the data for individual Event Classes is included as part of the EventData fields of the equivalent Events so there is no need to specify them as Actions.

Putting it all together

If you’ve spent a bunch of time figuring out the inner workings of SQL Trace, and who hasn’t, then you probably know that the typically set of Columns you find associated with any given Event Class in SQL Profiler is not fix, but is determine by the contents of the table sys.trace_event_bindings. We’ve used this table along with the mapping tables to produce a list of Event + Action combinations that duplicate the SQL Profiler Event Class definitions using the following query, which you can also find in the Books Online topic How To: View the Extended Events Equivalents to SQL Trace Event Classes.

USE MASTER;
GO
SELECT DISTINCT
tb.trace_event_id,
te.name AS 'Event Class',
em.package_name AS 'Package',
em.xe_event_name AS 'XEvent Name',
tb.trace_column_id,
tc.name AS 'SQL Trace Column',
am.xe_action_name as 'Extended Events action'
FROM (sys.trace_events te LEFT OUTER JOIN dbo.trace_xe_event_map em
ON te.trace_event_id = em.trace_event_id) LEFT OUTER JOIN sys.trace_event_bindings tb
ON em.trace_event_id = tb.trace_event_id LEFT OUTER JOIN sys.trace_columns tc
ON tb.trace_column_id = tc.trace_column_id LEFT OUTER JOIN dbo.trace_xe_action_map am
ON tc.trace_column_id = am.trace_column_id
ORDER BY te.name, tc.name

As you might imagine, it’s also possible to map an existing trace definition to the equivalent event session by judicious use of fn_trace_geteventinfo joined with the two mapping tables. This query extracts the list of Events and Actions equivalent to the trace with ID = 1, which is most likely the Default Trace. You can find this query, along with a set of other queries and steps required to migrate your existing traces over to Extended Events in the Books Online topic How to: Convert an Existing SQL Trace Script to an Extended Events Session.

USE MASTER;
GO
DECLARE @trace_id int
SET @trace_id = 1
SELECT DISTINCT el.eventid, em.package_name, em.xe_event_name AS 'event'
, el.columnid, ec.xe_action_name AS 'action'
FROM (sys.fn_trace_geteventinfo(@trace_id) AS el
LEFT OUTER JOIN dbo.trace_xe_event_map AS em
ON el.eventid = em.trace_event_id)
LEFT OUTER JOIN dbo.trace_xe_action_map AS ec
ON el.columnid = ec.trace_column_id
WHERE em.xe_event_name IS NOT NULL AND ec.xe_action_name IS NOT NULL

You’ll notice in the output that the list doesn’t include any of the security audit Event Classes, as I wrote earlier, those were not migrated.

But wait…there’s more!

If this were an infomercial there’d by some obnoxious guy next to me blogging “Well Mike…that’s pretty neat, but I’m sure you can do more. Can’t you make it even easier to migrate from SQL Trace?”  Needless to say, I’d blog back, in an overly excited way, “You bet I can' obnoxious blogger side-kick!” What I’ve got for you here is a Extended Events Team Blog only special – this tool will not be sold in any store; it’s a special offer for those of you reading the blog.

I’ve wrapped all the logic of pulling the configuration information out of an existing trace and and building the Extended Events DDL statement into a handy, dandy CLR stored procedure. Once you load the assembly and register the procedure you just supply the trace id (from sys.traces) and provide a name for the event session. Run the procedure and out pops the DDL required to create an equivalent session. Any aspects of the trace that could not be duplicated are included in comments within the DDL output.

This procedure does not actually create the event session – you need to copy the DDL out of the message tab and put it into a new query window to do that. It also requires an existing trace (but it doesn’t have to be running) to evaluate; there is no functionality to parse t-sql scripts. I’m not going to spend a bunch of time explaining the code here – the code is pretty well commented and hopefully easy to follow. If not, you can always post comments or hit the feedback button to send us some mail.

Sample code: TraceToExtendedEventDDL

Installing the procedure

Just in case you’re not familiar with installing CLR procedures…once you’ve compile the assembly you can load it using a script like this:

-- Context to master
USE master
GO

-- Create the assembly from a shared location.
CREATE ASSEMBLY TraceToXESessionConverter
FROM 'C:\Temp\TraceToXEventSessionConverter.dll'
WITH PERMISSION_SET = SAFE
GO

-- Create a stored procedure from the assembly.
CREATE PROCEDURE CreateEventSessionFromTrace
@trace_id int,
@session_name nvarchar(max)
AS
EXTERNAL NAME TraceToXESessionConverter.StoredProcedures.ConvertTraceToExtendedEvent
GO

Enjoy!

-Mike

Comments

• Anonymous
  February 16, 2012
  Great post Mike! Would you be kind enough to post the TraceToXEventSessionConverter.dll? Thx a lot

• Anonymous
  March 12, 2012
  I'm not a developper and not sure what to do with this TraceToExtendedEventDDL.cs file. Would it be possible for you to give us instructions as per how to ccreate the dll out of it, and how to register it as an assembly?

• Anonymous
  February 13, 2013
  Followed these steps and created the .dll file. Thanks Mike blogs.msdn.com/.../converting-a-sql-trace-to-an-extended-event-session.aspx

• Anonymous
  July 31, 2014
  dbo.trace_xe_event_map should be sys.trace_xe_event_map

• Anonymous
  February 25, 2015
  Hi,  We want to use Microsoft.SqlServer.XEvent.Linq.dll to read SQL Server Extended Events files and streaming data. We want to know if this dll is redistributable. -Aravindhan