Extracting data from ULS logs using logparser

  • [アーティクル]

Had an interesting problem today. Received an ULS log file that was 18GB large. Although I knew the approximate timeframe where the problem started happening the normal PowerShell commands I use to break the logs did not work.

Basically the challenge was: how to extract a specific timeframe from an 18GB log file.

Answer(as suggested by the great CristiG): using logparser. Initially meant for IIS logs, we can use logparser also for ULS with a little bit of adaptation. The main challenge was parsing the Timeframe column that is not in the standard format that logparser understands.

The resulting script looks like:

select [Timestamp],Process,TID,Area,Category,EventID,Level,Message,Correlation into test3.log from problem.log
where TO_TIME(TO_TIMESTAMP(STRCAT(STRCAT(STRCAT(STRCAT(STRCAT(STRCAT(TRIM(SUBSTR(Timestamp,6,4)),'-'),TRIM(SUBSTR(Timestamp,0,2))),'-'),TRIM(SUBSTR(Timestamp,3,2))),' '),TRIM(SUBSTR(Timestamp,11,8))), 'yyyy-MM-dd hh:mm:ss')) between timestamp('01:33:00', 'hh:mm:ss') and timestamp('01:43:00', 'hh:mm:ss')

After installing logparser and adding it to your path variable you can run the following command to filter the logs: LogParser -i:tsv -o:tsv file:test2.sql (given that you have the input file and the script file are in the same directory).

Logparser can be downloaded for free from: https://www.microsoft.com/en-us/download/details.aspx?id=24659

Comments

  • Anonymous
    December 13, 2012
    You can also use the readily available ULSViewer found @ codeplex. Many parameters to filter by, including date/time, correlation ID. Plus you can open a live ULS log file as it is being written. http://ulsviewer.codeplex.com/

  • Anonymous
    March 29, 2013
    thanks for sharing Daniel

  • Anonymous
    November 04, 2013
    OK,now how can I expand that to a date range as well.    I got lost three STRCATs in. I want something like:   between timestamp('2013-11-04 00:00:01', 'yyyy-MM-dd hh:mm:ss') and timestamp('2013-11-06 23:59:59', 'yyyy-MM-dd hh:mm:ss') But, I get zero results that way.