FAST ESP 5.x - Converting Clarity SearchStatus into CSVs.

Hi

Clarity is a great tool for monitoring ESP either through the UI or requesting the supporting data as xml.

The below script download clarity status in xml and update some .CSV files. The output files are pretty explanative to what they relate to.

 Disclaimer :

THE LICENSED SOFTWARE IS PROVIDED "AS IS" AND WE DO NOT MAKE, AND SPECIFICALLY DISCLAIM, ANY WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION THE WARRANTIES THAT IT IS FREE OF DEFECTS, VIRUS FREE, ABLE TO OPERATE ON AN UNINTERRUPTED BASIS, MERCHANTABLE, FIT FOR A PARTICULAR PURPOSE, ACCURATE, OR NON-INFRINGING.

THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LICENSED SOFTWARE IS BORNE BY YOU. SHOULD THE LICENSED SOFTWARE PROVE DEFECTIVE IN ANY RESPECT, YOU AND NOT WE ASSUME THE ENTIRE COST OF ANY SERVICE AND REPAIR. THIS DISCLAIMER OF WARRANTY CONSTITUTES AN ESSENTIAL PART OF THIS OBJECT CODE LICENSE AGREEMENT. NO USE OF THE LICENSED SOFTWARE IS AUTHORIZED HEREUNDER EXCEPT UNDER THIS DISCLAIMER.

Adapt the $ESPAdminHost and $ESPAdminPort to your environment.

ProcessClarityXML.ps1

 param ( $ClarityXML, [switch] $Clear, $ESPAdminHost="ESPAMDINSERVER", $ESPAdminPort="16000", $OutputFilePrefix="ESP-") 

if ( $Clear ) 
{
    Remove-Item -Path (Join-Path $PSScriptroot ($OutputFilePrefix+"*.csv"))
}

if ( $ClarityXML )
{
    Out-Host -InputObject ("Using given Clarity XML "+$ClarityXML)
}
else
{
    # Request the clarity Search-Status view (Extended) 

    $source = "https://"+$ESPAdminHost+":"+$ESPAdminPort+"/clarity/index.php?cluster=webcluster&view=search-status&xml=true"

    $ClarityXML = Join-Path $PSScriptroot ("clarity-webcluster-"+(get-Date -Format "yyyyMMddHHmmss")+".xml")

    $wc = New-Object System.Net.WebClient

    #Save the XML on disk 
    $wc.DownloadFile($source, $ClarityXML)
}

if ( Test-Path $ClarityXML ) 
{
    #Load and generate few csv 
    $status = [xml] (get-content $ClarityXML)

    #Print when the clarity xml has been taken. 
    Out-Host -InputObject $status.'search-status'

    #define output files 

    $qrserversout=Join-Path $PSScriptroot ($OutputFilePrefix+"qrservers.csv") 
    $dispatchersout=Join-Path $PSScriptroot ($OutputFilePrefix+"dispatchers.csv") 

    $indexersout=Join-Path $PSScriptroot ($OutputFilePrefix+"indexers.csv") 
    $partitionsout=Join-Path $PSScriptroot ($OutputFilePrefix+"partitions.csv") 
    $searchesout=Join-Path $PSScriptroot ($OutputFilePrefix+"searches.csv") 
    $rowtotalsout=Join-Path $PSScriptroot ($OutputFilePrefix+"rowtotals.csv") 

    #output the Search Status Header. 

    $data = $status.'search-status'

    #time : 2014-07-09 10:13:06
    #name : webcluster
    #version : 5.3
    #partitions : 6
    #st-warning : 2.000
    #st-critical : 5.000
    #lat-warning : 2.000
    #lat-critical : 5.000
    #qrserver : {qrserver, qrserver, qrserver, qrserver}
    #column : {column, column, column, column...}
    #row-totals : {row-totals, row-totals, row-totals, row-totals...}

    # QRSERVER - TOPFDISPATCH

    foreach ($qr in $data.qrserver)
    {
        $qr | Add-Member -MemberType NoteProperty -Name "Timestamp" -Value $data.time -ErrorAction SilentlyContinue
   
        $qr.dispatcher | Add-Member -MemberType NoteProperty -Name "Timestamp" -Value $data.time -ErrorAction SilentlyContinue
    }

    $data.qrserver.dispatcher | Export-Csv -Append -Encoding Unicode -Delimiter "`t" -Path  $dispatchersout -NoTypeInformation

    $data.qrserver            | Export-Csv -Append -Encoding Unicode -Delimiter "`t" -Path  $qrserversout -NoTypeInformation

    # Columns

    foreach ($col in $data.column )
    {
        foreach ($line in $col.row)
        {
            if ( $line.'rts-index')
            {
                $line.'rts-index' | Add-Member -MemberType NoteProperty -Name "Timestamp" -Value $data.time -ErrorAction SilentlyContinue            
                $line.'rts-index' | Add-Member -MemberType NoteProperty -Name "Column" -Value $col.id -ErrorAction SilentlyContinue            
                $line.'rts-index' | Add-Member -MemberType NoteProperty -Name "Row" -Value $line.id -ErrorAction SilentlyContinue            

                foreach ($part in $line.'rts-index'.partition) 
                {
                    $part | Add-Member -MemberType NoteProperty -Name "Timestamp" -Value $data.time -ErrorAction SilentlyContinue            
                    $part | Add-Member -MemberType NoteProperty -Name "Column" -Value $col.id -ErrorAction SilentlyContinue            
                    $part | Add-Member -MemberType NoteProperty -Name "Row" -Value $line.id -ErrorAction SilentlyContinue            
                    $part | Add-Member -MemberType NoteProperty -Name "host" -Value $line.'rts-index'.host -ErrorAction SilentlyContinue   
                    
                    #Remove the unwanted ~ in items count.
                    $part.items = $part.items -creplace "~",""
                }
            }

            if ( $line.'rts-search')
            {
                $line.'rts-search' | Add-Member -MemberType NoteProperty -Name "Timestamp" -Value $data.time -ErrorAction SilentlyContinue            
                $line.'rts-search' | Add-Member -MemberType NoteProperty -Name "Column" -Value $col.id -ErrorAction SilentlyContinue            
                $line.'rts-search' | Add-Member -MemberType NoteProperty -Name "Row" -Value $line.id -ErrorAction SilentlyContinue            
            }
        }   
    }

    $data.column.row.'rts-index'            | Export-Csv -Append -Encoding Unicode -Delimiter "`t" -Path  $indexersout -NoTypeInformation -ErrorAction SilentlyContinue
    $data.column.row.'rts-index'.partition  | Export-Csv -Append -Encoding Unicode -Delimiter "`t" -Path  $partitionsout -NoTypeInformation -ErrorAction SilentlyContinue
    $data.column.row.'rts-search'           | Export-Csv -Append -Encoding Unicode -Delimiter "`t" -Path  $searchesout -NoTypeInformation -ErrorAction SilentlyContinue

    # Row Totals

    foreach ($row in $data.'row-totals')
    {
        $row | Add-Member -MemberType NoteProperty -Name "Timestamp" -Value $data.time -ErrorAction SilentlyContinue    
    }

    $data.'row-totals' | Export-Csv -Append -Delimiter "`t" -Path  $rowtotalsout -Encoding Unicode -NoTypeInformation -ErrorAction SilentlyContinue
}
else
{
    Out-host -InputObject "Clarity XML file not found. Exiting." 
}

# End

You may schedule that script to run a few time a day and analyze the resulted CSV using your favorite BI client tool (MS Excel ?).

Happy ESP monitoring.