Use search administration reports (SharePoint Server 2010)

  • Article

 

Applies to: SharePoint Server 2010

Summary: Use and configure reports to monitor search system health, including crawl rate, crawl processing, query latency trend, and back-end latency.

Search administration reports help you to determine the health of search service applications on a SharePoint farm. There are three types of search administration reports:

  • Basic search administration reports. Enabled by default, these reports show high-level monitoring data aggregated from all components for the selected search service application.

  • Advanced search administration reports. Enabled by default, these reports show more in-depth monitoring data aggregated from all components for the selected search service application.

  • Verbose search administration reports. After enabling verbose query monitoring for search, this trend report uses per-query data to derive query latency percentiles.

Note

To obtain search administration reports, you must have a Microsoft SharePoint Server 2010, Microsoft Search Server 2010, or Microsoft FAST Search Server 2010 for SharePoint environment that contains crawled data and a Search Center. You also have to ensure that the State Service has been enabled through the Farm Configuration Wizard.

The following table shows the four basic search administration reports and the metrics they provide. For each report, you can filter the results by search service application and time. You can also filter the Crawl Rate per Content Source by content source.

Report Description Metrics Reported

Crawl Rate per Content Source

Provides a view of recent crawl activity, aggregated by content source. Anchor crawl appears as a separate (virtual) content source. You can filter the report by search service application, time, and content source.

Average items crawled per minute, separated by content source. Accompanying table shows crawl start times, duration, and crawl rate.

Crawl Rate per Type

Provides a view of recent crawl activity, sorted by items and actions for a given URL. These items and actions include modified items, deleted items, retries, errors, and others. You can filter the report by search service application and time.

Average items per minute, separated by type of crawl transaction:

  • Modified: Shows items that were modifed and re-crawled.

  • Not modified: Shows items that were not modified and not crawled.

  • Security only: shows items for which the security attributes were changed.

  • Deleted: Shows items that were deleted from the content source and must be deleted from the index.

  • Not indexed: Shows items that were crawled but not indexed.

  • Anchor text: Shows anchor text items crawled.

  • Retry: Shows crawl retries.

  • Error: Shows crawl errors.

Accompanying table shows totals for each type of crawl task.

Overall Query Latency

Provides a view of recent query activity, showing latency from the major segments of the query pipeline and query averages per minute. You can filter the report by search service application and time.

  • Average query count per minute (along the right side of the graph), represented bya light purple line. Usually fluctuates throughout the day.

  • Average overall query latency per minute (on the left side of the graph), represented by the height of the stacked bars for the major segments of the query pipeline:

    • Server Rendering: Shows time spent for the various web parts in the Web front-end.

    • Object Model: Shows the time spent for communication between the Web front-end and the search back-end.

    • Back-end: Shows the time spent querying all index and property databases, removing duplicates, and returning the results to the Object Model.

SharePoint Back-end Query Latency

Provides a view of recent query activity, showing latency details from the index and property database portion of the query pipeline and query averages per minute. You can filter the report by search service application and time. This report is only applicable to queries that go to the Microsoft SharePoint Server 2010 or Microsoft Search Server 2010 back-end.

Average back-end query count per minute (on the right side of the graph), represented bya light purple line. Usually fluctuates throughout the day.

Average back-end query latency per minute (on the left side of the graph) for the following back-end tasks:

  • Results Population: Average time spent creating the in memory table to be passed back to the Object Model.

  • High Confidence Results: Average time spent to retrieve high confidence results for queries.

  • Multiple Results Retrieval: Average time spent retrieving document metadata, such as title or author, to appear in the query results.

  • Best Bets: Average time spent determining whether there are best bets available for the query terms.

  • Final Sort: Average times spent sorting results.

  • Security Trimming: Average time spent removing items the user doesn’t have access to. You can increase the security cache to decrease the values.

  • Duplicate Removal: Average time spent removing duplicates.

  • Property Store Query: Average time spent querying the property database for property-based queries.

  • Full-text Query: Average time spent querying the full-text index for results. Sustained high values for your environment mean you should probably partition the index.

The following table shows the three advanced search administration reports and the metrics they provide.

Report Description Metrics Reported

Crawl Processing per Activity

Provides a view of where crawl processing occurs in the pipeline. The timings per component are grouped by activity, such as filtering or word breaking. You can filter the report by search service application and time.

Crawl activity in seconds, separated by major task:

  • Waiting: Time spent waiting.

  • Standard Properties: Time spent retrieving standard properties.

  • Protocol Handlers: Time spent running protocol handlers.

  • Initializing Filter: Time spent initializing filters.

  • Filtering: Time spent filtering.

  • Word Breaking: Time spent on word breaking.

  • Process Words: Time spent on processing words.

  • Chunk Processing: Time spent on processing chunks.

  • Initializing Document Filtering: Time spent on initializing document filtering.

  • Completing Document Filtering: Time spent on completing document filtering.

Crawl Processing per Component

Provides a view of where crawl processing occurs in the pipeline, per minute. The timings are grouped by component, such as File Protocol Handler or Anchor Plug-in.

Crawl activity in seconds, separated by component:

  • Waiting: Time spent on waiting.

  • BCS Protocol Handler: Crawl processing time for the Business Connectivity Services (BCS) protocol handler.

  • File Protocol Handler: Crawl processing time for the file protocol handler.

  • HTTP Protocol Handler: Crawl processing time for the HTTP protocol handler.

  • Lotus Protocol Handler: Crawl processing time for the Lotus protocol handler.

  • SPS Protocol Handler: Crawl processing time for the Microsoft Office SharePoint Portal Server 2003 protocol handler.

  • STS3 Protocol Handler: Crawl processing time for the Microsoft Office SharePoint Server 2007 protocol handler.

  • STS4 Protocol Handler: Crawl processing time for the SharePoint Server 2010 protocol handler.

  • Other Protocol Handler: Crawl processing time for other protocol handlers.

  • Initializing Filter: Crawl processing time for the filter initializer.

  • Filtering: Crawl processing time for the filtering component.

  • Word Breaking: Crawl processing time for the word-breaking component.

  • Anchor Plug-in: Crawl processing time for the anchor text plug-in.

  • ARPI Plug-in: Crawl processing tme for the ARPI plug-in.

  • Feature Extraction Plug-in: Crawl processing time for the feature extraction plug-in.

  • Indexer Plug-in: Crawl processing time for the indexer plug-in.

  • Matrix Plug-in: Crawl processing time for the matrix plug-in.

  • Scopes Plug-in: Crawl processing time for the scopes plug-in.

  • Gatherer Plug-in: Crawl processing time for the gatherer plug-in.

  • Other: Crawl processing time for other components.

Crawl Queue

Provides a view of the state of the crawl queue, displaying incoming links to process and outgoing transactions queued.

  • Links to Process: Incoming links to process.

  • Transactions Queued: Outgoing transactions queued.

The following table shows the verbose search administration report and the metrics it provides.

Report Description Metrics Reported

Query Latency Trend

This report provides a view of server-side query latency breakdown by percentile, in addition to the current crawl rate.

  • Crawl Rate (in milliseconds)

  • Query latency (in milliseconds), separated by percentile

Note

To enable the Query Latency Trend report, you must run the following Windows PowerShell cmdlets:

  • $app = Get-SPEnterpriseSearchServiceApplication "<application name>"

  • $app | Set-SPEnterpriseSearchServiceApplication -VerboseQueryMonitoring "True"

  • $app = Get-SPEnterpriseSearchServiceApplication "<application name>"

  • $app.Update()

Filter data for search administration reports

To view administrative reports, see View administrative reports (SharePoint Server 2010). You can filter data for search administration reports by using Central Administration. By default, search administration reports show data for all search applications over the last 12 hours. You can filter report data in the following ways:

  • Restrict a report to one or more search service applications

  • Restrict a report to a specific time period

To restrict a report to one or more search service applications

  1. Verify that you have the following administrative credentials:

    • To view reports in the Search administration reports folder, you must be a member of the Farm Administrators SharePoint group on the computer running the SharePoint Central Administration Web site.

  2. On the report page, click the filter icon next to the Application box.

  3. In the Select Filter Value(s) dialog box, select the search service applications that you want, and then click OK.

  4. Click the Apply Filters button to update the report.

To restrict a report to a specific time period

  1. Verify that you have the following administrative credentials:

    • To view reports in the Search administration reports folder, you must be a member of the Farm Administrators SharePoint group on the computer running the SharePoint Central Administration Web site.

  2. On the report page, click the Start Date and End Date calendars and times to specify the time period that you want.

  3. Click the Apply Filters button to update the report.