Metadata Navigation and Filtering in SharePoint Server 2010 (ECM)

Applies to: SharePoint Server 2010

In this article
Metadata Navigation and Filtering
Tips
Metadata Navigation and Filtering Programmability

Metadata Navigation and Filtering provides a way to filter Microsoft SharePoint Server 2010 list views by their item metadata. A list owner can use this feature to promote fields on the list as key navigational fields, and users viewing those lists see a filtering user interface (UI) that enables them to filter the current list view to show items with the desired values in those fields. Metadata Navigation and Filtering works with list-level indices to support a seamless browsing experience—even in very large lists that contain tens of millions of items. This feature promotes the use of metadata and tagging by ensuring that, when items are well-tagged across multiple fields, they are inherently more discoverable than list view items that are not tagged.

Metadata Navigation and Filtering

Metadata Navigation and Filtering includes:

• A new UI for browsing and filtering list views based on list item metadata.

• A process that creates list indices automatically, depending on the fields that are promoted as navigational fields in the list.

• A process that selects the best index for the query being run to avoid list view throttling, when possible.

New User Interface

The new user interface includes navigation hierarchies and key filters:

• Navigational hierarchies are fields that appear in the tree view when a user is viewing the parent list. The values within this field can be browsed, like a folder structure, and clicked to view documents that match the corresponding values. Like a good folder structure, a good navigation hierarchy field divides content into manageable chunks.

• Key filters are fields that appear below the tree level. They are supplemental filters that help you narrow down the content in your current view.

Automatic Index Creation

On the Metadata Navigation settings page for a list, you can also specify whether indices are automatically created on the list to match your currently selected navigation hierarchy and key filter fields. If this setting is enabled (as it is by default on a list), when the metadata navigation settings page is saved:

• Single column indices are created on all supported key filter fields, with the exception of the Content Type field and the Choice field.

• Compound indices are created on all supported combinations of navigation hierarchies and key filters.

Leaving the setting that specifies whether indices are automatically created on the list to match your currently selected navigation hierarchy and key filter fields settings is recommended for most scenarios, because if no indices are created on a list, every query on that list is unindexed. Unindexed queries are not supported in Microsoft SharePoint Server 2010 on large lists, or on lists that contain more items than the quantity defined in the site’s List View Query Threshhold. In some advanced scenarios, it may be appropriate to turn off this setting and configure indices manually. Important considerations include:

• SharePoint Server 2010 supports a maximum of 20 indices per list. Metadata Navigation and Filtering is unable to create an index if there is not enough room in this 20-index space for all of the eligible combinations from the navigation hierarchy and key filter fields that are selected.

• Creating new indices within a large list is prohibited when in a large list, unless you are a farm administrator or you are operating within a service window.

Index Types and Supported Fields

Table 1 summarizes the types of fields that can participate in metadata navigation, and in what capacity.

Table 1. Types of fields that can participate in metadata navigation

Indexed Queries

Metadata Navigation and Filtering does its best to put the most beneficial index to work every time a list view is loaded. Every time the user loads a list view, or refreshes it by applying a new filter, clearing a filter, or applying a sort on a field, there is some logic in the background that examines the request and determines a way to query the database that avoids large list throttling. This is not always straightforward; a query might have more than one way in which it can be run, and each way might produce different results. The process works like this:

1. When the user navigates to a list view page, or refreshes a list view, the page runs a Collaborative Application Markup Language (CAML) query against the list to get the items that are displayed in the List View Web Part. This query may include one or more filters, which come from selected values in a navigation hierarchy, one or more key filter selections, one or more filters specified in the view definition, or one or more filters specified in column filters of the List View Web Part. If the current list is large, this CAML query may throw a SPThrottledQueryException, which means that the query is not selective. In other words, it examines more items in the database than the List View Query Threshold defined by the administrator. If this exception is encountered, then steps 2 and later are executed.

2. Metadata navigation examines each of the filter operations (or clauses) in the CAML query and determines which have corresponding indices created on the list that may be used to satisfy that filter operation. For example, if there is an index on the ModifiedBy column, and the current list view filters on the Modified By column, that index may be used by the query.

3. For each of these valid index/filter combinations, Metadata Navigation rewrites the current CAML query to use that index.

4. Metadata Navigation and Filtering then attempts to run each of those indexed queries until it finds one that does not generate a SPThrottledQueryException. If all queries result in this exception, it constructs and executes a fallback query.

Fallback Queries

If Metadata Navigation and Filtering determines that the current user request cannot be expressed as an indexed query that is selective, it constructs and executes a fallback query. A fallback query is a modified version of the original user query that queries against only a portion of the list instead of the entire list. Fallback queries are intended to show the user a useful partial set of results, even when the original query could not be run due to large list throttling. Fallback queries can also serve as a warning to list owners that the data distribution in the list is skewed and certain queries that users are running are not able to return a full set of results, which means that users may be blocked from accessing content that they need. Fallback queries occasionally return 0 results if none of the items in the portion of the list scanned by the query contain results that match the original user query.

Because the results of a fallback query are only a partial set of the items that the user is requesting, the user is prompted by an onscreen message that he or she is viewing only a partial set of results, and must apply additional filters to view a complete set. Every time a user specifies an additional filter, this is another opportunity for the query engine to find a selective filter/index combination that does not result in an SPThrottledQueryException. There are two types of fallback queries, in order of preference: indexed fallback queries, and simple fallback queries.

1. Indexed fallback queries. This type of fallback query works against one of the indices on the list, examining the N most recently created items that match the filtered value in that index. To use indexed fallback query semantics, the user query must have a filter on an indexed column, and that filter/index combination must meet some additional constraints. Only the types of filter/index combinations listed in Table 2 can participate in an indexed fallback query:

   Table 2. Allowed filter/index combinations in fallback queries

   Index Type	Queries That Support Index Fallback
   Content Type	One value
   Choice single value	One value
   Managed Metadata single value	One value Many values
   Managed Metadata multi-value	One value Many values
   Person or Group single value	One value
   Date and Time	One value A range of values
   Number	One value A range of values
   Managed Metadata single value, compound index with Date and Time	One value on the Managed Metadata column Either one value or a range of values on the Date and Time column
   Managed Metadata single value, compound index with Number	One value on a Managed Metadata column Either one value or a range of values on the Number column
   Managed Metadata single value, compound index with Content Type	One value in the Managed Metadata column One value in the Content Type column
   Managed Metadata single value, compound index with Person or Group	One value in the Managed Metadata column One value in the Person or Group column
   Content Type, compound indexed with Date and Time	One value on the Content Type column Either one value or a range of values on the Date and Time column
   Content Type, compound indexed with Number	One value on the Content Type column Either one value or a range of values on the Number column
   Content Type, compound indexed with Person or Group	One value on the Content Type column One value on the Person or Group column

   Note

   In Table 2, "one value" means a filter that is expressible by using a clause in CAML such as 10. "Many values" means a filter that is expressible by using a clause in CAML such as 1011. A "range of values" means a filter expressible by using a clause in CAML such as 2010-02-04T08:00:00Z.

   For certain indexed combinations, an indexed fallback query cannot be used if the query is not in the specified format. For example, if a compound index exists on a single value taxonomy field and a date field, but the user query includes multiple values from the taxonomy field in a CAML clause, a simple fallback query is run instead of an indexed query.

2. Simple fallback queries. Queries that do not have an indexed filter clause that support an indexed fallback must run a simple fallback query. A simple fallback query operates against the top N most recently created items in the list. If none of these N items matches the query parameters in the user query, no results are returned.

   Note

   In both indexed fallback and simple fallback queries, "N" means (List View Query Threshold * .20). The List View Query Threshold can be configured by an administrator, but the .20 multiplier cannot be changed.

Tips

• Metadata Navigation is best used in conjunction with managed metadata fields; their robustness, hierarchical nature, and adoptability make them effective at dividing list content into manageable chunks.

• The Developer Dashboard can be used on list view pages with metadata navigation to see the CAML and SQL behind each of the indexed and fallback queries that were run for a user query.

• Some administrators may want to treat a fallback query as a warning that indicates that the data in a list is not well-distributed across indices and field values, and some data redistribution is necessary so that users of the list are able to access the documents that they need. If verbose logging is enabled, ULS log entries are written every time a fallback query is executed. A health rule can be developed to intercept these log entries and show a warning to the administrator that this is happening so action can be taken.

Metadata Navigation and Filtering Programmability

The object model for metadata navigation and filtering is found primarily in the Microsoft.Office.DocumentManagement.MetadataNavigation namespace. It includes the following types.

Table 3. Metadata navigation and filtering programming model

See Also

Concepts

Document Management Programming Model in SharePoint Server 2010 (ECM)

Developing with Document Management Features in SharePoint Server 2010 (ECM)

Managing Enterprise Metadata in SharePoint Server 2010 (ECM)