UI Automation and Active Accessibility

  • Article

Microsoft Active Accessibility is the legacy API that was introduced in Windows 95, and was designed to make Windows applications accessible. Microsoft UI Automation is the new accessibility model for Windows and is intended to address the needs of assistive technology products and automated testing tools. UI Automation offers many improvements over Microsoft Active Accessibility. This topic explains the differences between the two technologies.

This topic contains the following sections.

Programming Languages

Microsoft Active Accessibility is based on the Component Object Model (COM) with support for dual interfaces, and therefore, is programmable in C/C++ and scripting languages.

When UI Automation was introduced, the client API was limited to managed code, while the provider API included both managed and unmanaged implementations. With Windows 7, a new COM-based client API was introduced to make it easier to program UI Automation client applications in C/C++.

Servers and Clients

In Microsoft Active Accessibility, servers and clients communicate directly, largely through the server implementation of the IAccessible interface.

In UI Automation, a core service lies between the server (provider) and the client. The core service makes calls to the interfaces implemented by providers and provides additional services, such as generating unique run-time identifiers for UI elements. Client applications gain access to this core service by creating a CUIAutomation object. This object supports a set of client interfaces that are separate from the provider interfaces. For more information, see Creating the CUIAutomation Object.

UI Automation providers can provide information to Microsoft Active Accessibility clients, and Microsoft Active Accessibility servers can provide information to UI Automation client applications. However, because Microsoft Active Accessibility does not expose as much information as UI Automation, the two models are not fully compatible.

UI Elements

Microsoft Active Accessibility presents a UI element as an IAccessible interface paired with a child identifier. It is difficult to compare two IAccessible pointers to determine if they refer to the same element.

In UI Automation, every element is represented as an object that exposes the IUIAutomationElement interface to clients. Elements can be compared by their run-time identifiers, which are retrieved by using IUIAutomationElement::GetRuntimeId.

Tree Views and Navigation

The UI elements on the screen can be seen as a tree structure with the desktop as the root, application windows as immediate children, and elements within applications as further descendants.

In Microsoft Active Accessibility, many UI elements that are irrelevant to end users are exposed in the tree structure. Client applications must examine all elements in the tree to determine which elements are meaningful.

UI Automation client applications see the UI through a filtered view. The view contains only elements that give information to the user or that the user can interact with. Predefined views that include only control elements and only content elements are available, and client applications can define custom views. UI Automation makes it easier to describe the UI to the user, and to help the user interact with applications.

In Microsoft Active Accessibility, navigation between elements is spatial, for example, moving to the element that lies to the left on the screen, logical, for example, moving to the next menu item or the next item in the tab order in a dialog box, or hierarchical, for example, moving to the first child element in a container or from a child element to its parent element. Hierarchical navigation is complicated by the fact that child elements are not always objects that implement IAccessible.

In UI Automation, all UI elements are COM objects that expose the IUIAutomationElement interface and support the same basic functionality. From the standpoint of the provider, COM objects implement an interface that is inherited from IRawElementProviderSimple. Navigation is mainly hierarchical; that is, from parents to children, and from one sibling to the next. However, navigation between siblings has a logical element, as it may follow the tab order. A client can navigate from any starting-point, using any filtered view of the tree, by using IUIAutomationTreeWalker. A client can also navigate to particular children or descendants by using IUIAutomationElement::FindFirst and IUIAutomationElement::FindAll. For example, it is easy to retrieve all elements in a dialog box that support a specified control pattern.

Navigation in UI Automation is more consistent than in Microsoft Active Accessibility. Some elements, such as drop-down lists and pop-up windows, appear twice in the Microsoft Active Accessibility tree, and navigation from those elements may have unexpected results. It is difficult to implement Microsoft Active Accessibility properly for a rebar control. UI Automation enables reparenting and repositioning, so that an element can be placed anywhere in the tree, despite the hierarchy imposed by ownership of windows.

Roles and Control Types

Microsoft Active Accessibility uses the accRole property (IAccessible::get_accRole) to retrieve a description of the element role in the UI, such as ROLE_SYSTEM_SLIDER or ROLE_SYSTEM_MENUITEM. The role of an element is the main clue to its available functionality. Interaction with a control is achieved by using fixed methods such as IAccessible::accSelect and IAccessible::accDoDefaultAction. The interaction between the client application and the UI is limited to what can be done through IAccessible.

In contrast, UI Automation decouples the control type of the element, which is described by the IUIAutomationElement::CurrentControlType (or IUIAutomationElement::CachedControlType) property, from its expected functionality. Functionality is determined by the control patterns that are supported by the provider through its implementation of specialized interfaces. Control patterns can be combined to describe the full set of functionality that is supported by a particular UI element. Some providers are required to support a particular control pattern. For example, the provider for a check box must support the Toggle control pattern. Other providers are required to support one or more of a set of control patterns. For example, a button must support either the Toggle or the Invoke control pattern. Still others support no control patterns. For example, a pane that cannot be moved, resized, or docked has no control patterns.

UI Automation supports custom controls, which are identified by the UIA_CustomControlTypeId constant and can be described by the IUIAutomationElement::CurrentLocalizedControlType (or IUIAutomationElement::CachedLocalizedControlType) property.

The following table maps Microsoft Active Accessibility object roles to UI Automation control types.

Active Accessibility role UI Automation control type
ROLE_SYSTEM_PUSHBUTTON Button
ROLE_SYSTEM_CLIENT Calendar
ROLE_SYSTEM_CHECKBUTTON CheckBox
ROLE_SYSTEM_COMBOBOX ComboBox
ROLE_SYSTEM_CLIENT See Custom Control Types.
ROLE_SYSTEM_LIST DataGrid
ROLE_SYSTEM_LISTITEM DataItem
ROLE_SYSTEM_DOCUMENT Document
ROLE_SYSTEM_TEXT Edit
ROLE_SYSTEM_GROUPING Group
ROLE_SYSTEM_LIST Header
ROLE_SYSTEM_COLUMNHEADER HeaderItem
ROLE_SYSTEM_LINK Hyperlink
ROLE_SYSTEM_GRAPHIC Image
ROLE_SYSTEM_LIST List
ROLE_SYSTEM_LISTITEM ListItem
ROLE_SYSTEM_MENUPOPUP Menu
ROLE_SYSTEM_MENUBAR MenuBar
ROLE_SYSTEM_MENUITEM MenuItem
ROLE_SYSTEM_PANE Pane
ROLE_SYSTEM_PROGRESSBAR ProgressBar
ROLE_SYSTEM_RADIOBUTTON RadioButton
ROLE_SYSTEM_SCROLLBAR ScrollBar
ROLE_SYSTEM_SEPARATOR Separator
ROLE_SYSTEM_SLIDER Slider
ROLE_SYSTEM_SPINBUTTON Spinner
ROLE_SYSTEM_SPLITBUTTON SplitButton
ROLE_SYSTEM_STATUSBAR StatusBar
ROLE_SYSTEM_PAGETABLIST Tab
ROLE_SYSTEM_PAGETAB TabItem
ROLE_SYSTEM_TABLE Table
ROLE_SYSTEM_STATICTEXT Text
ROLE_SYSTEM_INDICATOR Thumb
ROLE_SYSTEM_TITLEBAR TitleBar
ROLE_SYSTEM_TOOLBAR ToolBar
ROLE_SYSTEM_TOOLTIP ToolTip
ROLE_SYSTEM_OUTLINE Tree
ROLE_SYSTEM_OUTLINEITEM TreeItem
ROLE_SYSTEM_WINDOW Window

 

States and Properties

Microsoft Active Accessibility elements support a common set of properties. Some properties, such as accState, must describe different conditions, depending on the element role. Servers must implement all methods of IAccessible that return a property, even those properties that are not relevant to the element.

UI Automation defines additional properties, some of which correspond to states in Microsoft Active Accessibility. Some properties are common to all elements, but other properties are specific to control types and control patterns. A UI Automation provider does not have to implement irrelevant properties, but can return a null value for any properties it does not support. The UI Automation core service can obtain some properties from the default window provider, and these are amalgamated with properties explicitly implemented by the provider.

As well as supporting many more properties, UI Automation enables better performance by allowing properties to be cached.

The following table shows the correspondence between some properties in the two models. For descriptions of the UI Automation property IDs, see Automation Element Property Identifiers.

Active Accessibility property accessor UI Automation property ID Remarks
get_accKeyboardShortcut UIA_AccessKeyPropertyId or UIA_AcceleratorKeyPropertyId UIA_AccessKeyPropertyId takes precedence if both are present.
get_accName UIA_NamePropertyId
get_accRole UIA_ControlTypePropertyId See the previous table for mapping roles to control types.
get_accValue UIA_ValueValuePropertyId or UIA_RangeValueValuePropertyId Valid only for control types that support IUIAutomationValuePattern or IUIAutomationRangeValuePattern. Range values are normalized to 0-100, to be consistent with Microsoft Active Accessibility behavior. Values are represented as strings.
get_accHelp UIA_HelpTextPropertyId
accLocation UIA_BoundingRectanglePropertyId
get_accDescription Not supported. accDescription did not have a clear specification in Microsoft Active Accessibility, which resulted in servers placing different pieces of information in this property.
get_accHelpTopic Not supported.

 

The following table shows the UI Automation property IDs that correspond to Microsoft Active Accessibility object state constants.

Active Accessibility state UI Automation property Triggers WinEvent state change?
STATE_SYSTEM_CHECKED UIA_ToggleToggleStatePropertyId for check box. UIA_SelectionItemIsSelectedPropertyId for radio button. Y
STATE_SYSTEM_COLLAPSED UIA_ExpandCollapseExpandCollapseStatePropertyId (value = ExpandCollapseState_Collapsed). Y
STATE_SYSTEM_EXPANDED UIA_ExpandCollapseExpandCollapseStatePropertyId (value = ExpandCollapseState_Expanded or ExpandCollapseState_PartiallyExpanded). Y
STATE_SYSTEM_FOCUSABLE UIA_IsKeyboardFocusablePropertyId. N
STATE_SYSTEM_FOCUSED UIA_HasKeyboardFocusPropertyId. N
STATE_SYSTEM_HASPOPUP UIA_ExpandCollapseExpandCollapseStatePropertyId for menu items. N
STATE_SYSTEM_INVISIBLE UIA_IsOffscreenPropertyId (value = True and IUIAutomationElement::GetClickablePoint fails). N
STATE_SYSTEM_LINKED UIA_ControlTypePropertyId (value = UIA_HyperlinkControlTypeId). N
STATE_SYSTEM_MIXED UIA_ToggleToggleStatePropertyId (value = ToggleState_Indeterminate. N
STATE_SYSTEM_MOVEABLE UIA_TransformCanMovePropertyId. N
STATE_SYSTEM_MULTISELECTABLE UIA_SelectionCanSelectMultiplePropertyId. N
STATE_SYSTEM_OFFSCREEN UIA_IsOffscreenPropertyId. N
STATE_SYSTEM_PROTECTED UIA_IsPasswordPropertyId. N
STATE_SYSTEM_READONLY UIA_RangeValueIsReadOnlyPropertyId and UIA_ValueIsReadOnlyPropertyId. N
STATE_SYSTEM_SELECTABLE UIA_IsSelectionItemPatternAvailablePropertyId . N
STATE_SYSTEM_SELECTED UIA_SelectionItemIsSelectedPropertyId. N
STATE_SYSTEM_SIZEABLE UIA_TransformCanResizePropertyId. N
STATE_SYSTEM_UNAVAILABLE UIA_IsEnabledPropertyId. Y

 

For a complete list of property IDs, see Property Identifiers.

Events

Unlike Microsoft Active Accessibility, the event mechanism in UI Automation, does not rely on Windows event routing, which is closely tied to window handles, and does not require the client application to set up hooks. Subscriptions to events can be fine-tuned to particular parts of the tree, not just to particular events. Providers can also fine-tune raising events by keeping track of which events are being listened for.

It is also easier for clients to retrieve the elements that raise events because these are passed directly to the event callback. Properties of the element are prefetched automatically, if a cache request was supplied when the client subscribed to the event.

The following table shows the correspondence of Microsoft Active Accessibility event constants and UI Automation event IDs.

WinEvent UI Automation Event ID
EVENT_OBJECT_ACCELERATORCHANGE UIA_AcceleratorKeyPropertyId property change.
EVENT_OBJECT_CONTENTSCROLLED UIA_ScrollVerticalScrollPercentPropertyId or UIA_ScrollHorizontalScrollPercentPropertyId property change on the associated scroll bars.
EVENT_OBJECT_CREATE UIA_StructureChangedEventId.
EVENT_OBJECT_DEFACTIONCHANGE No equivalent.
EVENT_OBJECT_DESCRIPTIONCHANGE No exact equivalent; perhaps UIA_HelpTextPropertyId or UIA_LocalizedControlTypePropertyId property change.
EVENT_OBJECT_DESTROY UIA_StructureChangedEventId.
EVENT_OBJECT_FOCUS UIA_AutomationFocusChangedEventId.
EVENT_OBJECT_HELPCHANGE UIA_HelpTextPropertyId change.
EVENT_OBJECT_HIDE UIA_StructureChangedEventId.
EVENT_OBJECT_LOCATIONCHANGE UIA_BoundingRectanglePropertyId property change.
EVENT_OBJECT_NAMECHANGE UIA_NamePropertyId property change.
EVENT_OBJECT_PARENTCHANGE UIA_StructureChangedEventId.
EVENT_OBJECT_REORDER Not consistently used in Microsoft Active Accessibility. No directly corresponding event is defined in UI Automation.
EVENT_OBJECT_SELECTION UIA_SelectionItem_ElementSelectedEventId.
EVENT_OBJECT_SELECTIONADD UIA_SelectionItem_ElementAddedToSelectionEventId.
EVENT_OBJECT_SELECTIONREMOVE UIA_SelectionItem_ElementRemovedFromSelectionEventId.
EVENT_OBJECT_SELECTIONWITHIN No equivalent.
EVENT_OBJECT_SHOW UIA_StructureChangedEventId.
EVENT_OBJECT_STATECHANGE Various property-changed events.
EVENT_OBJECT_VALUECHANGE UIA_RangeValueValuePropertyId and UIA_ValueValuePropertyId changed.
EVENT_SYSTEM_ALERT No equivalent.
EVENT_SYSTEM_CAPTUREEND No equivalent.
EVENT_SYSTEM_CAPTURESTART No equivalent.
EVENT_SYSTEM_CONTEXTHELPEND No equivalent.
EVENT_SYSTEM_CONTEXTHELPSTART No equivalent.
EVENT_SYSTEM_DIALOGEND UIA_Window_WindowClosedEventId.
EVENT_SYSTEM_DIALOGSTART UIA_Window_WindowOpenedEventId.
EVENT_SYSTEM_DRAGDROPEND No equivalent.
EVENT_SYSTEM_DRAGDROPSTART No equivalent.
EVENT_SYSTEM_FOREGROUND UIA_AutomationFocusChangedEventId.
EVENT_SYSTEM_MENUEND UIA_MenuModeEndEventId.
EVENT_SYSTEM_MENUPOPUPEND UIA_MenuClosedEventId.
EVENT_SYSTEM_MENUPOPUPSTART UIA_MenuOpenedEventId.
EVENT_SYSTEM_MENUSTART UIA_MenuModeStartEventId.
EVENT_SYSTEM_MINIMIZEEND UIA_WindowWindowVisualStatePropertyId property change.
EVENT_SYSTEM_MINIMIZESTART UIA_WindowWindowVisualStatePropertyId property change.
EVENT_SYSTEM_MOVESIZEEND UIA_BoundingRectanglePropertyId property change.
EVENT_SYSTEM_MOVESIZESTART UIA_BoundingRectanglePropertyId property change.
EVENT_SYSTEM_SCROLLINGEND UIA_ScrollVerticalScrollPercentPropertyId or UIA_ScrollHorizontalScrollPercentPropertyId property change.
EVENT_SYSTEM_SCROLLINGSTART UIA_ScrollVerticalScrollPercentPropertyId or UIA_ScrollHorizontalScrollPercentPropertyId property change.
EVENT_SYSTEM_SOUND No equivalent.
EVENT_SYSTEM_SWITCHEND No equivalent, but a UIA_AutomationFocusChangedEventId event signals that a new application has received the focus.
EVENT_SYSTEM_SWITCHSTART No equivalent.
No equivalent. UIA_MultipleViewCurrentViewPropertyId property change.
No equivalent. UIA_ScrollHorizontallyScrollablePropertyId property change.
No equivalent. UIA_ScrollVerticallyScrollablePropertyId property change.
No equivalent. UIA_ScrollHorizontalScrollPercentPropertyId property change.
No equivalent. UIA_ScrollVerticalScrollPercentPropertyId property change.
No equivalent. UIA_ScrollHorizontalViewSizePropertyId property change.
No equivalent. UIA_ScrollVerticalViewSizePropertyId property change.
No equivalent. UIA_ToggleToggleStatePropertyId property change.
No equivalent. UIA_WindowWindowVisualStatePropertyId property change
No equivalent. UIA_AsyncContentLoadedEventId event.
No equivalent. UIA_ToolTipOpenedEventId event.

 

Accessing Active Accessibility Properties and Objects from UI Automation

A key feature of UI Automation that is not available in Microsoft Active Accessibility is the ability to fetch multiple properties with a single cross-process operation.

Existing Microsoft Active Accessibility clients can take advantage of this ability by using the IUIAutomationLegacyIAccessiblePattern interface. This interface represents a control pattern that exposes Microsoft Active Accessibility properties and methods on UI elements. When retrieving elements, an application can request that this control pattern and its properties be cached.

IUIAutomationLegacyIAccessiblePattern also enables clients to obtain Microsoft Active Accessibility properties from elements that have no native support for IAccessible.

Changes in the properties of an IUIAutomationLegacyIAccessiblePattern do not raise UI Automation events.

Adding UI Automation Functionality to Active Accessibility Servers

UI Automation Tree Overview

UI Automation Properties Overview

UI Automation Control Types Overview

UI Automation Events Overview

Microsoft Active Accessibility