Custom UI Automation Providers in Depth: Part 2

  • Article

In Part 1, we introduced the idea of UI Automation custom control providers and created a TriColor control as a sample.  Now, we’re going to try to add one basic property: the “Hello World!” of UI Automation.  Here’s the sample code for this part.

To do this, we need to create a new object, called a provider, that answers questions on behalf of the control.  That object will implement the IRawElementProviderSimple interface, which defines its contract with UI Automation.  The interface name, although long, points out the important parts of this contract:

  • It is an ElementProvider, distinguishing it from other kinds of providers in Windows.
  • It is Raw, as opposed to the COM interface that a client would use.  This interface is intended to be easy to implement, rather than easy to consume.  (UI Automation itself provides the client objects and interfaces.)
  • This is the Simple interface.  There are more advanced interfaces with different suffixes.

There are just four methods on the interface, answering four basic questions about the object:

  • get_ProviderOptions(): What kind of provider is it? 
  • get_HostRawElementProvider(): What HWND does this belong to?
  • GetPropertyValue(): What are its properties?
  • GetPatternProvider(): What are its Control Patterns? (covered in a later lesson)

So, we simply declare a new object in our project, TriColorProvider, and start implementing this interface, starting with get_ProviderOptions:

        public const int ProviderOptionUseComThreading = 0x20;
        public override ProviderOptions ProviderOptions
        {
            // Request COM threading style - all calls on main thread
            get
            {
                return (ProviderOptions)((int)ProviderOptions.ServerSideProvider | ProviderOptionUseComThreading);
            }
        }

ServerSideProvider describes where our provider runs: inside the server (UI) process.  Also, I’m using a new Windows 7 feature here – I can tell UIA to use STA COM-style threading, such that all provider calls will be made to the thread that created my provider.  This avoids threading issues when I need to communicate with UI elements on the UI thread. 

The HostRawElementProvider method is similarly easy.  This function returns the HWND provider that corresponds to this control and supplies some basic properties for it, as we noted in the last post.  I just need to call the HostProviderFromHandle() function with my window handle and return its result.  At this point, though, I started thinking: this seems like cookie-cutter code.  If I ever have a second provider, I will have almost the same function body, but a different window handle. 

So, I refactored the class into a BaseSimpleProvider and a TriColorProvider.  The base provider will implement the interface literally, but delegate where useful to protected virtual methods, which the TriColorProvider can override.  This helps to separate what is common from what varies between classes.  If I ever need to write a second provider, I will have much less work to do.  I add the following method to BaseSimpleProvider:

        public IRawElementProviderSimple HostRawElementProvider
        {
            get
            {
                IntPtr hwnd = this.GetWindowHandle();
                if (hwnd != IntPtr.Zero)
                {
                    return AutomationInteropProvider.HostProviderFromHandle(this.GetWindowHandle());
                }
                else
                {
                    return null;
                }
            }
        }

And then the derived method to TriColorProvider:

        protected override IntPtr GetWindowHandle()
        {
            // Return our window handle, since we're a root provider
            return this.control.Handle;
        }

This technique turns out to be extremely useful when we get to GetPropertyValue().  GetPropertyValue() needs to return the value of whatever property the caller requests.  Much like a window procedure, the body of GetPropertyValue() is a big switch statement that routes various property queries to the code that answers them.  To avoid making GetPropertyValue() complicated and long, we factor the various queries into separate methods.  To be clear, you don’t have to do it this way – you can do the work in the switch statement – but it smells bad to me.  Following our pattern above, we add the base GetPropertyValue() method to BaseSimpleProvider, along with a protected GetName() virtual method.  Once we do that, all that TriColorProvider  has to do to say Hello World! is override the name getter:

        protected override string GetName()
        {
            // This should be localized; it's hard-coded only for the sample.
            return "Hello world!";
        }

We’re almost done.  We have a provider object that provides a complete implementation of IRawElementProviderSimple.  Your code would compile now – but it wouldn’t do anything different.  We still need to hook up our new object.  We’ll add a field to TriColorControl and then a property wrapper that creates the provider on demand (saving memory if we never actually use it):

        private TriColorProvider provider;
        private IRawElementProviderSimple Provider
        {
            get
            {
                if (this.provider == null)
                {
                    this.provider = new TriColorProvider(this);
                }
                return this.provider;
            }
        }

Finally, we need to respond to the WM_GETOBJECT window message by returning our provider.  WM_GETOBJECT is the message requesting the Accessibility provider for an HWND – if you are familiar with MSAA, you’ve seen this before.  In UIA, there is one correct way to implement WM_GETOBJECT, and it looks like this:

            if (m.Msg == 0x3D /* WM_GETOBJECT */)
            {
                m.Result = AutomationInteropProvider.ReturnRawElementProvider(
                    m.HWnd, m.WParam, m.LParam, this.Provider);
            }

You do not need to examine WParam or LParam – just pass them straight through to ReturnRawElementProvider.  This is important to ensure that the UIA-to-MSAA Bridge, which helps MSAA clients to use your UIA object, works properly.  Also note that I’m passing back the same provider object every time.  You don’t have to do this, but creating a new one each time would be wasteful. 

And I’m done.  I can compile and look at my control with Inspect and see my “Hello World!” property coming through:

original[2]

We’ve successfully provided one property!   

Next time: The rest of the basic properties.

Comments

  • Anonymous
    November 26, 2010
    Hi There, We have a control that has MarshalByRefObject as its subitems, I have been looking around these posts to make our control work with Coded UI Test Builder (VS2010). My question is that the subitem does not have a window handle to supply to 'HostProviderFromHandle()' In that case what could be the solution or do i have to implement a different provider ? Please assist me with this. Thanks and Regards Vallarasu S.

  • Anonymous
    November 27, 2010
    Hi, I have them done with IRawElementProviderFragmentRoot, However i encounter another exception with the subitems. "could not find accessible object of foreground window ", any idea of what causes it? Is there any way i can tell the builder which item has the focus? Best Regards Vallarasu S.

  • Anonymous
    November 29, 2010
    Vallarasu, Blog comments aren't monitored as much as our MSDN forum is -- you might have more luck here: social.msdn.microsoft.com/.../threads Thanks, Michael

  • Anonymous
    March 14, 2011
    the link " Here’s the sample code for this part." is not working

  • Anonymous
    August 05, 2011
    The comment has been removed

  • Anonymous
    August 08, 2011
    Interesting suggestion. In the context of a standalone application, this cleanup is not strictly necessary.  When a process terminates, it's memory is always cleaned up.   But this cleanup is helpful if the application creates and destroys a lot of different HWNDs (which my sample does not).  I'll consider incorporating that the next time I revise this sample.

  • Anonymous
    September 09, 2011
    Hey All,    The Definition of IUIAutomationElement.UIA_RuntimeIdPropertyId states that: "Identifies the RuntimeId property, which is an array of integers representing the identifier for an automation element. The identifier is unique on the desktop, but it is guaranteed to be unique only within the UI of the desktop on which it was generated. Identifiers can be reused over time. The format of RuntimeId can change. The returned identifier should be treated as an opaque value and used only for comparison; for example, to determine whether an automation element is in the cache." The part about "Identifiers can be reused over time." is unclear to me, can anybody explain what that means more clearly, specifically i want to know if this is unique per instance of the given application? Is this runtimeid reused in the same instance of the given application? Thanks in advance, Ray

  • Anonymous
    September 12, 2011
    Hi, Ray, The point about identifier re-use is that you could have an element 'A' with a runtime ID (ID(A)), and then have 'A' be destroyed.  Later on, an element 'B' is created and it turns out to have the runtime ID = ID(A).   This commonly happens in containers, where the container's runtime ID might be a concatenation of the container's ID plus an item position.  If you have a list of items, and item #5 has a particular ID, and then you refresh the list, you might find that the new item #5 has the same runtime ID as the old item #5. Runtime IDs need to be unique at a given point in time, but not unique across all points in time.

  • Michael

  • Anonymous
    December 13, 2011
    Michael - is it possible to imlpement an automation provider for a Visio document?  One that would show the shapes as navigatable items, and properties such as location, context menus, etc? thanks, -es

  • Anonymous
    December 13, 2011
    Hi Eric, I don't think it's possible for you to inject your own UIA provider such that a UIA client interacting with Visio will get custom results from your provider. But I've just been pointing the Inspect SDK tool to Visio 2010 to see what's currently exposed in a diagram. I posted the results of my test at social.msdn.microsoft.com/.../318cf9ca-4c36-4492-badd-b4a2f7fc632f. Do you think that the existing Visio functionality might be sufficient for your scenario? Thanks, Guy

  • Anonymous
    March 15, 2012
    Can anyone help me with the bounding rectangle problem ? How to get the same for Toolbar button..plse help..

  • Anonymous
    March 19, 2012
    Please bring your question to the MSDN forum for Accessibility: social.msdn.microsoft.com/.../threads

  • Anonymous
    March 08, 2013
    I program mainly in Vb.net so this example was a little too complex for me because of the Tri-Color control and the way it was embedded in the code, it made it completely impossible to reverse engineer so i could understand. After about a month i found this Simple Provider Sample in VB.net. I hope this helps somone. technet.microsoft.com/.../ms771658(v=vs.90)

  • Anonymous
    September 19, 2013
    The comment has been removed