Using touch in Hilo (Windows Store apps using C++ and XAML)

  • Articolo

From: Developing an end-to-end Windows Store app using C++ and XAML: Hilo

Previous page | Next page

Hilo provides examples of tap, slide, swipe, pinch and stretch, and turn gestures. We use XAML data binding for connecting standard Windows controls that use touch gestures to the view model that implements those gestures. Here we briefly explain how we applied the Windows 8 touch language to Hilo to provide a great experience on any device.

Download

After you download the code, see Getting started with Hilo for instructions.

You will learn

  • How the Windows 8 touch language was used in Hilo.
  • How the Windows Runtime supports non-touch devices.

Applies to

  • Windows Runtime for Windows 8
  • Visual C++ component extensions (C++/CX)
  • XAML

Part of providing a great experience means that the app is accessible and intuitive to use on a traditional desktop computer, and on a small tablet. For Hilo, we put touch at the forefront of our UX planning because we felt that touch was key to providing an engaging interaction between the user and the app.

As described in Designing the Hilo UX, touch is more than simply an alternative to using the mouse. We wanted to make touch a first-class part of the app because touch can add a personal connection between the user and the app. Touch is also a very natural way to enable users to crop and rotate their photos. We also realized that Semantic Zoom would be a great way to help users navigate large sets of pictures in a single view. When the user uses the pinch gesture on the browse page, the app switches to a more calendar-based view. Users can then browse photos more quickly.

Hilo uses the Windows 8 touch language. We use the standard touch gestures that Windows provides for these reasons:

  • The Windows Runtime provides an easy way to work with them.
  • We didn't want to confuse users by creating custom interactions.
  • We want users to use the gestures they already know to explore the app, and not need to learn new gestures.

Note  

Using the Model-View-ViewModel (MVVM) pattern plays an important role in defining user interaction. We use views to implement the app UI and models and view models to encapsulate the app’s state and actions. For more info about MVVM, see Using the MVVM pattern.

 

The document Touch interaction design (Windows Store apps) explains the Windows 8 touch language. The following sections describe how we applied the Windows 8 touch language to Hilo.

[Top]

Press and hold to learn

This touch interaction enables you to display a tooltip, context menu, or similar element without committing the user to an action. We use press and hold on the image view page to enable the user to learn about a photo, for example, its filename, dimensions, and date taken.

We used the Popup control to implement press and hold. The resulting popup menu contains aGrid that binds to photo data. We didn't have to set any properties to initially hide the popup because popups don't appear by default.

Here's the XAML for the Popup.

ImageView.xaml

<Popup x:Name="ImageViewFileInformationPopup"
       AutomationProperties.AutomationId="ImageViewFileInformationPopup">
    <Popup.Child>
        <Border Background="{StaticResource GreySplashScreenColor}"
                BorderBrush="{StaticResource HiloBorderBrush}"
                BorderThickness="3">
            <Grid Margin="10">
                <Grid.ColumnDefinitions>
                    <ColumnDefinition Width="Auto"/>
                    <ColumnDefinition Width="Auto"/>
                </Grid.ColumnDefinitions>
                <Image Source="{Binding Path=CurrentPhotoImage.Thumbnail}" 
                       Height="105" 
                       Width="105" 
                       Stretch="UniformToFill" />
                <StackPanel Grid.Column="1" 
                            Margin="10,0,10,0">
                    <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}" 
                               FontWeight="Bold"
                               Text="{Binding Path=CurrentPhotoImage.Name}" 
                               TextWrapping="Wrap"/>
                    <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}"
                               Text="{Binding Path=CurrentPhotoImage.DisplayType}"/>
                    <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}" 
                               Text="{Binding Path=CurrentPhotoImage.Resolution}" />
                    <StackPanel Orientation="Horizontal">
                        <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}" 
                                   Text="{Binding Path=CurrentPhotoImage.FormattedDateTaken}" />
                        <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}"
                                   Margin="4,0,0,0"
                                   Text="{Binding Path=CurrentPhotoImage.FormattedTimeTaken}" />
                    </StackPanel>
                    <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}" 
                               Text="{Binding Path=CurrentPhotoImage.FileSize, Converter={StaticResource FileSizeConverter}}" />
                    <TextBlock Foreground="{StaticResource ApplicationForegroundThemeBrush}"
                               Margin="0,20,0,0"
                               Text="{Binding Path=CurrentPhotoImage.FormattedPath}" />
                </StackPanel>
            </Grid>
        </Border>
    </Popup.Child>
</Popup>

The OnImagePointerPressed, OnImagePointerReleased, and OnImagePointerMoved methods handle the PointerPressed, PointerReleased, and PointerMoved events, respectively. The OnImagePointerPressed method sets a flag to indicate that the pointer is pressed. If the left button is pressed (the IsLeftButtonPressed method always returns true for touch input), the popup is placed 200 pixels to the left and top of the current touch point.

ImageView.xaml.cpp

void Hilo::ImageView::OnImagePointerPressed(Object^ sender, PointerRoutedEventArgs^ e)
{
    m_pointerPressed = true;
    PointerPoint^ point = e->GetCurrentPoint(PhotoGrid);
    m_pointer = point->Position;
    if (point->Properties->IsLeftButtonPressed)
    {
        ImageViewFileInformationPopup->HorizontalOffset = point->Position.X - 200;
        ImageViewFileInformationPopup->VerticalOffset = point->Position.Y - 200;
        ImageViewFileInformationPopup->IsOpen = true;
    }
}

Note  We present the popup to the left of the pointer because most people are right-handed. If the user is touching with the index finger, the user's right hand could block any popup that appears to the right of the pointer.

 

The OnImagePointerReleased method closes the popup. It's called when the user releases the pointer.

ImageView.xaml.cpp

void Hilo::ImageView::OnImagePointerReleased(Object^ sender, PointerRoutedEventArgs^ e)
{
    if (m_pointerPressed)
    {
        ImageViewFileInformationPopup->IsOpen = false;
        m_pointerPressed = false;
    }
}

The OnImagePointerMoved method also closes the popup. The OnImagePointerPressed method sets a flag and saves the current pointer position because the OnImagePointerMoved is also called when the pointer is initially pressed. Therefore, we need to compare the current position to the saved position and close the popup only if the position has changed.

ImageView.xaml.cpp

void Hilo::ImageView::OnImagePointerMoved(Object^ sender, PointerRoutedEventArgs^ e)
{
    if (m_pointerPressed)
    {
        PointerPoint^ point = e->GetCurrentPoint(PhotoGrid);
        if (point->Position != m_pointer)
        {
            OnImagePointerReleased(sender, e);
        }
    }
}

The Popup also defines a Converter object that converts from bytes to kilobytes or megabytes to make the image size on disk more human-readable. We found converters to be a convenient way to bind to data, that we want to display in different ways. For more info about converters, see Data converters in this guide.

Why not use the Holding event?   We considered using the UIElement::Holding event. However, this event is for apps that respond only to touch input, and not other pointer input such as the mouse. We also looked at the GestureRegognizer::Holding event. However, GestureRecognizer isn't recommended for Windows Store apps using XAML because XAML controls provide gesture recognition functionality for you.

[Top]

Tap for primary action

Tapping an element invokes its primary action. For example, in crop mode, you tap the crop area to crop the image.

To implement crop, we placed a Grid element on top of the crop control and implemented the Tapped event. We use a Grid control so that the entire crop area receives tap events, not just the control that defines the crop rectangle.

Here's the XAML for the Grid.

CropImageView.xaml

<Grid Background="Transparent"
      HorizontalAlignment="Stretch"
      VerticalAlignment="Stretch"
      Tapped="OnCropRectangleTapped">
</Grid>

The view for the crop image page handles the Tapped event by forwarding the operation to the view model.

CropImageView.xaml.cpp

void CropImageView::OnCropRectangleTapped(Object^ sender, TappedRoutedEventArgs^ e)
{
    if (!m_cropImageViewModel->InProgress)
    {
        m_cropImageViewModel->CropImageAsync(Photo->ActualWidth);
    }
}

The view model for the crop image page crops the image and then calls Hilo's BindableBase::OnPropertyChanged method to communicate to the view that the operation has completed. Using the MVVM pattern, in this guide, explains how Hilo uses property changed notifications to communicate state changes.

CropImageViewModel.cpp

task<void> CropImageViewModel::CropImageAsync(float64 actualWidth)
{
    assert(IsMainThread());
    ChangeInProgress(true);

    // Calculate crop values
    float64 scaleFactor = m_image->PixelWidth / actualWidth;
    unsigned int xOffset = safe_cast<unsigned int>((m_cropOverlayLeft - m_left) * scaleFactor);
    unsigned int yOffset = safe_cast<unsigned int>((m_cropOverlayTop - m_top) * scaleFactor);
    unsigned int newWidth = safe_cast<unsigned int>(m_cropOverlayWidth * scaleFactor); 
    unsigned int newHeight = safe_cast<unsigned int>(m_cropOverlayHeight * scaleFactor);

    if (newHeight < MINIMUMBMPSIZE || newWidth < MINIMUMBMPSIZE)
    {
        ChangeInProgress(false);
        m_isCropOverlayVisible = false;
        OnPropertyChanged("IsCropOverlayVisible");
        return create_empty_task();
    }

    m_cropX += xOffset;
    m_cropY += yOffset;

    // Create destination bitmap
    WriteableBitmap^ destImage = ref new WriteableBitmap(newWidth, newHeight);

    // Get pointers to the source and destination pixel data
    byte* pSrcPixels = GetPointerToPixelData(m_image->PixelBuffer, nullptr);
    byte* pDestPixels = GetPointerToPixelData(destImage->PixelBuffer, nullptr);
    auto oldWidth = m_image->PixelWidth;

    return create_task([this, xOffset, yOffset, newHeight, newWidth, oldWidth, pSrcPixels, pDestPixels] () {
        assert(IsBackgroundThread());
        DoCrop(xOffset, yOffset, newHeight, newWidth, oldWidth, pSrcPixels, pDestPixels);
    }).then([this, destImage](){
        assert(IsMainThread());

        // Update image on screen
        m_image = destImage;
        OnPropertyChanged("Image");
        ChangeInProgress(false);
    }, task_continuation_context::use_current()).then(ObserveException<void>(m_exceptionPolicy));
}

[Top]

Slide to pan

Hilo uses the slide gesture to navigate between images in a collection. For example, when you browse to an image, you can use the slide gesture to navigate to the previous or next image in the collection.

When you view an image, you can also quickly pan to any image in the current collection by using the filmstrip that appears when you activate the app bar. We used the GridView control to implement the filmstrip.

Here's the XAML for the GridView.

ImageView.xaml

<GridView x:Name="PhotosFilmStripGridView"
          Grid.Column="1"
          AutomationProperties.AutomationId="PhotosFilmStripGridView"
          IsItemClickEnabled="False"
          ItemContainerStyle="{StaticResource FilmStripGridViewItemStyle}"
          ItemsSource="{Binding Photos}"
          SelectedItem="{Binding Path=SelectedItem, Mode=TwoWay}"
          SelectionMode="Single"
          VerticalAlignment="Center">
    <GridView.ItemsPanel>
        <ItemsPanelTemplate>
            <VirtualizingStackPanel Height="138" Orientation="Horizontal" />
        </ItemsPanelTemplate>
    </GridView.ItemsPanel>
    <GridView.ItemTemplate>
        <DataTemplate>
            <Border>
                <Image Source="{Binding Path=Thumbnail}" 
                       Height="138" 
                       Width="200" 
                       Stretch="UniformToFill" />
            </Border>
        </DataTemplate>
    </GridView.ItemTemplate>
</GridView>

A benefit of using the GridView control is that it has touch capabilities built in, removing the need for additional code or logic.

[Top]

Swipe to select, command, and move

With the swipe gesture, you slide your finger perpendicular to the panning direction to select objects. In Hilo, when a page contains multiple images, you can use this gesture to select one image. When you display the app bar, the commands that appear apply to the selected image. ListView, GridView, and other controls provide built-in support for selection. You can use the SelectedItem property to retrieve or set the selected item.

[Top]

Pinch and stretch to zoom

Pinch and stretch gestures are not just for magnification, or performing "optical" zoom. Hilo uses Semantic Zoom to help navigate between large sets of pictures. Semantic Zoom enables you to switch between two different views of the same content. You typically have a main view of your content and a second view that allows users to quickly navigate through it. (Read Adding SemanticZoom controls for more info on Semantic Zoom.)

On the image browser page, when you zoom out, the view changes to a calendar-based view. The calendar view highlights those months that contain photos. You can then zoom back in to the image-based month view.

To implement Semantic Zoom, we used the SemanticZoom control. You provide ZoomedInView and ZoomedOutView sections in your XAML to define the zoomed-in and zoomed-out behaviors, respectively.

For the zoomed-in view, we display a GridView that binds to photo thumbnails that are grouped by month. The grid view also shows a title (the month and year) for each group. The ImageBrowserViewModel::MonthGroups property and Hilo's MonthGroup class define which photos are displayed on the grid. The MonthGroup class also defines the title of the group.

Here's the XAML for the ZoomedInView.

ImageBrowserView.xaml

<SemanticZoom.ZoomedInView>
    <GridView x:Name="MonthPhotosGridView" 
              AutomationProperties.AutomationId="MonthPhotosGridView"
              AutomationProperties.Name="Month Grouped Photos"
              ItemsSource="{Binding Source={StaticResource MonthGroupedItemsViewSource}}" 
              ItemContainerStyle="{StaticResource HiloGridViewItemStyle}"
              ItemClick="OnPhotoItemClicked"
              IsItemClickEnabled="True"
              Padding="116,0,40,46"
              SelectedItem="{Binding SelectedItem, Mode=TwoWay}"
              SelectionMode="Single">

The ItemsSource property specifies the items for the control. MonthGroupedItemsViewSource is a CollectionViewSource that provides the source data for the control.

ImageBrowserView.xaml

<CollectionViewSource
    x:Name="MonthGroupedItemsViewSource"
    d:Source="{Binding MonthGroups, Source={d:DesignInstance Type=local:DesignTimeData, IsDesignTimeCreatable=True}}"
    Source="{Binding MonthGroups}"
    IsSourceGrouped="true"
    ItemsPath="Items"/>

The MonthGroups property on the view model (ImageBrowserViewModel) specifies the data that is bound to the grid view for the zoomed-in view. MonthGroups is a collection of IPhotoGroup objects. Hilo defines the IPhotoGroup interface to provide the title of the group and info about each photo.

For the zoomed-out view, we display a GridView that binds to filled rectangles for calendar months that are grouped by year (Hilo's ImageBrowserViewModel::YearGroups property, YearGroup, and MonthBlock classes define the title and which months contain photos).

Here's the XAML for the ZoomedOutView.

ImageBrowserView.xaml

<SemanticZoom.ZoomedOutView>
    <GridView x:Name="YearPhotosGridView" 
              AutomationProperties.AutomationId="YearPhotosGridView"
              AutomationProperties.Name="Year Grouped Photos"
              ItemsSource="{Binding Source={StaticResource YearGroupedItemsViewSource}}"
              IsItemClickEnabled="True"
              Padding="116,0,40,46"
              SelectionMode="None">

The YearGroupedItemsViewSource is similar to MonthGroupedItemsViewSource, except that it binds to data that is grouped by year.

For more info about Semantic Zoom, see Quickstart: adding SemanticZoom controls, Adding SemanticZoom controls, Guidelines for Semantic Zoom, and XAML GridView grouping and SemanticZoom sample.

[Top]

Turn to rotate

On the rotate image view, you can use two fingers to rotate the image. When you release your fingers, the image snaps to the nearest 90-degree rotation.

The RotateImageView class defines the UX for rotating an image and the RotateImageViewModel class defines its view model. Because RotateImageView inherits from Control, it is set up to receive events when the user manipulates objects. To receive manipulation events on the image control for the rotation gesture, Hilo sets the ManipulationMode property in the XAML for the Image.

Here's the XAML for the Image.

RotateImageView.xaml

<Image x:Name="Photo" 
       AutomationProperties.AutomationId="ImageControl" 
       HorizontalAlignment="Center"
       ManipulationMode="Rotate"
       Margin="{Binding ImageMargin}"
       RenderTransformOrigin="0.5, 0.5"
       Source="{Binding Photo.Image}"  
       VerticalAlignment="Center">

The RotateImageView class overrides the OnManipulationDelta and OnManipulationCompleted methods to handle rotation events. The OnManipulationDelta method updates the current rotation angle and the OnManipulationCompleted method snaps the rotation to the nearest 90-degree value.

Caution  These event handlers are defined for the entire page. If your page contains more than one item, you would need additional logic to determine which object to manipulate.

 

RotateImageView.xaml.cpp

void RotateImageView::OnManipulationDelta(ManipulationDeltaRoutedEventArgs^ e)
{
    m_viewModel->RotationAngle += e->Delta.Rotation;
}

void RotateImageView::OnManipulationCompleted(ManipulationCompletedRoutedEventArgs^ e)
{
    m_viewModel->EndRotation();
}

RotateImageViewModel.cpp

void RotateImageViewModel::EndRotation()
{
    auto quarterTurns = (RotationAngle / 90);
    auto nearestQuarter = (int)floor(quarterTurns + 0.5) % 4;
    RotationAngle = (float64)nearestQuarter * 90;
}

A RenderTransform on the image binds the RotationAngle property on the view model to the displayed image.

RotateImageView.xaml

<Image.RenderTransform>
    <RotateTransform 
        x:Name="ImageRotateTransform"  
        Angle="{Binding RotationAngle}" />
</Image.RenderTransform>

Hilo uses a view model locator to associate views with their view models. For more info about how we use view model locators in Hilo, see Using the MVVM pattern.

[Top]

Swipe from edge for app commands

When there are relevant commands to show, Hilo shows the app bar when the user swipes from the bottom or top edge of the screen.

Every page can define a navigation bar, a bottom app bar, or both. For instance, Hilo shows both when you view an image and activate the app bar.

Here's the navigation bar:

Here's the bottom app bar for the same photo.

From XAML, use the Page::TopAppBar property to define the navigation bar and the Page::BottomAppBar property to define the bottom app bar. Each of these contains an AppBar control that holds the app bar's UI components. The following shows the XAML for the bottom app bar for the image view page. This app bar contains the commands to enter the modes to rotate, crop, or apply the cartoon effect to the image.

ImageView.xaml

<local:HiloPage.BottomAppBar>
    <AppBar x:Name="ImageViewBottomAppBar"
            x:Uid="AppBar"
            AutomationProperties.AutomationId="ImageViewBottomAppBar"
            Padding="10,0,10,0">
        <Grid>
            <StackPanel HorizontalAlignment="Left" 
                        Orientation="Horizontal">
                <Button x:Name="RotateButton"
                        x:Uid="RotateAppBarButton"
                        Command="{Binding RotateImageCommand}" 
                        Style="{StaticResource RotateAppBarButtonStyle}" 
                        Tag="Rotate" />
                <Button x:Name="CropButton"
                        x:Uid="CropAppBarButton"
                        Command="{Binding CropImageCommand}"
                        Style="{StaticResource CropAppBarButtonStyle}"
                        Tag="Crop" />
                <Button x:Name="CartoonizeButton"
                        x:Uid="CartoonizeAppBarButton"
                        Command="{Binding CartoonizeImageCommand}"
                        Style="{StaticResource CartoonEffectAppBarButtonStyle}"
                        Tag="Cartoon effect" />
                <Button x:Name="RotateButtonNoLabel"
                        Command="{Binding RotateImageCommand}" 
                        Style="{StaticResource RotateAppBarButtonNoLabelStyle}" 
                        Tag="Rotate"
                        Visibility="Collapsed">
                    <ToolTipService.ToolTip>
                        <ToolTip x:Uid="RotateAppBarButtonToolTip" />
                    </ToolTipService.ToolTip>
                </Button>
                <Button x:Name="CropButtonNoLabel"
                        Command="{Binding CropImageCommand}"
                        Style="{StaticResource CropAppBarButtonNoLabelStyle}"
                        Tag="Crop"
                        Visibility="Collapsed">
                    <ToolTipService.ToolTip>
                        <ToolTip x:Uid="CropAppBarButtonToolTip" />
                    </ToolTipService.ToolTip>
                </Button>
                <Button x:Name="CartoonizeButtonNoLabel"
                        Command="{Binding CartoonizeImageCommand}"
                        Style="{StaticResource CartoonEffectAppBarButtonNoLabelStyle}"
                        Tag="Cartoon effect"
                        Visibility="Collapsed">
                    <ToolTipService.ToolTip>
                        <ToolTip x:Uid="CartoonizeAppBarButtonToolTip" />
                    </ToolTipService.ToolTip>
                </Button>
            </StackPanel>
        </Grid>
    </AppBar>
</local:HiloPage.BottomAppBar>

Caution  In most cases, don't display the app bar if there are no relevant commands to show. For example, the main hub page binds the AppBar::IsOpen property to the MainHubViewModel::IsAppBarOpen property on the view model. AppBar::IsOpen controls whether the app bar is visible or not, and is set when an object is selected and cleared when no object is selected.

 

For more info about app bars, see Adding app bars and XAML AppBar control sample.

[Top]

Swipe from edge for system commands

Because touch interaction can be less precise than other pointing devices, such as the mouse, we maintained a sufficient margin between the app controls and the edge of the screen. We also strived to use the same margin on all pages. We relied on the project templates that come with Visual Studio to provide suggested margins. Because we didn't "crowd the edges" of the screen, the user can easily swipe from the edge of the screen to reveal the app bars, charms, or display previously used apps.

[Top]

What about non-touch devices?

We also wanted Hilo to be intuitive for users who use a mouse or similar pointing device. The built-in controls work equally well with the mouse and other pointing devices as they do with touch. So when you design for touch, you also get the mouse and pen/stylus for free.

For example, you can use the left mouse button to invoke commands. The Windows Runtime also provides mouse and keyboard equivalents for many commands. (For example, you can use the right mouse button to activate the app bar; holding the Ctrl key down while scrolling the mouse scroll wheel controls Semantic Zoom interaction.)

We used the PointerPoint class to get basic properties for the mouse, pen/stylus, and touch input. Here's an example where we use PointerPoint to get the pointer's position.

ImageView.xaml.cpp

void Hilo::ImageView::OnImagePointerPressed(Object^ sender, PointerRoutedEventArgs^ e)
{
    m_pointerPressed = true;
    PointerPoint^ point = e->GetCurrentPoint(PhotoGrid);
    m_pointer = point->Position;
    if (point->Properties->IsLeftButtonPressed)
    {
        ImageViewFileInformationPopup->HorizontalOffset = point->Position.X - 200;
        ImageViewFileInformationPopup->VerticalOffset = point->Position.Y - 200;
        ImageViewFileInformationPopup->IsOpen = true;
    }
}

[Top]