Analyze and Report on Code Churn and Code Coverage Using the Code Churn and Run Coverage Perspectives
You can report on the software quality by using the Code Churn and Run Coverage perspectives from the SQL Server Analysis Services cube for Visual Studio Team Foundation Server. By using these perspectives, you can view just the measures, dimensions, and attributes that are associated with the changes in lines of codes and the extent to which code is covered in builds and test runs.
These perspectives are based on the relational tables that you can use to report on code changes and coverage as a property of the build, the build assembly or platform, the test run, or the changeset. For more information, see Code Churn Tables and Run Coverage Tables.
.png "Code Churn Measure Group") |
By using the Code Churn perspective, you can create reports that answer the following questions:
|
.png "Code Coverage Measure Group") |
By using the Run Coverage perspective, you can create reports that answer the following questions:
Note If your data warehouse for Visual Studio Application Lifecycle Management (ALM) is using SQL Server Enterprise Edition, the list of cubes will include Team System and a set of perspectives. The perspectives provide a focused view of the data so that you do not have to scroll through all of the dimensions and measure groups in the whole Team System cube. |
In this topic
Example: Code Churn Report
Code Churn Measures
Run Coverage Measures
Dimensions and Attributes in the Code Churn Perspective That Support Filtering, and Categorization
Dimensions and Attributes in the Run Coverage Perspective That Support Filtering and Categorization
Required Activities for Monitoring Code Churn and Code Coverage
Example: Code Churn Report
By using a PivotChart report in Excel, you can create a trend report that displays the code churn over time, similar to the report that the following illustration shows.
.png "Code Churn Report")
The process templates for Microsoft Solutions Framework (MSF) v5.0 automatically provide the Code Churn report in Excel. For more information, see Code Churn Excel Report.
Back to top
Selecting and Filtering Pivot Fields
.png "Pivot Fields for Code Churn Report")
You can create a code churn report by performing the following steps:
In Excel, connect to the SQL Server Analysis Services cube for Visual Studio Team Foundation Server, and insert a PivotChart report.
For more information, see Create a Report in Microsoft Excel for Visual Studio ALM.
Right-click the chart and then choose Change Chart Type, Area, Stacked Area.
For each report filter, open the shortcut menu for each of the following fields, specify the hierarchies, weeks, or other elements of interest, and then drag the field to the Report Filter area.
Team Project Hierarchy from the Team Project dimension
Work Item.Iteration Hierarchy from the Work Item dimension
Work Item.Area Hierarchy from the Work Item dimension
Year Week Date from the Date dimension
In the Date dimension, expand More fields, and drag the Date, Week, or Month fields to the Axis Fields (Categories) area based on how granular a report you want to generate.
Drag the Lines Added, Lines Modified, and Lines Deleted fields from the Code Churn measure group to the Values area. You must drag each field separately.
Back to top
Code Churn Measures
Code churn measures quantify how much change is occurring in your project. In general, high levels of churn indicate project instability. You should expect high rates of churn at the start of a product cycle or after the team has implemented many changes. Toward the end of an iteration or before a release, you should expect the level of churn to decrease, which indicates that your project is more stable.
The following table describes the measures in the Code Churn measure group. By using these measures, you can create reports that show how many file versions are stored in Team Foundation version control and how much the code has changed. You can analyze metrics by file directory, build, or team member who checked in changes, and you can determine how those metrics change over time.
For information about similar metrics that you can collect for builds, see Analyze and Report on Build Details and Build Coverage Using the Build Perspective.
Measure |
Description |
|---|---|
Code Churn Count |
The number of times that the team changed files in version control. |
Lines Added |
The number of lines of code that the team added to files for the dimensions that you specify. |
Lines Deleted |
The number of lines of code that the team deleted from files for the dimensions that you specify. |
Lines Modified |
The number of lines of code that the team modified during the time period that you specify. |
Total Churn |
Churn in the code, computed as: [Lines Added] + [Lines Deleted] + [Lines Modified]. |
Total Lines |
The number of lines in the part of the file path hierarchy that you specify. You must also specify one or more builds to indicate the point or points at which to perform this calculation. If you do not specify one or more builds, NULL is returned. The number of lines is calculated by aggregating the lines added and lines deleted that have contributed to a specific combination of build type and operating system. Tip The Total Lines measure can cause the OLAP query to timeout. If your report takes too long to render, consider shortening the changeset, build, test run, or date range. |
Back to top
Run Coverage Measures
The following table describes the measures in the Run Coverage measure group. By using these measures, you can create reports that show the extent to which the code was covered by tests in a test run. For information about similar metrics that you can collect for builds, see Analyze and Report on Build Details and Build Coverage Using the Build Perspective.
Measure |
Description |
|---|---|
Run Coverage |
The number of test runs that have code coverage statistics associated with them. |
Run Coverage Blocks Covered |
The number of blocks that all tests in a run cover. However, coverage across the tests might overlap. |
Run Coverage Blocks Not Covered |
The number of blocks that are not covered by any tests in a run. However, coverage across the tests might overlap. |
Run Coverage Lines Covered |
The number of lines that all tests in a run cover. However, coverage across the tests might overlap. |
Run Coverage Lines Not Covered |
The number of lines that are not covered by any tests in a run. However, coverage across the tests might overlap. |
Run Coverage Lines Partially Covered |
The number of lines that tests in a run partially cover. However, coverage across the tests might overlap. |
Back to top
Dimension and Attributes in the Code Churn Perspective That Support Filtering and Categorization
The following table describes the dimensions and attributes in the Code Churn perspective. These attributes supplement the Team Project and Date shared dimensions, which Working with Shared Dimensions describes. You can aggregate the measures along each of these attributes.
Dimension |
Attribute |
Description |
|---|---|---|
Build |
Build Definition Name |
The name that is assigned to the build definition for which a build was run. |
Build ID |
The number that is assigned to the build. Each time that a particular build definition is run, this attribute is incremented by 1. |
|
Build Name |
The name or expression that uniquely identifies a build. For more information, see Work with Build Numbers. |
|
Build Start Time |
The date and time when the build started. |
|
Build Type |
The reason why the build was run. Build types are associated with the trigger that was defined for the build. Team Foundation Server supports the following types of builds: manual, continuous (triggered by every check-in), rolling (accumulate check-ins until the previous build finishes), gated check-in, and scheduled. For more information, see Specify Build Triggers and Reasons. |
|
Drop Location |
The Uniform Resource Locator (URL) for the completed build. A URL specifies the protocol with which web browsers will locate Internet resources. Each URL includes the name of the server on which the details of the build reside. You can also include the path to a resource. |
|
Version Control Changeset |
Changeset ID |
The number that is assigned to the changeset that included the file changes. |
Checked In By |
The user name of the team member who checked in the changeset. |
|
Description |
The check-in comment that is associated with the changeset. |
|
Policy Override Comment |
The comment that is provided when a policy is overridden. If a policy was not overridden with this changeset, this field is null. |
|
Version Control File |
Version Control File.File Hierarchy |
The full network path of the source file. |
Version Control File.File Extension |
The extension of the name of the source file. |
|
Work Item |
Work Item Type and more |
For more information, see Analyze and Report on Work Item and Test Case Data Using the Work Item Perspective. |
Back to top
Dimensions and Attributes in the Run Coverage Perspective That Support Filtering and Categorization
The following table describes the dimensions and attributes in the Run Coverage perspective. These attributes supplement the Team Project and Date shared dimensions that Working with Shared Dimensions describes later in this topic. You can aggregate the measures along each of these attributes.
Note
Before you can use the Assembly or Build Flavor attributes, the test team must specify them and publish the test results to the data store for Team Foundation Server. For more information, see Required Activities for Managing Builds and Tests later in this topic.
Dimension |
Attribute |
Description |
|---|---|---|
Assembly |
Assembly |
(Published test results only) The name of the code of the application that is tested as part of the build. For more information, see Run Tests in Your Build Process. |
Build |
Build Definition Name |
The name that is assigned to the build definition for which a build was run. |
Build ID |
The number that is assigned to the build. Each time that a particular build definition is run, the Build ID is incremented by 1. |
|
Build Name |
The name or expression that uniquely identifies a build. For more information, see Work with Build Numbers. |
|
Build Start Time |
Date and time when the build started. |
|
Build Type |
The reason why the build was run. Build types are associated with the trigger that was defined for the build. Team Foundation Server supports the following types of builds: manual, continuous (triggered by every check-in), rolling (accumulate check-ins until the previous build finishes), gated check-in, and scheduled. For more information, see Specify Build Triggers and Reasons. |
|
Drop Location |
The Uniform Resource Locator (URL) for the completed build. A URL specifies the protocol with which web browsers will locate Internet resources. The URL also includes the name of the server on which the resource resides. You can also specify the path to a resource. |
|
Build Flavor |
Build Flavor |
(Published test results only) A name that designates the category that is assigned to a set of completed builds that were published as part of a test run. For example, you can use a build flavor to designate a beta release or final release. For more information, see Command-Line Options for Publishing Test Results. |
Build Platform |
Build Platform |
(Published test results only) The name of the machine platform for which an end-to-end (not desktop) build was made and that was published as part of a test run (for example, x86 or Any CPU). For an example of a report that uses this attribute, see Build Summary Report. For more information, see Command-Line Options for Publishing Test Results. |
Test Run |
Complete Date Hierarchy by Month or by Week Creation Date Hierarchy by Month or by Week |
Date dimensions that are based on the date when the test run was created and finished. For more information, see Working with Shared Dimensions in the Analysis Services Cube. |
Back to top
Required Activities for Monitoring Code Churn and Code Coverage
To create build reports that contain useful data, team members must perform the following activities to manage builds and tests:
Configure a build system. To use Team Foundation Build, the team must set up a build system.
For more information, see Configure Your Build System.
Create build definitions. The team must create at least one build definition. The team can create several definitions, each of which can be run to produce code for a different platform or a different configuration.
For more information, see Create a Build Definition.
(Recommended) Run builds regularly. The team can automatically run builds at intervals that they specify or after every check-in. By using the schedule trigger, the team can automatically run builds at the same time or times on the same day or days that they specify. For more information, see Specify Build Triggers and Reasons and Run, Monitor, and Manage Builds.
(Optional) Define tests to run automatically as part of the build. As part of the build definition, the team can define automated tests to run as part of the build and analyze the impact of code changes on the tests.
For more information, see Run Tests in Your Build Process.
Configure tests to gather code coverage data. For code coverage data to appear in the report, team members must instrument tests to gather that data.
Important
To collect data about code coverage, the team must have installed Visual Studio Premium or Visual Studio Ultimate on the machine with the build agent. For more information, see Deploy and Configure Build Agents.
For more information, see How to: Configure Code Coverage (Visual Studio 2010) Using Test Settings in Visual Studio and How to: Gather Code-Coverage Data with Generic Tests.
Publish tests. As part of the build and test activities, the test team must publish test results to the data store for Team Foundation Server.
For more information, see Team Foundation Build Activities and Command-Line Options for Publishing Test Results.
Back to top
See Also
Concepts
Perspectives and Measure Groups Provided in the Analysis Services Cube for Team System