the future of software testing (part 5)

  • [アーティクル]

Visualization

What does software look like? Wouldn’t it be helpful if we had a visualization of software that we could use while the software was being constructed or tested? With a single glance we could see that parts of it remain unfinished. Dependencies, interfaces and data would be easy to see and, one would hope, easier to test. At the very least we could watch the software grow and evolve as it was being built and watch it consume input and interact with its environment as it was being tested.

Other engineering disciplines have such visuals. Consider the folks who make automobiles. Everyone involved in the assembly process can see the car. They can see that it has yet to have bumpers or a steering wheel installed. They can watch it progress down the mechanized line from an empty shell to a fully functional product ready to be driven to a dealer. How much longer until it is complete? Well, its forty feet from the end of the line!

The fact that everyone involved in making the car has this shared vision of the product is extremely helpful. They speak in terms they can all understand because every part, every connection, every interface is where it is supposed to be when it is supposed to be there.

Unfortunately, that is not our world. Questions or the sort asked above how long until it is complete? or what tasks remain undone? vex us. This is a problem that 21st century testers will solve.

Architects and developers are already solving it. Visual Studio is replete with diagrams and visualizations from sequence charts to dependency graphs. Testers are solving it too. Visualization solutions exist within the empire’s walls for seeing code changes in an Xbox title (objects whose code has churned glow green when rendered and then revert to normal after they have been tested) to identifying untested complexity within the Windows code base (heat maps of code coverage versus code complexity can be viewed in three dimensional space leading testers right to the problem areas). The visualizations are stunning, beautiful and allow testers to determine what needs testing simply by glancing at the visual.

We need more of this but we need to approach the problem carefully. We can’t simply accept the diagrams provided to us by the UML and modeling crowds. Those visuals are meant to solve other problems that may or may not overlap with the problems we face. Many of the existing visuals were created to serve architects or developers whose needs are different. We need to think this through as testers. We need visuals that map requirements to code, tests to interfaces, code churn to the GUI, and code coverage to controls. Wouldn’t it be nice to launch the app under test and be able to see controls glow with an intensity that reflects the amount of coverage or the number of tests that have touched them? Wouldn’t it be nice to be able to see a graphic animating network utilization or real time database communication? Why shouldn’t we be able to see the network traffic and the SQL queries as they happen? There is much that is going on unseen beneath the covers of our application and it's time we surfaced it and leveraged it to improve code quality.

This is an imminently solvable problem and one that many smart people are working on.

This is software testing in living color.

Comments

  • Anonymous
    September 19, 2008
    PingBack from http://www.easycoded.com/the-future-of-software-testing-part-5/

  • Anonymous
    September 23, 2008
    James Whittaker on the future of software testing (part 5) AjoyK on Index cards and Team System The...

  • Anonymous
    September 25, 2008
    So one thought towards this convergance and engraining of testing into the entire team at least from the design perspective, might be taking design patterns and lining up test patterns right along with them. The two are always discussed separately versus together. Thanks for your blogs and have a great day :) Bob Hanson

  • Anonymous
    October 08, 2008
    James, You’ve got a great knack for posting about issues which are very pertinent to testers at the moment. Virtualisation as a way to display coverage is something which we’re currently starting to look at; primarily because we’re struggling with showing how much coverage we’ve achieved of which areas of a product via any other way. We’re toying with the idea of some kind of graphic map which shows what’s been tested but as you point out, you also need some kind of representation for showing how much testing you’ve done in a particular area and what type of testing that’s been (was it functional, usability, performance etc). I plan to get around to looking into it sometime before Christmas but I hope people read this and reply with ways in which they visualise their testing / coverage because I feel it’s one of the great un-cracked challenges which face testing today. Simon

  • Anonymous
    February 03, 2009
    [Nacsa Sándor, 2009. január 13. – február 3.]  A minőségbiztosítás kérdésköre szinte alig ismert

  • Anonymous
    February 05, 2009
    [ Nacsa Sándor , 2009. február 6.] Ez a Team System változat a webalkalmazások és –szolgáltatások teszteléséhez

  • Anonymous
    July 21, 2010
    Something like this? gwaredd.freehostingcloud.com/heatmaps