Are Web Tests "Functional" tests?

  • [アーティクル]

When most people talk about automated functional testing, they are talking about UI Automated functional testing, where the test drives the UI (clicks buttons, types in text, etc.).

In fact there are many other strategies for functional testing, and most “unit tests” written today are not true Unit tests (with a capital U) that isolate and only test a single class, but test from the API down. For example, a unit test that tests a web service, or a client API that hits a server, are really functional tests.

Web tests work at the http layer, as Sean explains in this blog post. Really a better name for these is "Web Server Tests", or "Web Application Tests" since they actually test the server. The first thing users think when they see "web test" is a test that drives the browser. The browser preview window in web test playback only adds to this confusion.

Web tests do not actually instantiate IE when running, but only send and receive http requests. The main reason we chose to drive web tests at the HTTP layer is we wanted to be able to drive load efficiently in a load test. Instantiating IE, even just the guts of IE (the web browser control), consumes a lot more memory and constrains the amount of load that can be generated from a single machine. Web tests are super-efficient when running.

So what does and does not get tested by a web test? Obviously you can test your server code this way, which is super-valuable as that’s where most of your business logic is. Given a particular request, you can validate that the server sends back the correct response. However, since the test is not driven through IE, it will not test any client side code or behaviors, such as any java script code (including Ajax code). However, you can simulate the HTTP requests that Ajax will make and validate that the server is sending back the right response. Web tests do validation using Validation Rules, which enable you to validate the response sent back by the server.

There are advantages and disadvantages to driving at the protocol layer vs. the UI. UI tests are easier to understand and debug, but more fragile. To really be effective at scripting web tests you have to understand what is happening at the http layer, which most developers and testers are not usually concerned with. Most are . Also doing things like adding validation is easier in a UI automation tool, harder in web tests (although this is more a tooling problem than protocol vs. UI, we have some features lined up to make this easier).

So the bottom line answer is "yes", web tests are functional tests, but they work at the http layer rather than the UI layer.

Comments