Saturday, September 15, 2012

knockout.composite: Functional Testing: Automated UI Tests, Anywhere, Any Time

*UPDATE* Tribe is here! Check out http://tribejs.com/ for guides and API reference.

This is part 7 of a multi-part post. It’s assumed that you have a basic knowledge of the data binding and observability features of knockout.js and have read parts 4 through 6.

Part 1: Introducing knockout.composite
Part 2: The Basics
Part 3: Getting Around
Part 4: Putting it All Together
Part 5: Unit Testing: Model Citizens 
Part 6: Integration Testing: Playing Nicely Together
Part 7: Functional Testing: Automated UI Tests, Anywhere, Any Time
Part 8: Pack Your App and Make it Fly
Part 9: Parting Thoughts and Future Directions

 

The JavaScript / HTML / CSS stack also makes it quite easy to perform automated UI testing – simply open your deployed app in an iframe element and we can then manipulate the UI with JavaScript, all driven by your favourite unit testing framework. This works very well in practice. Check out the yoursports.net functional test suite here and here. Click on each test to see the series of steps that was performed.

knockout.composite gives us an additional layer of useful stuff to plug into. Using the rendered and paneRendered events that are published on the event bus, we can ensure that our application is ready for manipulation and assertions.

A significant challenge to overcome is dealing with the asynchronous nature of applications. Trying to write this stuff in a procedural style with JavaScript ends up being a deeply nested mess of callbacks and deferred objects.

Instead, creating an API and specifying tests as a set of steps that are queued up and executed serially is a much simpler approach, and also allows reusing these sets of steps. Here is an example from the yoursports.net test suite.

var logon = [
    click(tabButton('Log On')),
    waitForVisible('.logon'),
    setValue('.logon input:eq(0)', 'demo'),
    setValue('.logon input:eq(1)', 'qweqwe'),
    click('.logon button:contains("Log On")'),
    waitForPane('/Team/details')
];

var logoff = [
    click(tabButton('Account')),
    waitForVisible('.account'),
    click('.account a:contains("Log off")'),
    waitForNotVisible('.contentHeader a:contains("Enter Scoresheet")')
];

functionalTest('Logon / Logoff', [
    logon,
    textEqual('.heading:eq(0) span', 'Team: Hit and Run', 'Correct team loaded'),
    logoff
]);

We can see that the API consists of a bunch of functions that correspond to either actions or assertions, and an entry point function, functionalTest.

Under the Covers

The functionalTest function performs all necessary setup for the test, primarily creating the iframe and waiting for our application to finish rendering. It accepts a test name and an array of steps. Each step can be either another array of steps or a function that returns another function to be queued for execution later.

If you’re not familiar with JavaScript, that might sound a bit confusing, but it is one of the powerful features of the language. For an in depth discussion of function currying in JavaScript, check out this Crockford article.

The functionalTest function also ensures that qunit if started once all steps have completed or an uncaught exception occurs.

Step by Step

So each individual step consists of a function that returns a function. Let’s have a look at an example.

function setValue(selector, value) {
    return function () {
        singleExists(selector)();
        ok(true, "Setting value of '" + selector + "' to " + value);
        $$(selector).val(value).change();
    };
};

This example sets the value of an element with a specified selector. The initial setValue call “loads up” our internal function with the necessary arguments and returns the internal function to the test framework for execution in series.

The singleExists function is another “step” function that asserts that only one and only one element exists with the specified selector. The $$ variable gives us access to the jQuery instance inside our iframe.

We can also return a jQuery deferred object from each step and the test framework will wait until the deferred is either resolved or rejected. We can use this to wait for specific events like rendering completion and elements becoming visible, etc. The implementation is somewhat beyond the scope of this article.

Builds, Environments and Browsers

The yoursports.net tests are run against a deployed service layer and a predefined set of data, so we are testing the integration of the entire system, from UI through to the data store. They can easily be deployed with the application and run as part of a build, targeting any environment and run from multiple browsers.

Part of the roadmap is for a simple test agent that can execute these tests in multiple browsers, taking screenshots at specified points in time, gather timings and provide an interface for reporting results.

Conclusion

I have to say at this point that this is still in an experimental stage. While it does work well, it can be somewhat painful to debug. Subtle temporal couplings can make things incredibly confusing! This will improve drastically in the next version of knockout.composite.

I have not yet made the source code for this available on github, but will do so in the near future. If you’re particularly keen, drop me a tweet.

Next time, the last piece, at least for now. Phew!

1 Comments:

Blogger andrea james said...

Clearly explained about Functional Testing. Keep posting :)

11:37 PM  

Post a Comment

<< Home