Automated unit testing with JavaScript

Javascript Jquery Unit Testing Ant Automation

Javascript Problem Overview

I'm trying to incorporate some JavaScript unit testing into my automated build process. Currently JSUnit works well with JUnit, but it seems to be abandonware and lacks good support for Ajax, debugging, and timeouts.

Has anyone had any luck automating (with Ant) a unit testing library such as YUI test, jQuery's QUnit, or jQUnit?

Note: I use a custom built Ajax library, so the problem with Dojo's DOH is that it requires you to use their own Ajax function calls and event handlers to work with any Ajax unit testing.

Javascript Solutions

Solution 1 - Javascript

I'm just about to start doing JavaScript TDD on a new project I am working on. My current plan is to use QUnit to do the unit testing. While developing the tests can be run by simply refreshing the test page in a browser.

For continuous integration (and ensuring the tests run in all browsers), I will use Selenium to automatically load the test harness in each browser, and read the result. These tests will be run on every checkin to source control.

I am also going to use JSCoverage to get code coverage analysis of the tests. This will also be automated with Selenium.

I'm currently in the middle of setting this up. I'll update this answer with more exact details once I have the setup hammered out.

Testing tools:

Solution 2 - Javascript

There are many JavaScript unit test framework out there (JSUnit, scriptaculous, ...), but JSUnit is the only one I know that may be used with an automated build.

If you are doing 'true' unit test you should not need AJAX support. For example, if you are using an RPC Ajax framework such as DWR, you can easily write a mock function:

function mockFunction(someArg, callback) {
var result = ...; // Some treatments
setTimeout(

function() { callback(result); },
300  // Some fake latency
);
}

And yes, JSUnit does handle timeouts: Simulating Time in JSUnit Tests

Solution 3 - Javascript

I'm a big fan of js-test-driver.

It works well in a CI environment and is able to capture actual browsers for cross-browser testing.

Solution 4 - Javascript

I recently read an article by Bruno using JSUnit and creating a JsMock framework on top of that... very interesting. I'm thinking of using his work to start unit testing my JavaScript code.

Mock JavaScript or How to unit test JavaScript outside the Browser environment

Solution 5 - Javascript

I just got Hudson CI to run JasmineBDD (headless), at least for pure JavaScript unit testing.

(Hudson running Java via shell, running Envjs, running JasmineBDD.)

I haven't got it to play nice with a big library yet, though, like prototype.

Solution 6 - Javascript

Look into YUITest

Solution 7 - Javascript

I am in agreement that JSUnit is kind of dying on the vine. We just finished up replacing it with YUI Test.

Similar to the example using qUnit, we are running the tests using Selenium. We are running this test independently from our other Selenium tests simply because it does not have the dependencies that the normal UI regression tests have (e.g. deploying the application to a server).

To start out, we have a base JavaScript file that is included in all of our test HTML files. This handles setting up the YUI instance, the test runner, the YUI.Test.Suite object as well as the Test.Case. It has methods that can be accessed via Selenium to run the test suite, check to see if the test runner is still running (results are not available until after it's done), and get the test results (we chose JSON format):

var yui_instance; // The YUI instance
var runner;  // The YAHOO.Test.Runner
var Assert;    // An instance of YAHOO.Test.Assert to save coding
var testSuite; // The YAHOO.Test.Suite that will get run.

/**
 * Sets the required value for the name property on the given template, creates
 * and returns a new YUI Test.Case object.
 *
 * @param template the template object containing all of the tests
 */
function setupTestCase(template) {
    template.name = "jsTestCase";
    var test_case = new yui_instance.Test.Case(template);
    return test_case;
}

/**
 * Sets up the test suite with a single test case using the given
 * template.
 *
 * @param template the template object containing all of the tests
 */
function setupTestSuite(template) {
    var test_case = setupTestCase(template);
      testSuite = new yui_instance.Test.Suite("Bond JS Test Suite");
      testSuite.add(test_case);
}

/**
 * Runs the YAHOO.Test.Suite
 */
function runTestSuite() {
    runner = yui_instance.Test.Runner;
    Assert = yui_instance.Assert;

    runner.clear();
    runner.add(testSuite);
    runner.run();
}

/**
 * Used to see if the YAHOO.Test.Runner is still running.  The
 * test results are not available until it is done running.
 */
function isRunning() {
    return runner.isRunning();
}

/**
 * Gets the results from the YAHOO.Test.Runner
 */
function getTestResults() {
    return runner.getResults(yui_instance.Test.Format.JSON);
}

As for the Selenium side of things, we used a parameterized test. We run our tests in both Internet Explorer and Firefox in the data method, parsing the test results into a list of Object arrays with each array containing the browser name, the test file name, the test name, the result (pass, fail or ignore) and the message.

The actual test just asserts the test result. If it is not equal to "pass" then it fails the test with the message returned from the YUI Test result.

@Parameters
public static List<Object[]> data() throws Exception {
    yui_test_codebase = "file:///c://myapppath/yui/tests";

    List<Object[]> testResults = new ArrayList<Object[]>();

    pageNames = new ArrayList<String>();
    pageNames.add("yuiTest1.html");
    pageNames.add("yuiTest2.html");

    testResults.addAll(runJSTestsInBrowser(IE_NOPROXY));
    testResults.addAll(runJSTestsInBrowser(FIREFOX));
    return testResults;
}

/**
 * Creates a Selenium instance for the given browser, and runs each
 * YUI Test page.
 *
 * @param aBrowser
 * @return
 */
private static List<Object[]> runJSTestsInBrowser(Browser aBrowser) {
    String yui_test_codebase = "file:///c://myapppath/yui/tests/";
    String browser_bot = "this.browserbot.getCurrentWindow()"
    List<Object[]> testResults = new ArrayList<Object[]>();
    selenium = new DefaultSelenium(APPLICATION_SERVER, REMOTE_CONTROL_PORT, aBrowser.getCommand(), yui_test_codebase);
    try {
        selenium.start();

        /*
         * Run the test here
         */
        for (String page_name : pageNames) {
            selenium.open(yui_test_codebase + page_name);
            //Wait for the YAHOO instance to be available
            selenium.waitForCondition(browser_bot + ".yui_instance != undefined", "10000");
            selenium.getEval("dom=runYUITestSuite(" + browser_bot + ")");

            // Output from the tests is not available until
            // the YAHOO.Test.Runner is done running the suite
            selenium.waitForCondition("!" + browser_bot + ".isRunning()", "10000");
            String output = selenium.getEval("dom=getYUITestResults(" + browser_bot + ")");

            JSONObject results = JSONObject.fromObject(output);
            JSONObject test_case = results.getJSONObject("jsTestCase");
            JSONArray testCasePropertyNames = test_case.names();
            Iterator itr = testCasePropertyNames.iterator();

            /*
             * From the output, build an array with the following:
             *     Test file
             *     Test name
             *     status (result)
             *     message
             */
            while(itr.hasNext()) {
                String name = (String)itr.next();
                if(name.startsWith("test")) {
                    JSONObject testResult = test_case.getJSONObject(name);
                    String test_name = testResult.getString("name");
                    String test_result = testResult.getString("result");
                    String test_message = testResult.getString("message");
                    Object[] testResultObject = {aBrowser.getCommand(), page_name, test_name, test_result, test_message};
                    testResults.add(testResultObject);
                }
            }
        }
    } finally {
        // If an exception is thrown, this will guarantee that the selenium instance
        // is shut down properly
        selenium.stop();
        selenium = null;
    }
    return testResults;
}

/**
 * Inspects each test result and fails if the testResult was not "pass"
 */
@Test
public void inspectTestResults() {
    if(!this.testResult.equalsIgnoreCase("pass")) {
        fail(String.format(MESSAGE_FORMAT, this.browser, this.pageName, this.testName, this.message));
    }
}

Solution 8 - Javascript

There's a new project that lets you run QUnit tests in a Java environment (like Ant) so you can fully integrate your client-side test suite with your other unit tests.

http://qunit-test-runner.googlecode.com

I've used it to unit test jQuery plugins, objx code, custom OO JavaScript and it works for everything without modification.

Solution 9 - Javascript

The project I'm working on uses Js-Test-Driver hosting Jasmine on Chrome 10 with Jasmine-JSTD-Adapter including making use of code coverage tests included in JS-Test-Driver.

While there are some problems each time we change or update browsers on the CI environment the Jasmine tests are running pretty smoothly with only minor issues with ansynchronous tests, but as far as I'm aware these can be worked around using Jasmine Clock, but I haven't had a chance to patch them yet.

Solution 10 - Javascript

I've published a little library for verifying browser-dependent JavaScript tests without having to use a browser. It is a Node.js module that uses zombie.js to load the test page and inspect the results. I've wrote about it on my blog. Here is what the automation looks like:

var browsertest = require('../browsertest.js').browsertest;

describe('browser tests', function () {

    it('should properly report the result of a mocha test page', function (done) {
        browsertest({
            url: "file:///home/liam/work/browser-js-testing/tests.html",
            callback: function() {
                done();
            }
        });
    });

});

Solution 11 - Javascript

I looked on your question date and back then there were a few good JavaScript testing libraries and frameworks.

Today you can find much more and in different focus like TDD, BDD, Assetion and with/without runners support.

There are many players in this game, like Mocha, Chai, QUnit, Jasmine, etc...

You can find some more information in this blog about JavaScript, mobile, and web testing...

Solution 12 - Javascript

Another JavaScript testing framework that can be run with Ant is CrossCheck. There's an example of running CrossCheck via Ant in the build file for the project.

CrossCheck attempts, with limited success, to emulate a browser, including mock-style implementations of XMLHttpRequest and timeout/interval.

It does not currently handle loading JavaScript from a web page, though. You have to specify the JavaScript files that you want to load and test. If you keep all of your JavaScript code separated from your HTML, it might work for you.

Solution 13 - Javascript

I've written an Ant task which uses PhantomJS, a headless WebKit browser, to run QUnit HTML test files within an Ant build process. It can also fail the build if any tests fail.

https://github.com/philmander/ant-jstestrunner

Solution 14 - Javascript

This is a good evaluation of several testing tools.

https://stackoverflow.com/questions/300855/looking-for-a-better-javascript-unit-test-tool

I personally prefer https://code.google.com/p/js-test-driver/

Content Type	Original Author	Original Content on Stackoverflow
Question	Chris MacDonald	View Question on Stackoverflow
Solution 1 - Javascript	Karl	View Answer on Stackoverflow
Solution 2 - Javascript	Alexandre Victoor	View Answer on Stackoverflow
Solution 3 - Javascript	groodt	View Answer on Stackoverflow
Solution 4 - Javascript	Elijah Manor	View Answer on Stackoverflow
Solution 5 - Javascript	Ingvald	View Answer on Stackoverflow
Solution 6 - Javascript	Hank Gay	View Answer on Stackoverflow
Solution 7 - Javascript	Josh	View Answer on Stackoverflow
Solution 8 - Javascript	Mat Ryer	View Answer on Stackoverflow
Solution 9 - Javascript	Steven de Salas	View Answer on Stackoverflow
Solution 10 - Javascript	liammclennan	View Answer on Stackoverflow
Solution 11 - Javascript	lastboy	View Answer on Stackoverflow
Solution 12 - Javascript	Jason Wadsworth	View Answer on Stackoverflow
Solution 13 - Javascript	Phil Mander	View Answer on Stackoverflow
Solution 14 - Javascript	Alexvx	View Answer on Stackoverflow