This is the first post in this new series of blog posts about continuous integration. A variety of topics related to continuous integration in software projects will be discussed as new posts are written. This first entry is about the unit testing of JavaScript for web browser applications. In particular, JsTestDriver. What it does, how you can use it, and the benefits it brings.

I was first introduced to JsTestDriver while reading Test Driven JavaScript Development, by Christian Johansen. I’ll refer to JsTestDriver as JSTD for the rest of the post.

What is JSTD?

JSTD is a Java based server packaged as a .jar file. It is not another unit testing framework. To use JSTD you will have to choose a unit testing framework that is compatible with JSTD. JSTD has several unit testing frameworks that it is compatible with. Of course, you are free to build your own adapter if your unit testing framework of choice is not supported. Personally I’ve been using JSTD with the Jasmine BDD framework.

So, if JSTD is not a unit testing framework, and its described as a server, what does it do as a server? JSTD uses a capture url that you can navigate to with any web browser and ‘capture’ the browser so the JSTD server can run JavaScript with it while it is captured. This essentially makes the browser, or browser tab, a slave to JSTD.

Using JSTD

For example, if you started a JSTD server you would run a command on your command line that look something like this:

This would start up a new JSTD server on port 8080. You can double check that the server is running by navigating to http://localhost:8080 in your browser. You should see this page below:

Port 8080 is arbitrary, you can make it any port number that isn’t already in use. Also good to know is that JSTD will also use the next port up, so if you wanted to start a different server type you couldn’t do it on the next port, instead of 8081 you would use 8082. Not common but good to know.

Now that the server is running, the next step would be to capture a browser with JSTD so it can run JavaScript against it. To capture a browser with JSTD simply point your web browser to http://localhost:8080/capture or you can click one of the links in the index page.

Once you capture a browser the count will increase on the page that looks like below, the green bar indicating that you have established a connection with JSTD and the browser is waiting to run tests. This provides a lot of possibilities with the amount of browsers that you can run your JavaScript tests against. You can capture as many browsers as you want and each of your unit tests will be executed on each captured browser. Let me say that again… you can capture as many browsers as you want and each of your unit tests will be executed on each captured browser. This includes browsers from remote computers connecting to your JSTD server. This opens up a lot of possibilities in testing set ups, including using services like BrowserStack which has an API for starting up different browsers in virtual machines and running unit tests via JSTD on their machines. But I’ll get more into that in a different post.

Once you have captured browsers with JSTD you can then run unit tests against those captured browsers.

Configuring JSTD for your project

JSTD uses a YAML configuration file where you can configure a lot of different aspects of your test run.

The configuration file above would be for a simple example, the structure of that project is below to help visualize what the files look like. The locations of everything can be changed, the conf file understands relative paths so you can use paths that start with “../” and it will understand.

The JSTD.conf file lets us define which JavaScript source files to load in the first section named “-load”. This example is written with Jasmine so it is loading the Jasmine library JS file, then the JasmineAdapter.js, which is needed so JSTD can communicate with the Jasmine unit testing library. The lines after the adapter are for the JavaScript production files that you want to test.

The section after the “-load” list is “-test”, here is where you set paths to where JSTD should find unit test files to run. It is also a list so more paths can be added. Or you can make several .conf files that run different sets of tests.

It also has a “-timeout” setting you can set for the test run. You can set it to however many seconds you want to allow your test runs to take. You may have to increase it if your suite gets huge, I haven’t had to mess with it much.

The first line of the conf file is a “-server” setting. This lets you configure the address at which the tests should be run. Although you are using the .jar file to execute the tests you can point it at the server that was started using the same .jar file. It sounds confusing but being able to use the .jar file this way and with the .conf file the possibilities for build set ups is very flexible.

JSTD is very unobtrusive, so you don’t have to add anything to the tests that you would normally write in Jasmine or QUnit, or whatever your usual JSTD-supported unit testing framework of choice is. The spec file and the unit test remains the same, as you can see below, there is nothing added specifically for JSTD.

So without changing any of your unit tests you can start to test across multiple browsers, quickly, with JSTD and be ready to use JSTD in an automated CI set up. The JSTD wiki has examples for basic JSTD integration with Atlassian Bamboo and QUnit.

Running Jasmine with JSTD

Once your project is configured running the tests with JSTD is simple. The following command would run all the tests it found with the configuration specified, assuming the project structure pictured in the image above.

java -jar JsTestDriver-1.3.3d.jar –tests all –config jsTestDriver.conf

This would run the tests and you should see output that looks like this:

Of course if there are any errors the output will also display what you would expect from Jasmine normally, giving you stack trace info and failed test info, like below.

Continuous Integration setups

This example assumes that the CI set up you are using is running on a computer that is also running the web browsers that you will be running your unit tests against. The –browser flag lets you tell JSTD the paths to the web browser executables so it can open the browser and navigate to the capture URL. But what if you’re running JSTD on a headless Linux box and you want to run tests against multiple browsers? To do this you have to break up this command into several steps:

java -jar JsTestDriver.jar --port 4224 --browser open --tests all --testOutput .

This command is performing several steps. First it is starting a new JSTD server on port 4224. Then it is telling JSTD to open the default browser on the computer running JSTD to run the tests on. Next its instructing JSTD to run all of the tests that it can find within the definition of jsTestDriver.conf in the directory where this command was run from because there is no –config flag to define which .conf file to use. Finally, it is instructing JSTD to generate test output, using “.” to indicate that JSTD should generate those XML test reports in the directory where JSTD was run from. When this command completes it automatically shuts down the JSTD server that the command started at port 4224.

This sequence of events needs to be broken up into separate steps if you want to use JSTD to run tests on browsers that are on a different computer, or if you just want more control over the different steps as you prepare a test run in a CI configuration, like a Jenkins set up.

First you have to start the JSTD server. Second, you must automate the process of navigating a web browser to the JSTD capture url for each of the browsers that you want to run tests against. Once all the browsers are captured you can then tell JSTD to execute the tests. Finally, you’ll probably want to shut down the server, or you might want to leave it running, shutting it down at this point is personal preference. I like to shut it down. JSTD makes all of these steps easily separated into different commands.

Start server

java -jar JsTestDriver.jar --port 4224

Run Tests

java -jar JsTestDriver.jar --tests all --config myJstd.conf --testOutput path/to/xml/reports

Shut down

curl http://localhost:4224/quit

These JSTD commands are also easily integrated into CI builds using ANT, Phing, or whatever way your particular CI system lets you execute shell commands. I usually use ANT with Jenkins to integrate JSTD into my automated builds, perhaps another topic for another post.

IntelliJ Setup

There is an IntelliJ and an Eclipse plugin if you want a visual runner integrated with your IDE, or you can just run the tests via the command line using the JSTD commands. I find myself using the IntelliJ runner more, but sometimes I run it via Terminal when I’m also writing and running cucumber.js tests on the command line (another topic for another post).

To use the IntelliJ runner open Preferences and select Plugins from the list on the left of the window. Then press the Browse repositories… button and scroll down to the JsTestDriver Plugin entry. Right click it and select ‘Download and Install’.

Once its installed and IntelliJ has restarted there will be a tab at the bottom of IntelliJ labeled JsTestDriver Server. Click on it to open the JSTD server area. It will have a red bar indicating the server is not running, to turn it on press the green play button in the top left, the bar will turn yellow, indicating the server is running but no browsers are currently captured. The pane has icons for the different browsers, you can press the Google Chrome icon, for example, and it’ll open a tab in Chrome and capture the tab. Once you do that the bar will turn green and you’re ready to test.

To run the tests from IntelliJ navigate to the JSTD .conf file and right click the file. Select the Run option to run the conf file and IntelliJ will run the tests and show the output in the test runner.

Hopefully everything is good with your code and you get a green bar, like below! Running tests directly from IntelliJ is a nice feature to have. I don’t like leaving the IDE if I don’t have to.

JSTD is pretty simple to use. Although there are many command line flags (http://code.google.com/p/js-test-driver/wiki/CommandLineFlags) there are basically four actions you do with it. Starting a server, capturing browsers, running tests, and shutting down. And since it doesn’t require that you change your existing Jasmine, or QUnit tests you can easily integrate JSTD into your workflows unobtrusively and start catching problems in your JavaScript in specific browser environments more efficiently.

The part which isn’t as easily or quickly set up is automating a web browsers to navigate to the JSTD /capture URL so the JSTD server can capture web browsers, which would happen between starting the server and running the tests. In my next post I will go into detail on setting up JSTD to run tests and how you could use BrowserStack’s API to automate capturing browsers and running tests across as many browsers as BrowserStack has available (it’s a lot!).

Feel free to post any questions you have in the comments and I’ll try my best to help!

Download Source

Check the example source code on GitHub: https://github.com/s9tpepper/BasicJSTDExample