Karma Chameleon

Thanks to Karma [1], test-driven development should be "easy, fast and fun." The tool launches a web application written in JavaScript and automatically executes the tests specified by the developer in several browsers at the same time while logging the results. The browsers can even run on external devices such as smartphones or tablets, which lets you test your own code under real conditions.

For initial testing, Karma also launches the code, if so required, in a PhantomJS instance running without a GUI (headless operation). Karma also monitors the code and automatically re-executes the stored tests every time a change is made.

Delivered

Karma itself comprises several plugins, each of which solves a specific task, typically with the use of additional software. Karma also delegates the tests to established test frameworks, such as Jasmine [2], Mocha [3], or QUnit [4], relying on plugins to do so and negating the need for web developers to learn new software. You can also integrate in-house test frameworks as DIY plugins. On the same principle, Karma plugins support continuous integration with Jenkins [5], Travis [6], or Semaphore [7].

After launching, Karma first loads all the plugins and parses its configuration file before launching a web server in the background that serves up the web application to be tested. Karma then runs the browsers against the web server URL.

If required, web developers can also control the web server manually in the browser. Karma detects this action in the background and automatically adds these browsers to the tests.

The web server delivers a specially prepared page that uses web sockets in the browser to communicate with the web server. The page contains an iframe into which the web server injects another subpage that contains the code to be tested, the tests to be executed, and more code that controls the test execution itself. The code is part of an adapter plugin that passes the tests to the corresponding test framework. If, for example, the web developer chooses the Jasmine framework, the Jasmine adapter ensures that these tests run in the browsers.

The test results are fed back to the server via the web socket and are intercepted there by reporter plugins that process all results and return them in the desired format. In the simplest case, the reporter displays progress as a message on the command line.

On Call

The developers recommend installing Karma and all necessary plugins in the project directory of the web application to be tested. The only requirements are Node.js and its NPM package manager. The following two commands fetch Karma and drop it into the current project directory:

npm install karma --save-dev
npm install karma-jasmine jasmine-core karma-chrome-launcher --save-device

The second command loads three plugins: The test adapter karma-jasmine lets you use Jasmine as a test framework, jasmine-core fetches the Jasmine framework and drops it on your disk, and karma-chrome-launcher ensures that Karma can start Chrome independently. The packages ultimately end up in the project directory's node_modules subfolder. After the install, you can launch Karma by calling:

./node_modules/karma/bin/karma

To save typing, you can install the karma-cli package globally with

sudo npm install -g karma-cli

which sets up the karma command-line program that acts as an alias for the unwieldy command above, automatically invoking the currently available Karma version.

Questionable

Although you now have Karma on your hard drive, it still does not know what it is supposed to test – or how and where. A configuration file created by the

karma init karma.conf.js

command, gives the tool the exact details. You can choose the name of the configuration file to suit your needs.

As the extension indicates, this is a JavaScript file. Karma simply imports it when launched and evaluates the variables set there. Alternatively, the configuration can also exist in CoffeeScript or TypeScript format. In this case, simply give the configuration file a .coffee or .ts extension.

In all cases, you need to answer a few questions about the project posed by the wizard (Figure 1), including the test framework used and the browsers to be used. The storage location of the source code files can be added manually later, if required.

Within the configuration file, module.export points to a function that expects a configuration object. The function then sets the appropriate variables in the object (Figure 2). Their significance is explained by detailed comments. In the following example, Karma would load all JavaScript files from the folders js and tests:

files: [
        'js/*.js',
        'tests/*.js'
],

The files command lists all the files the browser has to load, including all tests. Wildcards like the asterisk encompass several files at once.

Browser Selection

The browsers keyword is followed by all the browsers in which you want Karma to test the code. Table 1 shows which names to add to the configuration file to cover the most important browsers. For browsers added in this way, a corresponding launcher is usually available. You also need the appropriate plugin.

To test the code in Firefox, you would add

browsers: ['Chrome', 'Firefox']

to the configuration file and then install the appropriate launcher from the command line:

npm install karma-firefox-launcher --save-dev

Table 1 shows the plugin required for each browser. Many more launchers can be found online [8]. As a headless browser, PhantomJS is particularly useful for quick initial tests during development. Because it does not start a GUI, it delivers the results significantly faster, which will not necessarily match those for Chrome, Firefox, and others.

Web developers can also launch any other browser by simply specifying the complete path to its binary file in the configuration file:

browsers: ['/usr/local/bin/<Exotic_Browser>.sh']

Another option simply controls the web server started by Karma with a browser. In this way, you can additionally test the browsers on mobile devices, tablet PCs, and other operating systems.

The test frameworks to be used follow frameworks. By default this is only jasmine; all available frameworks can be found online [9]. Web developers can also discover which adapter plugins they need to install with npm.

Covered

Karma can send the JavaScript code through preprocessors before the test run. In this way, for example, the test coverage can be determined with the Istanbul tool [10]. First, install a suitable plugin for Istanbul with the

npm install karma-coverage --save-dev

command. Then, you need to tell Karma to launch in Istanbul with the following settings in the configuration file:

preprocessors: {
    'js/*.js': 'coverage'
},

To prevent the results from Istanbul needlessly disappearing without a trace, you need to process them with a reporter:

reporters: ['progress', 'coverage'],

Further adjustments to the configuration file are usually not necessary – with one exception. The web server launched by Karma later listens on port 9876 by default. If this is already occupied, you will need to adjust the port setting. If you need or want to make further changes, you will find a detailed reference for all variables or settings on the Karma website [11].

Lift Off!

The only thing missing now is the code to be tested and the test cases. For your first steps with Karma, the simple addition program in Listing 1 is a good choice. The function there "accidentally" only adds the two numbers if they are equal. Listing 2 shows a suitable test for Jasmine.

01 function add(x, y) {
02   if (x === y) {
03     return x + y;
04   }
05 }

01 describe('Add-Function', function() {
02
03   it('adds two odd numbers', () => {
04     expect(add(2, 3)).toEqual(5);
05   });
06
07 });

Now, launch Karma with the command:

karma start karma.conf.js

The name of the configuration file can be left out if it is karma.conf.js, karma.conf.coffee, or karma.conf.ts. Karma first launches the web server, then launches the corresponding browsers, and finally executes all the tests.

Karma finally displays the results in the console, with the test in Figure 3 obviously failing. If Istanbul is also included, information about the test coverage is sent to the project directory's coverage subfolder (Figure 4).

To fix the reported error, the web developer only has to remove the query in Listing 1. The active Karma instance detects the changes to the corresponding file and immediately re-runs the test.

Conclusions

Karma only automates test execution with established test frameworks like Jasmine. Developers can continue to use their own frameworks and do not have to learn how to work in a new environment.

Simultaneous, automated testing in multiple browsers under real conditions drastically accelerates the entire testing process. Moreover, Karma is a proven tool that is stable and easy to extend with plugins. If you are looking to develop complex web applications, you might want to give the test tool a trial run.

For more information on how Karma works, see VojtÍch JÌna's Master's thesis, which is included with the Karma source code [12].