Code Coverage Reporting in JavaScript

Now that I am working on dom.js, I need to learn an entirely new environment and all the tools that go with it. JavaScript is also fundamentally different from other scripting languages because usually the environment it is executing in is the browser rather than the command line. However, dom.js is designed to be used in environments where a native DOM does not already exist, such as in Node.js or in SpiderMonkey. Since it's such a unique project many of the existing tools don't really apply.

I went looking for code coverage tools that we could use to determine how much of the dom.js code was being exercised by the test suites we have in place right now. Several coverage tools exist for JavaScript, as discussed on StackOverflow here: http://stackoverflow.com/questions/53249/are-there-any-good-javascript-code-coverage-tools

For example, the ffhrtimer project (http://hrtimer.mozdev.org/) is a nice firefox extension that provides high resolution timers and UI to display JavaScript code coverage, but it can't easily be integrated into the SpiderMonkey command line js and only runs on FireFox 3.0.

JSCoverage is interesting (http://siliconforks.com/jscoverage/) but it requires source-level translations on the javascript in order to record coverage information. It makes this easy by providing a web server that automatically translates javascript that it serves as well as a proxy that translates any javascript that passes through it, but this does not really fit into our model where we are testing from the command line.

js-test-driver (http://code.google.com/p/js-test-driver/wiki/CodeCoverage) looks good but it also is designed to work in a browser environment.

Finally, JSChiliCat (http://jschilicat.sourceforge.net/) is getting closer because it allows running of tests without a browser being involved, but it also requires Rhino for running the tests. dom.js needs the HEAD version of SpiderMonkey since it uses extensions to JavaScript which are not widely implemented, Proxies and WeakMaps.

So, I looked at the way ffhrtimer gathers coverage data and modified SpiderMonkey to have a --coverage switch which installs a simple hook that prints out the current javascript filename and line number for every line of execution.

JSTrapStatus
CoverageHook(JSContext *cx, JSScript *script,
jsbytecode *pc, jsval *rval, void *closure)
{
const char* filename = JS_GetScriptFilename(cx, script);
uintN lineno = JS_PCToLineNumber(cx, script, pc);

printf("CoverageHook %s %d\n", filename, lineno);

return JSTRAP_CONTINUE;
}

Here's how this hook is installed as a callback:

JS_SetInterrupt(cx->runtime, &CoverageHook, NULL);

Now, the question is how best to store the data for use by an analysis tool. ffhrtimer used an in-memory data structure to keep track of which lines in which files had been visited, but for simplicity I think I am going to use the code above, writing the files and line numbers to a file 'coverage.out' for post-processing with a python script.