# jskit

jskit contains infrastructure and in particular a py.test plugin to enable running tests for JavaScript code inside browsers directly using py.test as the test driver. Running inside the browsers comes with some speed cost, on the other hand it means for example the code is tested against the real-word DOM implementations.

The approach also enables to write integration tests such that the JavaScript code is tested against server-side Python code mocked as necessary. Any server-side framework that can already be exposed through WSGI (or for which a subset of WSGI can be written to accommodate the jskit own needs) can play along.

jskit has also some support to run JavaScript tests from `unittest.py` based test suites.

jskit also contains code to help modularizing JavaScript code which can be used to describe and track dependencies dynamically during development and that can help resolving them statically when deploying/packaging.

jskit depends on simplejson on Python 2.5. It also uses MochiKit - of which it ships a version within itself for convenience - for its own working though in does not imposes its usage on tested code.

jskit was initially developed by Open End AB and is released under the MIT license.

# Basics of writing and running tests

For the simplest cases jskit allows to write tests simply as just JavaScript files. As an example one could have a `jstest_example_any.js` in a test directory in a project:

    Tests = {

    test_simple: function() {
        ais(21*2, 42)
    }

    }

`jstest_*.js` is the globbing pattern used by jskit to recognize JavaScript test files. Each of these need to contain a `Tests` object with test methods starting with `test_`. `ais` is one of the assertion functions predefined by jskit, it raises an exception causing the test to fail if the two arguments are not equal (in JavaScript `=` sense).

The suffix `any` separated by an underscore (`_`) in the test file name controls the kind of browsers the test should be run against, `any` is predefined by jskit to be

- Firefox and Safari on Mac OS X
- Internet Explorer and Firefox on Windows
- Firefox (started as `firefox`) otherwise (e.g. Linux)

later we'll see how to define other groups of browsers.

To run the test we need to use py.test with the jskit plugin activated, a `conftest.py` file at the top of the project with:

    pytest_plugins = ["jstests"]

should work (see also the py.test and plugin usage docs).

Running the test is just a matter of invoking `py.test` that has an installed jskit available on the Python path:

    $ ../v/bin/py.test test/
    inserting into sys.path: /u/pedronis/scratch/oejskit-play/v/lib/python2.5/site-packages
    ========================================= test session starts =========================================
    python: platform linux2 -- Python 2.5.2
    test object 1: /u/pedronis/scratch/oejskit-play/proj/test

    test/jstest_example_any.js .

    ====================================== 1 passed in 2.52 seconds =======================================

(concretely here I'm using a virtualenv with both the py.lib and the `oejskit` egg installed in it)

Tests from different `Tests` objects are run isolated in IFRAMEs which will also display test results enumerating both assertions and tests, not just tests. Each JavaScript `test_` method is run and its result reported by py.test as it would for Python tests. `test_` methods from a `Tests` object are run in alphabetical order.

# Finding JavaScript code

Tests of course need to be able to load and refer to tested code. Let's assume for example that JavaScript code lives under the `static/js` directory in the project, for example there could be a file `ex.js` there:

    function ok() {
        return "ok"
    }

jskit way to refer to tested code is based on its modularity functionality, though for the simplest cases this is involve just providing enough information such that parts of the URL space can be mapped back to parts of the filesystem where JavaScript lives.

Adding the following to the `conftest.py` of the project:

    class jstests_setup:
        staticDirs = {
           '/static': os.path.join(os.path.dirname(__file__), 'static')
        }
        jsRepos = ['/static']

tells jskit to serve files from the `static` directory (the value in the `staticDirs` mapping) for the `static` subtree of the URL space (the key).

The values in `jsRepos` list which prefixes of the URL space contain JavaScript, they play a bigger role when using the full modularity functionality, they are ncessary for the simple case too though.

With this configuration we can write a test `jstest_ok_any.js` for the `ok` function in `ex.js` that looks like this:

    OpenEnd.require("/static/js/ex.js")

    Tests = {
    test_ok: function() {
       var res = ok()
       ais(res, "ok")
    }
    }

With the information provided jskit can infer that `ex.js` is needed for the test and serve it.

Running all the tests now should give:

    $ ../v/bin/py.test
    inserting into sys.path: /u/pedronis/scratch/oejskit-play/v/lib/python2.5/site-packages
    ========================================= test session starts =========================================
    python: platform linux2 -- Python 2.5.2
    test object 1: /u/pedronis/scratch/oejskit-play/proj

    test/jstest_example_any.js .
    test/jstest_ok_any.js .

    ====================================== 2 passed in 5.06 seconds =======================================

# Assertion functions and helpers

jskit predefine a set of functions to make assertions in tests, JavaScript not having an assertion statement at all

- `aok(condition [, label])` which will succeed respectively fail depending on the JavaScript truth value of `condition`
- `ais(value, expected [, label])` which will succeed respectively fail depending on whether `value` and `expected` are equal in JavaScript `=` operator sense
- `aisDeeply(value, expected [, label])` which differently from `ais` will recursively compare containers (JavaScript non-primitive objects and arrays) succeeding when they have the same properties/indexes sets with matching values in `aisDeeply` sense, for primitive objects it behaves like `ais`

There's another predefined helper `insertTestNode` which will return a new `DIV` node at the end of test page styled with a border and labeled with the test name, the node can then be used to host DOM fragments to be manipulated by the test. If a test using this feature fails it should then be easy to identify the relevant DOM part and eyeball it or inspect with tools such as Firebug.

# Test failures

Given a failing test like (`proj/fail/jstest_fail_any.js`):

    Tests = {

    test_fail: function() {
        aisDeeply([1].concat([2, 3]), [1,[2,3]])
    }

    }

this is part of the kind of output to be expected at least when the test is run against Firefox which provides proper traceback information:

    ...
               raise JsFailed(name, outcome['diag'])
    E           JsFailed: test_fail: FAILED:     Structures begin differing at:
    E           got[1] = 2
    E           expected[1] = 2,3
    E
    E           stack: Error()@:0
    E           ("isDeeply in test_fail in ?","    Structures begin differing at:\r\ngot[1] = 2\r\nexpected[1] = 2,3\r\n")@http://localhost:49279/browser_testing/rt/testing-new.js:199
    E           (false,"isDeeply in test_fail in ?","    Structures begin differing at:\r\ngot[1] = 2\r\nexpected[1] = 2,3\r\n")@http://localhost:49279/browser_testing/rt/testing-new.js:217
    E           ([object Array],[object Array])@http://localhost:49279/browser_testing/rt/testing-new.js:427
    E           ()@http://localhost:49279/test/jstest_fail_any.js:6
    ...

Notice that the line numbers and involved JavaScript files are presented (`@http://localhost:49279/test/jstest_fail_any.js:6`).

# Browser kind specifications

On top of the predefined `any`, browser kind groupings against which to run tests can be defined that are meaningful to one's project. As we have seen these are used as suffixes for test files or as values for the later introduced `jstests_browser_kind` class attribute.

All is needed is to define a `jstest_browser_specs` mapping in the `conftest.py` file, with the kinds as keys and lists of browser names as values:

    jstests_browser_specs = {
        'supported': ['firefox', 'iexplore', 'safari'],
        'extrafeatures': ['safari']
    }

browser names themselves can also be used directly as kinds. Given the mapping `jstests_basic_supported.js`, `jstests_extras_extrafeatures.js` and in any case `jstests_details_firefox.js` or `jstests_details_iexplore.js` would be meaningful JavaScript test file names.

With the plugin activated a `py.test` command-line option `--jstests-browser-spec=JSTESTS_BROWSER_SPEC` can also be used to define browser kinds as in:

    --jstests-browser-spec any=firefox,safari --jstests-browser-spec extra=safari

As we see here `any` itself can be redefined this way or via the mapping.

# Integration testing

One central jskit feature is the ability to run JavaScript tests against server-side Python code setup through test code. For example one can add a `test_integration.py`:

    from oejskit.testing import jstests_suite
    import cgi

    class TestIntegration(object):
        jstests_browser_kind = 'any'

        @jstests_suite('test_integration.js')
        def integration_server_side(self):
            def server(environ, start_response):
                p = environ['PATH_INFO']
                start_response('200 OK', [('content-type', 'text/plain')])
                q = cgi.parse_qs(environ['QUERY_STRING'])
                if p == '/add':
                    a, b = int(q['a'][0]), int(q['b'][0])
                    return [str(a+b)]
                elif p == '/sub':
                    a, b = int(q['a'][0]), int(q['b'][0])
                    return [str(a-b)]
                return []

            return server

with a parallelly located `test_integration.js`:

    OpenEnd.require("/static/js/get.js")

    Tests = {

    test_add: function() {
        var res = http_get("/add?a=2&b=3")
        ais(parseInt(res), 5)
    },

    test_sub: function() {
        var res = http_get("/sub?a=6&b=2")
        ais(parseInt(res), 4)
    }

    }

This test code will result in the tests in test_integration.js` to be run while the testing framework is serving the WSGI application `server` at the root of the URL space. The code in `integration_server_side` will be run once before the JavaScript tests in the suite will be run, in a sense it is akin to a setup method but funcargs can be requested in its arguments.

The tests will be run for each browser identified by the kind put in `jstests_browser_kind`. Both `jstests_browser_kind` and the decorator `jstests_suite` are required to achieve this behavior, `jstests_browser_kind` marks that the tests in the class need to be executed with a browser available to control.

`get.js` here defines `http_get` a simple function wrapping XMLHttpRequest t do synchronous GET requests.

# Remote browsers

Contained in the `oejskit` package there is a module `browser.py` which can be used as standalone script. The script can be run as a server that allows starting browsers on a remote machine, this means that the platforms for the python/server-side test code and the platforms for target browsers don't need to coincide. The script can be simply be copied as standalone file and it just needs a plain python installed without further dependencies to run. The script uses HMAC and a secret token to secure its access though it is still recommend to run it behind a firewall.

Setup servers to start browsers on the the relevant machine:

    Z:\>c:\Python26\python.exe browser.py server 10010
    JSTESTS_REMOTE_BROWSERS_TOKEN=eca424b610245337c80f32e3a08c50c6
    serving browsers on 10010...

On the machine that will drive the tests set one environment variable with the secret token, and another that lists separated by spaces `<hostname>:<port>:<browsers-separated-by-commas>` for the browser servers:

    $ export JSTESTS_REMOTE_BROWSERS_TOKEN=eca424b610245337c80f32e3a08c50c6
    $ export JSTESTS_REMOTE_BROWSERS=bigboard:10010:iexplore
    $ ../v/bin/py.test --jstests-browser-spec any=iexplore,firefox
    inserting into sys.path: /u/pedronis/scratch/oejskit-play/v/lib/python2.5/site-packages
    ======================================================= test session starts ========================================================
    python: platform linux2 -- Python 2.5.2 -- /u/pedronis/scratch/oejskit-play/v/bin/python
    test object 1: /u/pedronis/scratch/oejskit-play/proj

    test/jstest_example_any.js:1: proj.test.jstest_example_any.js[=iexplore][test_simple] PASS
    test/jstest_example_any.js:1: proj.test.jstest_example_any.js[=firefox][test_simple] PASS
    test/jstest_ok_any.js:1: proj.test.jstest_ok_any.js[=iexplore][test_ok] PASS
    test/jstest_ok_any.js:1: proj.test.jstest_ok_any.js[=firefox][test_ok] PASS
    test/test_integration.py:7: TestIntegration[=iexplore].integration_server_side[test_add] PASS
    test/test_integration.py:7: TestIntegration[=iexplore].integration_server_side[test_sub] PASS
    test/test_integration.py:7: TestIntegration[=firefox].integration_server_side[test_add] PASS
    test/test_integration.py:7: TestIntegration[=firefox].integration_server_side[test_sub] PASS

    ==================================================== 8 passed in 28.39 seconds =====================================================

Firefox is not listed in the environment variable, so a local one is started or reused.

There are also commands to shutdown browsers, respectively servers, they will also consider the environment variables to find their targets:

    $ ../v/bin/python -m oejskit.browser cleanup iexplore

    $ ../v/bin/python -m oejskit.browser shutdown-servers

For convenience when setting up multiple machines there is the possibility to first generate a token and pass it in as an argument to the `server` command:

    Z:\>c:\Python26\python.exe browser.py gentoken
    eca424b610245337c80f32e3a08c50c6
    Z:\>c:\Python26\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6
    serving browsers on 10010...

It is possible to give user-defined name for greater flexibility to browsers or browser command lines, for example to run tests against both Internet Explorer 7 and 8 one could run `browser.py` on a Windows machine with the latter this way:

    Z:\>c:\Python26\python.exe browser.py gentoken
    eca424b610245337c80f32e3a08c50c6
    Z:\>c:\Python26\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 ie8=iexplore
    serving browsers on 10010...

and on other with the older IE7 with:

    Y:\>c:\Python25\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 ie7=iexplore
    serving browsers on 10010...

A series of definitions with `user-defined-name=command-line` can be specified after the main `browser.py server` parameters. `user-defined-names` can then be used as browser names and in browser specifications, like:

    --jstests-browser-spec any=firefox,ie7,ie8

If it makes sense a full command line can be specified:

    Y:\>c:\Python25\python.exe browser.py server 10010 eca424b610245337c80f32e3a08c50c6 "ff-testing=...\firefox.exe -P testing" ie7=iexplore

# Support for `unittest.py` based test suites

The `oejskit` package contains a `unittest_support` module whose content is a subclass of `unittest.TestSuite` which can be used to incorporate JavaScript tests with `unittest.py` tests. Typically one will make a per-project subclass of `unittest_support.JSTestSuite`, the parameters we previously described like `staticDirs` that go `jstests_setup` and the mapping `jstests_browser_specs` can be specified as attributes attached to such a subclass. An example usage would then be like:

    class ProjectJSTestSuite(unittest_support.JSTestSuite):
        jstests_browser_specs = {
            'supported': ['firefox', 'iexplore', 'safari']
        }

    def ok_root():
        def ok(environ, start_response):
            start_response('200 OK', [('content-type', 'text/plain')])
            return ['ok\n']
        return ok

    if __name__ == '__main__':
        runner = unittest.TextTestRunner(verbosity=1)
        all = unittest.TestSuite()
        example_suite = ProjectJSTestSuite('jstest_example_supported.js')
        integration_suite = ProjectJSTestSuite('test_integration_style.js',
                                               root=ok_root,
                                               browser_kind='supported')
        all.addTest(example_suite)
        all.addTest(integration_suite)
        runner.run(all)

the suites are instantiated with a JavaScript test file which will be located relative to the Python test file, optionally a `browser_kind` and a factory `root` for e.g. a WSGI application (see also Integration testing) can be specified to serve for the test. If a `browser_kind` is not specified it will be extracted out of the JavaScript test file name.