It is not karma itself that determines the order in which tests are performed. The component that primarily determines the order of the tests is a test runner that you decide to use with karma. If you use Mocha, then Mocha logic will be used. If you use Jasmine, then Jasmine logic will be used. If you use something else, then different logic will be used.
Mocha, for example, runs tests in the order that describe and it are called in the test files. If you have two test files a.js and b.js and a.js run first, Mocha will a.js tests in a.js . If b.js is executed b.js , then Mocha will first test in b.js Mocha is no longer sorting. (Mocha has a sort parameter, but it is only used when using the mocha command-line mocha to launch Mocha in Node. It does not apply to using Mocha in Karma because Karma launches Mocha in a browser where Mocha does not support sort .)
If you use a module loader (e.g. RequireJS or SystemJS) to dynamically load test files, this complicates the situation. If you execute require(["a", "b"]) and both modules are independent of each other, then the order in which they are loaded is undefined. a may be loaded first, or b may be loaded first, so the order in which tests are performed may be inconsistent from one run to the next. You can force the order through the configuration or by nesting the require calls. (For example, if you again use modules that are independent of each other, require(["a"], () => require("b")) ensures that tests in a will run before those in b .)
Another complication is that some test runners will interrupt the test run if they determine that your test suite is faulty. . This does not change the order of the tests, but can make it look like this is missing. For example, Mocha considers errors in hooks that are used to set and break test data (as opposed to errors in the test itself), to be errors in your test suite and to interrupt execution. If three of your tests depend on a failed installation code, Mocha will simply skip the tests. You will see one failure in three tests, not three failures. Here is an example test file with three tests and a failure in before . (The before key, aka “before everyone,” is for setting up test data in front of a group of tests.)
before(() => { throw new Error("oh no!"); }); it("one", () => {}); it("two", () => {}); it("three", () => {});
Here is my karma.conf.js :
module.exports = function(config) { config.set({ basePath: '', frameworks: ['mocha'], files: [ 'test.js' ], exclude: [ ], preprocessors: { }, reporters: ['progress'], port: 9876, colors: true, logLevel: config.LOG_INFO, autoWatch: false, browsers: ['Chrome'], singleRun: false, concurrency: Infinity }) }
Here's the output (I replaced the timestamp with <ts> ):
<ts>:INFO [karma]: Karma v1.7.0 server started at http://0.0.0.0:9876/ <ts>:INFO [launcher]: Launching browser Chrome with unlimited concurrency <ts>:INFO [launcher]: Starting browser Chrome <ts>:INFO [Chrome 58.0.3029 (Linux 0.0.0)]: Connected on socket qfNuMyp4q3SyUBOsAAAA with id 24880742 Chrome 58.0.3029 (Linux 0.0.0) "before all" hook FAILED Error: oh no! at Context.before (test.js:3:11) Chrome 58.0.3029 (Linux 0.0.0): Executed 1 of 3 (1 FAILED) ERROR (0.013 secs / 0.001 secs)
He says "Completed 1 of 3" because he tried to run the first test, but this attempt failed in the hook before , so Mocha did not try to run any of the following tests.