问题描述:

I have tests written in JavaScript and I use TravisCI for tests.

Setup

My package.json is something like:

"scripts": {

"test": "node testsRunner.js"

}

And my .travis.yml is:

language: node_js

node_js:

- '0.12.7'

'testsRunner.js' is:

var nodeunit = require('nodeunit');

var path = require('path');

nodeunit.reporters.default.run([

path.join(__dirname, 'suite1/test.js')

]);

And suite1/test.js finally is:

module.exports = {

setUp: function(callback) {

// Initialization code...

callback();

},

tearDown: function(callback) {

// Cleanup...

callback();

},

test1: function(test) {

test.expect(10); // This test expects 10 assertions to be run

// Doing stuff...

test.done();

},

test2: function(test) {

test.expect(10); // This test expects 20 assertions to be run

// Doing stuff...

test.done();

}

};

Log from Travis

Here is the build log and test execution from Travis:

Using worker: worker-linux-docker-19fc8ef0.prod.travis-ci.org:travis-linux-6

system_info

Build system information

Build language: node_js

Build image provisioning date and time

Thu Feb 5 15:09:33 UTC 2015

Operating System Details

Distributor ID: Ubuntu

Description: Ubuntu 12.04.5 LTS

Release: 12.04

Codename: precise

Linux Version

3.13.0-29-generic

Cookbooks Version

a68419e https://github.com/travis-ci/travis-cookbooks/tree/a68419e

GCC version

gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3

Copyright (C) 2011 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

LLVM version

clang version 3.4 (tags/RELEASE_34/final)

Target: x86_64-unknown-linux-gnu

Thread model: posix

Pre-installed Ruby versions

ruby-1.9.3-p551

Pre-installed Node.js versions

v0.10.36

Pre-installed Go versions

1.4.1

Redis version

redis-server 2.8.19

riak version

2.0.2

MongoDB version

MongoDB 2.4.12

CouchDB version

couchdb 1.6.1

Neo4j version

1.9.4

RabbitMQ Version

3.4.3

ElasticSearch version

1.4.0

Installed Sphinx versions

2.0.10

2.1.9

2.2.6

Default Sphinx version

2.2.6

Installed Firefox version

firefox 31.0esr

PhantomJS version

1.9.8

ant -version

Apache Ant(TM) version 1.8.2 compiled on December 3 2011

mvn -version

Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)

Maven home: /usr/local/maven

Java version: 1.7.0_76, vendor: Oracle Corporation

Java home: /usr/lib/jvm/java-7-oracle/jre

Default locale: en_US, platform encoding: ANSI_X3.4-1968

OS name: "linux", version: "3.13.0-29-generic", arch: "amd64", family: "unix"

git.checkout

0.61s$ git clone --depth=50 --branch=master git://github.com/myuser/myproj.git myuser/myproj

Cloning into 'myuser/myproj'...

remote: Counting objects: 1473, done.

remote: Compressing objects: 100% (1053/1053), done.

remote: Total 1473 (delta 300), reused 1401 (delta 250), pack-reused 0

Receiving objects: 100% (1473/1473), 8.93 MiB | 0 bytes/s, done.

Resolving deltas: 100% (300/300), done.

Checking connectivity... done.

$ cd myuser/myproj

$ git checkout -qf d603836a30ea1bc213f7b97682df507c91af8404

This job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.

If you require sudo, add 'sudo: required' to your .travis.yml

See http://docs.travis-ci.com/user/workers/container-based-infrastructure/ for details.

2.99s$ nvm install 0.12.7

######################################################################## 100.0%

Now using node v0.12.7

$ node --version

v0.12.7

$ npm --version

2.11.3

$ nvm --version

0.23.3

install

0.84s$ npm install

npm WARN package.json [email protected] No license field.

0.64s$ npm test

> [email protected] test /home/travis/build/myuser/myproj

> node testsRunner.js

test.js

✔ test1

✖ test2

TypeError: Cannot read property...

at extract (...)

at Object.at (...)

at Object.module.exports...

... <stacktrace continues>

FAILURES: 1/45 assertions failed (19ms)

The command "npm test" exited with 0.

Done. Your build exited with 0.

The problem

Problem is that test1 succeeds and test2 does not pass. Before test2 threw exceptions and this made Travis report my build as a failure.

However now i fixed the test, so test2 does not throw exceptions, but its asserts fail. However Travis is reporting that build as a pass, while it should report it as a failed build.

How to handle this?

Edit

I have changes my runner like this:

var nodeunit = require('nodeunit');

var path = require('path');

nodeunit.reporters.default.run([

path.join(__dirname, 'suite1/test.js')

]);

console.log('Done!'); // ADDED THIS LINE

Well, in the log I can see that Done! is displayed before everithing, after that I can see each single test log and stacktraces. Is it possible that these tests are executing async and this is causing the main process to exit with 0 status?

Fixed

Following Chris Beck's suggestion I changed the runner like this:

var nodeunit = require('nodeunit');

var path = require('path');

nodeunit.reporters.default.run([

path.join(__dirname, 'suite1/test.js')

], null function(data) {

// `data` has a value if errors occurred

if (data) { throw 'Error1'; }

});

网友答案:

I have used travis-ci for unit tests.

If you want the test failures to represent a build failure, the command-line utility that invokes the tests needs to report a nonzero error code result (as would be visible in the bash value $? after running the tests.)

If your js environment does not cause assert failures to do this, the most simple thing most likely is to write your own assert function which throws an uncaught exception, or, which sets a global variable to indicate failure, and before exist your test runner script checks the global variable and throws an uncaught exception if it is set.

(The latter has the advantage that you run all the tests even if some asserts fail, but the disadvantage that if the tests loop infinitely then your build will stall for a while even after an earlier assertion failure.)

网友答案:

There are usually plugins for unit test integration through JUnit xml. Is this default in travis ci? Are you reporting your tests usin Junit?

nodeunit has a built in junit reporter:

junit - Creates jUnit compatible XML reports, which can be used with continuous integration tools such as Hudson.

相关阅读:
Top