There is no need to run the job on both since one always implies
the other. A same-repo PR by definition has every commit in it
trigger a push.
Triggering the job both ways is not only inefficient, but means
SauceLabs connection end up refused during the Nightwatch test
since it will find its free concurrency limit being reached from
the two parallel versions of the same job.
The Karma version seems to not suffer from this as it re-tries
and merely waits a little bit. We may encounter this issue more
in the future e.g. from unrelated commits being worked on, but
alas, this is an improvement either way.
Follows-up 55186c8. I had this check originally, but then
removed it after GitHub merged the upstream improvement.
It looks like some CI runners might not yet have it applied,
so keep it a little longer for now.
By default `npm test` will run concurrenty in Firefox and one of
Chromium/Chrome. Chrome for macOS and Win, and Chromium on Linux
unless CHROME_BIN is configured by the user's env variables, in
which case they're likely to prefer that and/or not have Chromium
installed.
Also provide `npm run test-unit-…` commands which run unit tests
in a single specific browser only (overrides the conf array).
Follows-up https://github.com/kiwix/kiwix-js/pull/672.
* workflows: Rename "CI" to "Release"
Rename this to make space for a separate "CI" workflow that will
run continuous integration and testing jobs for commits and PRs.
* workflows: add CI workflow for unit tests and support running locally
* Load qunit package from npm, this is the start of a larger transition.
ref at https://github.com/kiwix/kiwix-js/issues/554.
* Update QUnit from 2.3 to 2.13.
* Use Karma for the running of unit tests (instead of Nightwatch).
While it was possible to use a fake "UI" test to open the QUnit
web page with Nightwatch, this had numerous limitations:
- relies on fragile and unsupported DOM scraping to collect
test results, which breaks between framework versions.
ref https://github.com/kiwix/kiwix-js/pull/660.
- severely limits debugging information for failing tests.
- cannot easily be reproduced or debugged locally from the command-line
as the Nightwatch config was pinned to Sauce Labs, and creating
a local configuration is not easy because Nightwatch has a hard
requirement for installing and running a WebDriver server.
People usually do not have this installed and it's non-trivial
to set up and keep working in the long term, and across multiple
different software projects.
- cannot easily be run in a secure container separate from your
personal computer, thus putting personal data at risk.
- lacks wider integration and plugins to enrich unit testing,
such as test coverage reports.
Using Karma means:
- We can run 'npm test' locally during development and have it
automatically run the tests in headless Firefox and Chrome
and report back, all from the command-line.
- The same exact same stack is also used in CI with SauceLabs
for additional browser coverage (same as before).
- It has no external dependencies other than the plain web
browser itself. This means if you have a development container
(e.g. based on Docker) that has Node.js + Firefox + Chromium,
you can run the tests there without exposing anything from
your personal computer, besides the current directory.
<https://timotijhof.net/posts/2019/protect-yourself-from-npm/>
- In a future change, we can plug in karma-coverage to generate
a test coverage report, to submit to Codecov or Coveralls.
ref https://github.com/kiwix/kiwix-js/issues/528.
* I have pinned the version of 'http-server' and 'nightwatch'
in package.json so that these don't silently upgrade in a way
that may introduce security issues or drop compatibility for
the environment we currently support.
Fixes https://github.com/kiwix/kiwix-js/issues/653.
* Re-enable unit and UI tests in latest Edge
The tests were disabled after https://github.com/kiwix/kiwix-js/pull/499
due to an issue with the Edge version that was the default "edge" on
SauceLabs in May 2019 (not sure which version that was, the last pre-Chromium
Edge version was 44, which was passing, so perhaps SauceLabs defaulted
to a beta release, or used a much older version like 15-18?)
Now that Edge uses Chromium, try re-enabling the tests.
Fixes https://github.com/kiwix/kiwix-js/issues/502.
Follows-up 0c530101573cf95b, which removed this from www/index.html,
but it was still present in the test suite.
This is now loaded automatically by RequireJS.
