Jasmine had a major version bump and required a few minor changes in our
booting code. Most notably, using `pending` in a `describe` block is no
longer supported, so we can only return early there. On the positive
side, the unit tests now run in a random order by default, which
eliminates any dependencies between unit tests.
Note that upgrading to Webpack 4 is out of scope for this patch since
the bots cannot work well with the newly generated bundles (both
browsers on both bots do not react within 120 seconds). Webpack 4 is not
faster for us than Webpack 3, so for now there is no need to upgrade.
Please note that this build target, and the resulting `build/pdf.combined.js` file, is equivalent to setting the `PDFJS.disableWorker` option to `true` which is a performance footgun.
These were removed in PR 9170, since they were unused in the browsers that we'll support in PDF.js version `2.0`.
However looking at the output of Travis, where a subset of the unit-tests are run using Node.js, there's warnings about `btoa` being undefined. This doesn't appear to cause any errors, which probably explains why we didn't notice this before (despite PR 9201).
The `gulp-util` module is now deprecated and authors are asked to stop
using it (refer to https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5 for
more information).
PDF.js does not rely on it that much, fortunately, so it's relatively
easy for us to remove the dependency. This patch does that by making the
following changes:
- Require `gulp-zip` version 4.1.0 or higher since they already removed
their `gulp-util` dependency in that version.
- Replace `gulp-util.log` with the `fancylog` module as recommended in
the article above.
- Replace `gulp-util.File` with the `Vinyl` module as recommended in the
article above.
The only change I had to make for Vinyl is removing the `base` and `cwd`
lines since they may not be empty strings anymore. This way we fall back
to the defaults Vinyl provides, which for us doesn't matter since we
move the file afterwards anyway. Moreover, I used `vfs` for `vinyl-fs`
in the `Gulpfile` to avoid confusion with `vinyl` (which is also how the
documentation names the variable).
This is all we can do on our side; the other modules that still use
`gulp-util` must be updated upstream.
Nothing uses this option anymore, so setting it is a no-op now. We can
safely remove it.
Use `SKIP_BABEL` (instead of `PDFJS_NEXT`) now if you want to skip Babel
translation for a build.
To support this, the following changes have to be made as well:
- Ignore `package-lock.json` since NPM creates it, but we should not
have it in the repository.
- Switch from `babel-preset-es2015` to `babel-preset-env` to resolve
the deprecation warning in the test logs. The latter is more recent
and flexible, but should be the same functionality-wise.
- `transform` now needs to have the `utf-8` encoding option provided.
If not given, it will call the callback with a `Buffer` object,
which results in an unhandled promise rejection since what is
returned from the callback is a string, not a `Buffer`.
Rather than (basically) duplicating the `SimpleLinkService` in `test/driver.js`, with potential test failuires if you forget to update the test mock, it seems much nicer to just re-use the viewer component.
Note that `SimpleLinkService` is already bundled into the `build/components/pdf_viewer.js` file. Hence we only need to expose it similar to the other viewer components in that file, and make sure that the `gulp components` command runs as part of the test-setup.
By updating to uglify-es, rather than uglify-js, the minifier *itself* now supports ES6 code. This means that it's now possible to minify code built with `PDFJS_NEXT = true` set, i.e. with Babel transpilation disabled, which wasn't the case previously.
Note that uglify-es is based on the API of uglify-js v3, which differs from the one that we previously used.
Of particular importance is the fact that it's no longer possible to provide a path to a file for minification, but one must instead directly provide the source of the file.
For more information, please see https://github.com/mishoo/UglifyJS2/tree/harmony
Use the environment's zlib implementation if available to get
reasonably-sized SVG files when an XObject image is converted to PNG.
The generated PNG is not optimal because we do not use a PNG predictor.
Futher, when our SVG backend is run in a browser, the generated PNG
images will still be unnecessarily large (though the use of blob:-URLs
when available should reduce the impact on memory usage). If we want to
optimize PNG images in browsers too, we can either try to use a DEFLATE
library such as pako, or re-use our XObject image painting logic in
src/display/canvas.js. This potential improvement is not implemented by
this commit
Tested with:
- Node.js 8.1.3 (uses zlib)
- Node.js 0.11.12 (uses zlib)
- Node.js 0.10.48 (falls back to inferior existing implementation).
- Chrome 59.0.3071.86
- Firefox 54.0
Tests:
Unit test on Node.js:
```
$ gulp lib
$ JASMINE_CONFIG_PATH=test/unit/clitests.json node ./node_modules/.bin/jasmine --filter=SVG
```
Unit test in browser: Run `gulp server` and open
http://localhost:8888/test/unit/unit_test.html?spec=SVGGraphics
To verify that the patch works as desired,
```
$ node examples/node/pdf2svg.js test/pdfs/xobject-image.pdf
$ du -b svgdump/xobject-image-1.svg
# ^ Calculates the file size. Confirm that the size is small
# (784 instead of 80664 bytes).
```
`__pdfjsdev_webpack__` was used to skip evaluating part of an AST,
in order to not mangle some `require` symbols.
This commit removes `__pdfjsdev_webpack__`, and:
- Uses `__non_webpack_require__` when one wants the output to
contain `require` instead of `__webpack_require__`.
- Adds options to the webpack config to prevent "polyfills" for
some Node.js-specific APIs to be added.
- Use `// eslint-disable-next-line no-undef` instead of `/* globals ... */`
for variables that are not meant to be used globally.
http://eslint.org/docs/rules/comma-danglehttp://eslint.org/docs/rules/object-curly-spacing
*Please note:* This patch was created automatically, using the ESLint `--fix` command line option. In a couple of places this caused lines to become too long, and I've fixed those manually; please refer to the interdiff below for the only hand-edits in this patch.
```diff
diff --git a/gulpfile.js b/gulpfile.js
index d18b9c58..7d47fd8d 100644
--- a/gulpfile.js
+++ b/gulpfile.js
@@ -1247,7 +1247,8 @@ gulp.task('gh-pages-git', ['gh-pages-prepare', 'wintersmith'], function () {
var reason = process.env['PDFJS_UPDATE_REASON'];
safeSpawnSync('git', ['init'], { cwd: GH_PAGES_DIR, });
- safeSpawnSync('git', ['remote', 'add', 'origin', REPO], { cwd: GH_PAGES_DIR, });
+ safeSpawnSync('git', ['remote', 'add', 'origin', REPO],
+ { cwd: GH_PAGES_DIR, });
safeSpawnSync('git', ['add', '-A'], { cwd: GH_PAGES_DIR, });
safeSpawnSync('git', [
'commit', '-am', 'gh-pages site created via gulpfile.js script',
```
After PR 8459, the run-time of the various `gulp` test commands has regressed quite badly on Windows. For me, `gulp test` now takes approximately *twice* as long when run locally on Windows.
The problem seems to be the Babel transpilation step, which takes well over five minutes to run.[1]
For someone like me, who runs tests a lot locally, this slowdown is really hurting the overall development experience.
To get around this I tested setting `PDFJS_NEXT = true` in `gulpfile.js`, since the transpilation step isn't necessary when testing in a modern browser.
However, having to edit `gulpfile.js` every time that I need to run tests isn't very practical. Hence this patch, which adds an environment variable that allows you to disable the transpilation simply by using e.g. `PDFJS_NEXT=true gulp test`.
I hope that this can be considered an acceptable solution, such that I don't need to maintain this patch locally (or worse, edit `gulpfile.js` locally before testing).
---
[1] This can also be observed on the Windows bot, but it seems fine on Linux.
Also updates a `console.log` statement in the `gulp importl10n` command (since I forgot it in a previous patch, and it didn't seem necessary with a separate patch for it).
Added test for ReadableStream.
Adds ref-implementation license-header in streams-lib
and change gulp task to copy external/streams/ in build/
external/streams/ and build/dist/external/streams folder.
Adds README.md and LICENSE.md
Note that we have to use `fs.writeFileSync` since `.to()` is not
available anymore. Moreover, introduce `safeSpawnSync` to make sure that
we check the return codes of the spawned processes properly.
To run the regression tests, developers use `gulp browsertest` and the
bot uses `gulp bottest`. We're not passing the `noreftest` option
anywhere in the code (probably because the `bottest` command takes care
of this already), so we should remove this.
Please see http://eslint.org/docs/rules/object-shorthand.
Unfortunately, based on commit 9276d1dcd9, it seems that we still need to maintain compatibility with old Node.js versions, hence certain files/directories that are executed in Node.js are currently exempt from this rule.
Furthermore, since the files specific to the Chromium extension are not run through Babel, the `/extensions/chromium/` directory is also exempt from this rule.
With the exception of just one test-case, all the current `ui_utils` unit-tests can run successfully on Node.js (since most of them doesn't rely on the DOM).
To get this working, I had to first of all add a new `LIB` build flag such that `gulp lib` produces a `web/pdfjs.js` file that is able to load `pdf.js` successfully.
Second of all, since neither `document` nor `navigator` is available in Node.js, `web/ui_utils.js` was adjusted slightly to avoid errors.
pdfjschildbootstrap.js will always be run, but
pdfjschildbootstrap-enabled.js will only be run if PdfJs.enabled is
true. This will let us avoid some work in the child process in the
next patch.
This will need to be landed in the mozilla-central repository at the
same time as a change to nsBrowserGlue.js. See bug 1352218.
This patch is another step towards enabling Babel. Since we're moving
towards ES6 modules, we will not be using UMD headers anymore, so we can
remove the validation.
First of all, `compatibility.js` is already available in `lib/shared/compatibility.js`. Second of all, as can be seen in 9142301f35 (diff-9432ebaa58e10ab02874fcb86f689caa), the `lib/web/compatibility.js` file cannot work since the `require` statement isn't compatible with the output of `gulp lib`.
*Yet another thing that I unfortunately missed during review of PR 8023.*
Note that previously, in `make.js` this file was being preprocessed, however as far as I can tell that wasn't actually necessary. Hence this patch just copies the file to the proper output directory.
Fixes https://bugzilla.mozilla.org/show_bug.cgi?id=1338395#c8.
See http://eslint.org/docs/rules/brace-style.
Having the opening/closing braces on the same line can often make the code slightly more difficult to read, in particular for `if`/`else if` statements, compared to using new lines.
This patch also, for consistency with `mozilla-central`, enables the [`no-iterator`](http://eslint.org/docs/rules/no-iterator) rule. Note that this rule didn't require a single code change.
*Please note that most of the necessary code adjustments were made in PR 7890.*
ESLint has a number of advantageous properties, compared to JSHint. Among those are:
- The ability to find subtle bugs, thanks to more rules (e.g. PR 7881).
- Much more customizable in general, and many rules allow fine-tuned behaviour rather than the just the on/off rules in JSHint.
- Many more rules that can help developers avoid bugs, and a lot of rules that can be used to enforce a consistent coding style. The latter should be particularily useful for new contributors (and reduce the amount of stylistic review comments necessary).
- The ability to easily specify exactly what rules to use/not to use, as opposed to JSHint which has a default set. *Note:* in future JSHint version some of the rules we depend on will be removed, according to warnings in http://jshint.com/docs/options/, so we wouldn't be able to update without losing lint coverage.
- More easily disable one, or more, rules temporarily. In JSHint this requires using a numeric code, which isn't very user friendly, whereas in ESLint the rule name is simply used instead.
By default there's no rules enabled in ESLint, but there are some default rule sets available. However, to prevent linting failures if we update ESLint in the future, it seemed easier to just explicitly specify what rules we want.
Obviously this makes the ESLint config file somewhat bigger than the old JSHint config file, but given how rarely that one has been updated over the years I don't think that matters too much.
I've tried, to the best of my ability, to ensure that we enable the same rules for ESLint that we had for JSHint. Furthermore, I've also enabled a number of rules that seemed to make sense, both to catch possible errors *and* various style guide violations.
Despite the ESLint README claiming that it's slower that JSHint, https://github.com/eslint/eslint#how-does-eslint-performance-compare-to-jshint, locally this patch actually reduces the runtime for `gulp` lint (by approximately 20-25%).
A couple of stylistic rules that would have been nice to enable, but where our code currently differs to much to make it feasible:
- `comma-dangle`, controls trailing commas in Objects and Arrays (among others).
- `object-curly-spacing`, controls spacing inside of Objects.
- `spaced-comment`, used to enforce spaces after `//` and `/*. (This is made difficult by the fact that there's still some usage of the old preprocessor left.)
Rules that I indend to look into possibly enabling in follow-ups, if it seems to make sense: `no-else-return`, `no-lonely-if`, `brace-style` with the `allowSingleLine` parameter removed.
Useful links:
- http://eslint.org/docs/user-guide/configuring
- http://eslint.org/docs/rules/
This requires the `run-sequence` dependency because Gulp does not have a
way to run sequences of tasks inside a Gulp task. Gulp 4.0 will have
support for this, but until that is released this is the recommended way
to implement it.
Privacy policy: https://github.com/Rob--W/pdfjs-telemetry#privacy-policy
Unit tests (offline):
```
node test/chromium/test-telemetry.js
```
Server tests (requires that Nginx is installed):
```
git clone https://github.com/Rob--W/pdfjs-telemetry.git
cd pdfjs-telemetry/
python testserver.py TestHttp TestHttps
```
Integration test (extension + server):
- Build the extension
- Edit build/chromium/telemetry.js and remove the check for
chrome.runtime.id.
- Start Chrome (preferably a new profile):
chromium --user-data-dir=/tmp/pdftest --no-first-run
- Open chrome://net-internals#events
- Visit chrome://extensions and enable Developer mode.
- Load unpacked extension, select build/chromium.
- Go to the chrome://net-internals tab and filter on pdfjs.robwu.nl.
- Click on URL_REQUEST and verify that the server replied with 204.
- Reload the extension.
- Verify that chrome://net-internals did not contain a new log request.
