Compared to the, previously removed, `sandbox`/`watch-sandbox` gulp-tasks, these ones should work even when run against an non-existent/empty `build`-folder.
Also, to ensure that the development viewer actually works out-of-the-box, `gulp server` will now also include `gulp watch-dev-sandbox` to remove the need to *manually* invoke the build-tasks.
Finally, this patch also removes the `web/devcom.js` file since it shouldn't actually be needed, assuming that the "sandbox"-loading code in the `web/genericcom.js` file is actually *correctly* implemented.
The way that the `pdf.sandbox.js` building was implemented feels all kinds of inconsistent/wrong, and it "sticks out" quite a bit when compared to the rest of the `gulpfile.js`. This patch thus attempts to improve the current situation slightly, to hopefully make future maintenance easier.
One thing that strikes you, pretty immediately, when looking at PR 12604 is that the two new `gulp`-tasks added (i.e. `sandbox` and `watch-sandbox`) don't even work!?
The reason for this is that they implicitly dependent upon the result of the `buildnumber`-task, which isn't listed as a dependency. (Try running `gulp clean` *first*, and invoking any of the new `gulp`-tasks will inevitably fail.)
Furthermore, there's another (potentially big) problem with the implementation of e.g. the `gulp sandbox` task, since it doesn't actually wait for all building to complete before the task is considered as "done". This has the potential to cause all sorts of subtle bugs elsewhere, and the fact that things even "work" as-is can probably be attributed mostly to luck.
Unfortunately there's no *perfect* way to improve things here, since the `pdf.sandbox.js` file depends on including the `pdf.scripting.js` file as a string, however I firmly believe that improvements are still possible here.
To that end, this patch updates all relevant build-targets to create a *temporary* `pdf.scripting.js` file as part of the setup in the `gulp`-tasks, and then reads that file during the `pdf.sandbox.js` building.
This at least allows us to bring all of this "sandbox"-build code much more in-line with the existing build-system.
Given the somewhat "specialized" nature of the `pdf.sandbox.js` building, it ought to be possible to re-factor how some of the options are handled.
Note in particular that the `gulp-strip-comments` dependency seems somewhat unncessary, since the *main* source of comments are just the default license header. Hence I seems much more reasonable to simply not include that to begin with, rather than removing it after the fact (the few remaining Webpack-related should be few/small enough to not really matter much in practice).
This way we're able to further reduce the special-casing related to the `pdf.sandbox.js`-building, which will make future changes/maintenance easier by bringing this code more in-line with existing patterns in `gulpfile.js`.
(If we really want to reduce the filesize, we might want to consider always minifying the `GENERIC`-build of the `pdf.sandbox.js` file.)
There's no good reason, as far as I can tell, to use search-and-replace to include the *stringified* `pdf.scripting.js` file in the built `pdf.sandbox.js` file. Instead we could, and even should, utilize the existing `PDFJSDev.eval(...)`-functionality, which is not only simpler but will also be more efficient as well (no need for a regular expression).
The current location feels somewhat strange, and also inconsistent with the existing way that bundling is done.
Finally, add the version/build numbers at the top of the *built* `pdf.sandbox.js` files, since all other built files include that information given that it's often helpful to be able to easily determine the *exact* version.
* quickjs-eval.js has been generated using https://github.com/mozilla/pdf.js.quickjs/
* lazy load of sandbox code
* Rewrite tests to use the sandbox
* Add a task `watch-sandbox` which update bundle pdf.sandbox.js on change in the sandbox code
- Add support for logical assignment operators, i.e. `&&=`, `||=`, and `??=`, with a Babel-plugin. Given that these required incrementing the ECMAScript version in the ESLint and Acorn configurations, and that platform/browser support is still fairly limited, always transpiling them seems appropriate for now.
- Cache the `hasJSActions` promise in the API, similar to the existing `getAnnotations` caching. With this implemented, the lookup should now be cheap enough that it can be called unconditionally in the viewer.
- Slightly improve cleanup of resources when destroying the `WorkerTransport`.
- Remove the `annotationStorage`-property from the `PDFPageView` constructor, since it's not necessary and also brings it more inline with the `BaseViewer`.
- Update the `BaseViewer.createAnnotationLayerBuilder` method to actaually agree with the `IPDFAnnotationLayerFactory` interface.[1]
- Slightly tweak a couple of JSDoc comments.
---
[1] We probably ought to re-factor both the `IPDFTextLayerFactory` and `IPDFAnnotationLayerFactory` interfaces to take parameter objects instead, since especially the `IPDFAnnotationLayerFactory` one is becoming quite unwieldy. Given that that would likely be a breaking change for any custom viewer-components implementation, this probably requires careful deprecation.
Note that a number of these cases are covered by existing unit-tests, and a few others only matter for the development/build scripts.
Furthermore, I've also tried to the best of my ability to test each case *manually* to hopefully further reduce the likelihood of this patch introducing any bugs.
Please find additional details about the ESLint rule at https://eslint.org/docs/rules/no-useless-escape
This brings the new `pdf.scripting.js` bundling more in-line with the pre-existing handling for the `pdf.js`/`pdf.worker.js` files:
- Add a new `src/pdf.scripting.js` file as the entry-point for the build scripts.
- Add the version/build numbers at the top of the *built* `pdf.scripting.js` files, since all other built files include that information given that it's often helpful to be able to easily determine the *exact* version.
- Tweak the `createScriptingBundle` in the gulp-file, since it looks like a little bit too much copy-and-paste in the variable names.
The only noticeable changes are that the built files are now *slightly* smaller, and that Webpack now supports optional chaining and nullish coalescing without the need for Babel plugins.
This dependency hasn't been updated in two years and the only place that
uses it is the `externaltest` target in the Gulpfile. We can simply
replace `fancy-log` usage there with `console.log` like we do in all
other places in the Gulpfile because we're not interested in the
timestamps here. Gulp already prints timestamps and these tests finish
within a second anyway.
Note that it remains in `package-lock.json` because other Gulp-related
packages have it as a dependency, but at least we're no longer depending
on it directly anymore now.
For now we need to use a Babel-plugin, since part of our build system doesn't support this fully (e.g. Babel-loader, Webpack 4.x, and SystemJS).
While the `?.` operator will thus always be transpiled by Babel, even in modern builds, simply supporting it for development purposes seems like a step in the right direction.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining
This is *similar* to the existing linting for JavaScript files, but covers CSS files instead.
While there's a lot of rules that could potentially be used, the main advantage of using Stylelint is that it has Prettier integration which means that we can automatically enforce a *consistent* style for our CSS files as well.
As a proof of concept, this patch is purposely limited to:
- Adding a simple rule, here `block-no-empty` is chosen; see https://stylelint.io/user-guide/rules/block-no-empty
- Adding Prettier integration, to unify the style of our CSS files.
Please find additional information at https://stylelint.io/
Note that this will increase the run-time of `gulp dist` and `gulp dist-install`, but that's unavoidable given that there's now additional building happening.
While this will obviously increase the size of the output of `gulp minified`/`gulp minified-es5` *slightly*, the resulting files are still a lot smaller than the non-minified builds.
See https://github.com/terser/terser#minify-options for information about various Terser options.
- Fix the `gulp types` task to run on Windows. Currently this fails, and the solution was to "borrow" the same formatting as used in the `gulp jsdoc` task.
- Place the TypeScript definitions in their own `types` directory, when building `pdfjs-dist`. These should *not* be cluttering the main `build` directory, especially since the generated TypeScript definitions consists of *multiple folders*. (Only if the TypeScript definitions would be concatenated into *a single file*, would placing them directly in `pdfjs-dist/build` be acceptable.)
This commit:
- moves the preparation work to a new `typestest-pre` target similar to
how the other targets work;
- moves the `TYPESTEST_DIR` definition to the top of the file like we
did for all other directory variables;
- renames the `TYPES_BUILD_DIR` variable to `TYPES_DIR` since it's
shorter and the naming scheme then corresponds to the other directory
variables;
- switches to `const`/template strings in the types targets where needed;
- converts the `if (err !== null)` check to `if (err)` similar to other
targets.
This PR adds typescript definitions from the JSDoc already present.
It adds a new gulp-target 'types' that calls 'tsc', the typescript
compiler, to create the definitions.
To use the definitions, users can simply do the following:
```
import {getDocument, GlobalWorkerOptions} from "pdfjs-dist";
import pdfjsWorker from "pdfjs-dist/build/pdf.worker.entry";
GlobalWorkerOptions.workerSrc = pdfjsWorker;
const pdf = await getDocument("file:///some.pdf").promise;
```
Co-authored-by: @oBusk
Co-authored-by: @tamuratak
This moves, and slightly simplifies, code that's currently residing in the unit-test utils into the actual library, such that it's bundled with `GENERIC`-builds and used in e.g. the API-code.
As an added bonus, this also brings out-of-the-box support for CMaps in e.g. the Node.js examples.
For now we need to use a Babel-plugin, since Webpack 4.x doesn't seem to support it yet. (Most likely we'll have to update to Webpack 5, once that becomes available, in order for this to be directly supported. This is thus also blocked on removing the `webpack-stream` package.)
While the `??` operator will thus always be transpiled by Babel, even in modern builds, simply supporting it for development purposes seems like a step in the right direction.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_operator
With the changes made in the previous patch, the `web/app_options.js` file no longer depends on anything *except* files residing in the `web/` folder. Hence the `gulp default_preferences` task can now be further simplified and thus becomes even faster than before; see also PR 11724.
By re-factoring the existing gulp tasks, most of the code can be re-used for both the existing `gulp minified` as well as the new `gulp minified-es5` task.
All of the other *similar* helper functions only target one file per function, and there's no particular reason for this one to be different.
This patch will simplify future changes, e.g. experimenting with using `gulp watch` instead of SystemJS for the development viewer.
Not only is there no code depending on it now, the actual task itself doesn't even make sense as-is. Note that it uses the default `DEFINES` configuration *unaltered*, which is neither useful nor correct since the resulting build thus won't make sense without an actual built target set.
To save time or resources during development it can be useful to run
tests only in Firefox. Previously this could be done by editing the
browser manifest file, but since that file is no longer used for
Puppeteer, this command line flag replaces it. For example, executing
`gulp unittest --noChrome` will only run the unit tests in Firefox.
This commit replaces our own infrastructure for handling browsers during
tests with Puppeteer. Using our own infrastructure for this had a few
downsides:
- It has proven to not always be reliable, especially when closing the
browser, causing failures on the bots because browsers were still
running even though they should have been stopped. Puppeteer should do
a better job with this because it uses the browser's test built-in
instrumentation tools for this (the devtools protocol) which our code
didn't. This also means that we don't have to pass
parameters/preferences to tweak browser behavior anymore.
- It requires the browsers under test to be installed on the system,
whereas Puppeteer downloads the browsers before the test. This means
that setup is much easier (no more manual installations and browser
manifest files) as well as testing with different browser versions
(since they can be provisioned on demand). Moreover, this ensures that
contributors always run the tests in both Firefox and Chrome,
regardless of which browsers they have installed locally.
- It's all code we have to maintain, so Puppeteer abstracts away how the
browsers start/stop for us so we don't have to keep that code.
By default, Puppeteer only installs one browser during installation,
hence the need for a post-install script to install the second browser.
This requires `cross-env` to make passing the environment variable work
on both Linux and Windows.
*Please note:* These changes were done automatically, using the `gulp lint --fix` command.
This rule is already enabled in mozilla-central, see https://searchfox.org/mozilla-central/rev/567b68b8ff4b6d607ba34a6f1926873d21a7b4d7/tools/lint/eslint/eslint-plugin-mozilla/lib/configs/recommended.js#103-104
The main advantage, besides improved consistency, of this rule is that it reduces the size of the code (by 3 bytes for each case). In the PDF.js code-base there's close to 8000 instances being fixed by the `dot-notation` ESLint rule, which end up reducing the size of even the *built* files significantly; the total size of the `gulp mozcentral` build target changes from `3 247 456` to `3 224 278` bytes, which is a *reduction* of `23 178` bytes (or ~0.7%) for a completely mechanical change.
A large number of these changes affect the (large) lookup tables used on the worker-thread, but given that they are still initialized lazily I don't *think* that the new formatting this patch introduces should undo any of the improvements from PR 6915.
Please find additional details about the ESLint rule at https://eslint.org/docs/rules/dot-notation
Please note that these changes were done automatically, using `gulp lint --fix`.
Given that the major version number was increased, there's a fair number of (primarily whitespace) changes; please see https://prettier.io/blog/2020/03/21/2.0.0.html
In order to reduce the size of these changes somewhat, this patch maintains the old "arrowParens" style for now (once mozilla-central updates Prettier we can simply choose the same formatting, assuming it will differ here).
I completely overlooked the fact that we had *one* occurrence of an asynchronous `mkdirp` call in the gulpfile, which thus breaks since the package now uses Promises rather than a callback function; sorry about that!
With two kind of builds now being produced, with/without translation/polyfills, it's unfortunately somewhat easy for users to accidentally pick the wrong one.
In the case where a user would attempt to use a modern build of PDF.js in an older browser, such as e.g. IE11, the failure would be immediate when the code is loaded (given the use of unsupported ECMAScript features).
However in some browsers/environments, in particular Node.js, a modern PDF.js build may load correctly and thus *appear* to function, only to fail for e.g. certain API calls. To hopefully lessen the support burden, and to try and improve things overall, this patch adds checks to ensure that a modern build of PDF.js cannot be used in browsers/environments which lack native support for critical functionality (such as e.g. `ReadableStream`). Hence we'll fail early, with an error message telling users to pick an ES5-compatible build instead.
To ensure that we actually test things better especially w.r.t. usage of the PDF.js library in Node.js environments, the `gulp npm-test` task as used by Node.js/Travis was changed (back) to test an ES5-compatible build.
(Since the bots still test the code as-is, without transpilation/polyfills, this shouldn't really be a problem as far as I can tell.)
As part of these changes there's now both `gulp lib` and `gulp lib-es5` build targets, similar to e.g. the generic builds, which thanks to some re-factoring only required adding a small amount of code.
*Please note:* While it's probably too early to tell if this will be a widespread issue, it's possible that this is the sort of patch that *may* warrant being `git cherry-pick`ed onto the current beta version (v2.4.456).
This rule is *not* currently enabled in mozilla-central, but it appears commented out[1] in the ESLint definition file; see https://searchfox.org/mozilla-central/rev/c80fa7258c935223fe319c5345b58eae85d4c6ae/tools/lint/eslint/eslint-plugin-mozilla/lib/configs/recommended.js#238-239
Unfortunately this rule is, for fairly obvious reasons, impossible to `--fix` automatically (even partially) and each case thus required careful manual analysis.
Hence this ESLint rule is, by some margin, probably the most difficult one that we've enabled thus far. However, using this rule does seem like a good idea in general since allowing variable shadowing could lead to subtle (and difficult to find) bugs or at the very least confusing code.
Please find additional details about the ESLint rule at https://eslint.org/docs/rules/no-shadow
---
[1] Most likely, a very large number of lint errors have prevented this rule from being enabled thus far.
*I've had this patch locally for awhile, but have apparently missed to upstream it.*
This simplifies enabling of new ESLint rules, since most of them support automatic fixing of errors, without having to edit `gulpfile.js` or manually invoke ESLint directly.
With the way that the `default_preferences.json` file is now generated at build time, the `gulp lint` task is now noticeably slower than before. This slowdown has been, and still is, somewhat annoying during the deployment of new ESLint rules.
Hence this patch, which moves the `chromium/preferences_schema.json` validation from `gulp lint` and into a new `gulp lint-chromium` task instead. *Obviously* this new task is run as part of the `gulp npm-test` task, and thus through `npm test` on Node.js/Travis, such that it's still being tested as before.
Having thought *briefly* about using `css-vars-ponyfill`, I'm no longer convinced that it'd be a good idea. The reason is that if we actually want to properly support CSS variables, then that functionality should be available in *all* of our CSS files.
Note in particular the `pdf_viewer.css` file that's built as part of the `COMPONENTS` target, in which case I really cannot see how a rewrite-at-the-client solution would ever be guaranteed to always work correctly and without accidentally touching other CSS in the surrounding application.
All-in-all, simply re-writing the CSS variables at build-time seems much easier and is thus the approach taken in this patch; courtesy of https://github.com/MadLittleMods/postcss-css-variables
By using its `preserve` option, the built files will thus include *both* a fallback and a modern `var(...)` format[1]. As a proof-of-concept this patch removes a couple of manually added fallback values, and converts an additional sidebar related property to use a CSS variable.
---
[1] Comparing the `master` branch with this patch, when using `gulp generic`, produces the following diff for the built `web/viewer.css` file:
```diff
@@ -408,6 +408,7 @@
:root {
--sidebar-width: 200px;
+ --sidebar-transition-duration: 200ms;
}
* {
@@ -550,27 +551,28 @@
position: absolute;
top: 32px;
bottom: 0;
- width: 200px; /* Here, and elsewhere below, keep the constant value for compatibility
- with older browsers that lack support for CSS variables. */
+ width: 200px;
width: var(--sidebar-width);
visibility: hidden;
z-index: 100;
border-top: 1px solid rgba(51, 51, 51, 1);
-webkit-transition-duration: 200ms;
transition-duration: 200ms;
+ -webkit-transition-duration: var(--sidebar-transition-duration);
+ transition-duration: var(--sidebar-transition-duration);
-webkit-transition-timing-function: ease;
transition-timing-function: ease;
}
html[dir='ltr'] #sidebarContainer {
-webkit-transition-property: left;
transition-property: left;
- left: -200px;
+ left: calc(-1 * 200px);
left: calc(-1 * var(--sidebar-width));
}
html[dir='rtl'] #sidebarContainer {
-webkit-transition-property: right;
transition-property: right;
- right: -200px;
+ right: calc(-1 * 200px);
right: calc(-1 * var(--sidebar-width));
}
@@ -640,6 +642,8 @@
#viewerContainer:not(.pdfPresentationMode) {
-webkit-transition-duration: 200ms;
transition-duration: 200ms;
+ -webkit-transition-duration: var(--sidebar-transition-duration);
+ transition-duration: var(--sidebar-transition-duration);
-webkit-transition-timing-function: ease;
transition-timing-function: ease;
}
```
After PR 9566, which removed all of the old Firefox extension code, the `FIREFOX` build flag is no longer used for anything.
It thus seems to me that it should be removed, for a couple of reasons:
- It's simply dead code now, which only serves to add confusion when looking at the `PDFJSDev` calls.
- It used to be that `MOZCENTRAL` and `FIREFOX` was *almost* always used together. However, ever since PR 9566 there's obviously been no effort put into keeping the `FIREFOX` build flags up to date.
- In the event that a new, Webextension based, Firefox addon is created in the future you'd still need to audit all `MOZCENTRAL` (and possibly `CHROME`) build flags to see what'd make sense for the addon.
Currently *all* users of `pdfjs-dist` are forced to install the `webpack` and `worker-loader` packages, despite the fact that they are *only* relevant if the `webpack.js` file is being used (with a custom Webpack build).
This really doesn't seem great, especially since those packages are the only remaining dependencies in the `pdfjs-dist` library, and it thus seem more reasonable overall that Webpack users handle those dependencies themselves.
To prevent unnecessarily cryptic runtime failures, when people update to newer `pdfjs-dist` versions, the `webpack.js` file was updated to explicitly check for the existence of the `worker-loader` package and error otherwise.
Furthermore, note that `webpack` was only listed as a dependency because of the `worker-loader` package itself (see issue 9248).
Obviously these changes may not be seen as great by Webpack users who rely on `pdfjs-dist`, since it forces them to handle the dependencies themselves, however it should improve things considerably for "general" users of `pdfjs-dist` by not burdening them with unnecessary dependencies.
These sort of changes are also in line with other recent changes, see PR 11418, which removed built-in fake worker loader code for specific JS builders/bundlers/frameworks. This work was prompted not only by a desire to simplify/clean-up old code, but also to lessen future support burden since the PDF.js contributors cannot be assumed to be experts in various JS bundlers.
This patch makes the follow changes:
- Remove no longer necessary inline `// eslint-disable-...` comments.
- Fix `// eslint-disable-...` comments that Prettier moved down, thus causing new linting errors.
- Concatenate strings which now fit on just one line.
- Fix comments that are now too long.
- Finally, and most importantly, adjust comments that Prettier moved down, since the new positions often is confusing or outright wrong.
Note that Prettier, purposely, has only limited [configuration options](https://prettier.io/docs/en/options.html). The configuration file is based on [the one in `mozilla central`](https://searchfox.org/mozilla-central/source/.prettierrc) with just a few additions (to avoid future breakage if the defaults ever changes).
Prettier is being used for a couple of reasons:
- To be consistent with `mozilla-central`, where Prettier is already in use across the tree.
- To ensure a *consistent* coding style everywhere, which is automatically enforced during linting (since Prettier is used as an ESLint plugin). This thus ends "all" formatting disussions once and for all, removing the need for review comments on most stylistic matters.
Many ESLint options are now redundant, and I've tried my best to remove all the now unnecessary options (but I may have missed some).
Note also that since Prettier considers the `printWidth` option as a guide, rather than a hard rule, this patch resorts to a small hack in the ESLint config to ensure that *comments* won't become too long.
*Please note:* This patch is generated automatically, by appending the `--fix` argument to the ESLint call used in the `gulp lint` task. It will thus require some additional clean-up, which will be done in a *separate* commit.
(On a more personal note, I'll readily admit that some of the changes Prettier makes are *extremely* ugly. However, in the name of consistency we'll probably have to live with that.)
For performance reasons, and to avoid hanging the browser UI, the PDF.js library should *always* be used with web workers enabled.
At this point in time all of the supported browsers should have proper worker support, and Node.js is thus the only environment where workers aren't supported. Hence it no longer seems relevant/necessary to provide, by default, fake worker loaders for various JS builders/bundlers/frameworks in the PDF.js code itself.[1]
In order to simplify things, the fake worker loader code is thus simplified to now *only* support Node.js usage respectively "normal" browser usage out-of-the-box.[2]
*Please note:* The officially intended way of using the PDF.js library is with workers enabled, which can be done by setting `GlobalWorkerOptions.workerSrc`, `GlobalWorkerOptions.workerPort`, or manually providing a `PDFWorker` instance when calling `getDocument`.
---
[1] Note that it's still possible to *manually* disable workers, simply my manually loading the built `pdf.worker.js` file into the (current) global scope, however this's mostly intended for testing/debugging purposes.
[2] Unfortunately some bundlers such as Webpack, when used with third-party deployments of the PDF.js library, will start to print `Critical dependency: ...` warnings when run against the built `pdf.js` file from this patch. The reason is that despite the `require` calls being protected by *runtime* `isNodeJS` checks, it's not possible to simply tell Webpack to just ignore the `require`; please see [Webpack issue 8826](https://github.com/webpack/webpack) and libraries such as [require-fool-webpack](https://github.com/sindresorhus/require-fool-webpack).
*Please note:* This patch does *not* prevent the 'default_preferences' task from running more than once during `npm test`, but it does ensure that the tasks won't run *concurrently* by running the relevant tests in *series*.
While it would obviously still make sense to re-factor the gulpfile to account for changes in `gulp` version 4, by at least tweaking the `npm test` command the intermittent failures on Travis should at least go away.
Unfortunately I forgot to test `gulp mozcentraldiff` with PR 11256, which is really bad since it will cause the en-US `chrome.properties` l10n file to be deleted at mozilla-central; sorry about breaking this!
In order to address this we'll have to re-add (only) the en-US `chrome.properties` l10n file, which is simple enough. Furthermore, since we're not doing any sort of build-specific parsing of the l10n files, we can just copy the en-US files as-is rather than having to run `gulp locale` during `gulp mozcentral`.
With the removal of the (standalone) Firefox building code in PR 9566 (a year and a half ago), these files are now completely unused in the GitHub repository[1].
Hence it doesn't really seem necessary to keep fetching them with `gulp importl10n`, and the existing files in the `l10n` folder can also be removed (thanks to version control, they're easy enough to restore should the need ever arise).
The patch also allows an additional simplification, for the `gulp locale` and `gulp mozcentral` commands, since it's now possible to stop writing `l10n` files to the `extensions/firefox/` folder and instead just copy them similar to other build targets.
---
[1] They're obviously still used in `mozilla-central`, for fallback messages displayed through `PdfStreamConverter.jsm`, but that doesn't make it necessary to keep them *here* as far as I'm concerned.
With the removal of the (standalone) Firefox building code in PR 9566 (a year and a half ago), these files are now completely unused.
Hence it doesn't really make sense to keep building them as part of `gulp locale`, and the existing files in the `l10n` folder can also be removed (thanks to version control, they're easy enough to restore should the need ever arise).
The following is printed with every build, which gets kind of annoying when looking at the logs:
```
Replace Autoprefixer browsers option to Browserslist config.
Use browserslist key in package.json or .browserslistrc file.
Using browsers option cause some error. Browserslist config
can be used for Babel, Autoprefixer, postcss-normalize and other tools.
If you really need to use option, rename it to overrideBrowserslist.
Learn more at:
https://github.com/browserslist/browserslist#readmehttps://twitter.com/browserslist
```
Given how we're using Autoprefixer, with a number of build-specific configs, simply changing the option name seems like the easiest solution here.
(I'm also adding a couple of newlines at the `autoprefixer` call-sites, to aid readability.)
Compared to the recently replaced `URL` polyfill, the new `ReadableStream` polyfill isn't being exported globally for two reasons:
- We're currently checking for the existence of a global `ReadableStream` implementation when determining if the Fetch API will be used; please see `isFetchSupported` in the src/display/display_utils.js file.
- Given that it's much newer functionality (compared to `URL`) and that not all browsers may implement all parts of the specification yet, not exposing the `ReadableStream` globally seems safer for now.
Note how at https://mozilla.github.io/pdf.js/api/ it's being described as API docs, however `src/core/annotation.js` is not part of the public API.
Furthermore, given that the code residing in the `src/core/` folder is run in a worker-thread, it's not even accessible on the main-thread (since `postMessage` is being used to transfer the data).
Hence the different API methods simply returns a "proxy" to the underlying data, but not actually the same objects and data structures as in the worker-thread itself; thus it doesn't make a whole lot of sense to expose this in API docs as far as I'm concerned.
Finally, the patch fixes a small JSDoc related typo in `src/display/api.js` when referring to the `TextStyle` typedef.
The generated `default_preferences.json` file is necessary when initializing the Firefox preferences, which only supports certain types, hence this patch adds additional validation to help prevent run-time errors in Firefox.
Given that these changes add a code-path to `AppOptions.getAll` which could throw, the `OptionKind.PREFERENCE` branch is also modified to require *exact* matching to prevent (future) errors in the viewer.
Finally the conditionally defined `defaultOptions` will no longer (potentially) be considered during the `gulp default_preferences` task, to make it more difficult for them to be accidentally included.
In order to, quickly, unblock future Webpack updates this patch implements a *temporary* work-around for the regression; please refer to issue 10177 for additional details.
Please note that this patch doesn't require *any* changes to the source code, but only (small) build-time changes to `gulpfile.js` and as such will be easy to revert.
Currently any editing of the preferences require updates in *three* separate files, which isn't a great developer experience to say the least.
This has annoyed me sufficiently to write this patch, which moves the definition of all preferences into `AppOptions` and adds a new `gulp` task to generate the `default_preferences.json` file for the builds where it's needed.
Based on the discussion in https://bugzilla.mozilla.org/show_bug.cgi?id=1521413, this patch simply removes the `ReadableStream` polyfill completely from MOZCENTRAL builds.
With this patch, the size of the `gulp mozcentral` build target is thus further reduced (building on PR 10470):
| | `build/mozcentral`
|-------|-------------------
|master | 3 339 666
|patch | 3 209 572
With https://bugzilla.mozilla.org/show_bug.cgi?id=1505122 landing in Firefox 65, the native `ReadableStream` implementation is now enabled by default in Firefox.
Obviously it would be nice to simply stop bundling the polyfill in MOZCENTRAL builds altogether, however given that it's still possible to disable[1] `ReadableStream` this is probably not a good idea just yet.
Nonetheless, now that native support is available, it seems unnecessary (and wasteful) to keep bundling the polyfill twice[2] in MOZCENTRAL builds. Hence this patch, which contains a suggest approach for packing the polyfill in a *separate* file which is then *only* loaded if/when needed.
With this patch, the size of the `gulp mozcentral` build target is thus reduced accordingly:
| | `build/mozcentral`
|-------|-------------------
|master | 3 461 089
|patch | 3 340 268
Besides the PDF.js files taking up less space in Firefox this way, the additional benefit is that there's (by default) less code that needs to be loaded and parsed when the PDF Viewer is used which also cannot hurt.
---
[1] In `about:config`, by toggling the `javascript.options.streams` preference.
[2] Once in the `build/pdf.js` file, and once in the `build/pdf.worker.js` file.
This required the following changes in the Gulpfile:
- Defining a series of tasks is no longer done with arrays, but with the
`gulp.series` function. The `web` target is refactored to use a
smaller number of tasks to prevent tasks from running multiple times.
- Getting all tasks must now be done through the task registry.
- Tasks that don't return anything must call `done` upon completion.
Moreover, this upgrade allows us to use the latest Node.js on Travis CI
again.
This patch does three things:
- Updates the `gulp unittestcli` command, using `gulp lint` as a guide, such that it can be run locally on Windows without any modifications.
- Updates the `gulp lib` command to support disabling of Babel through the `SKIP_BABEL` environment variable. Note that all other build targets support this mode, and there's no good reason for `lib` to be any different here.
- Updates the `npm test` command, used in Node.js/Travis, to test the code as-is test. Since modern Node.js versions seem to have no problems with ES6 compatible code in general, we should just test the source code as-is instead (similar to the tests running on the regular bots).
Update configuration to work with Babel 7
Explicitly require globals - eslint-plugin-mozilla needs it, but doesn't require it on its own.
Fix Regexp to match Babel 7's inlined _interopRequireDefault
For proof-of-concept, this patch converts a couple of `Promise` returning methods to use `async` instead.
Please note that the `generic` build, based on this patch, has been successfully testing in IE11 (i.e. the viewer loads and nothing is obviously broken).
Being able to use modern JavaScript features like `async`/`await` is a huge plus, but there's one (obvious) side-effect: The size of the built files will increase slightly (unless `SKIP_BABEL == true`). That's unavoidable, but seems like a small price to pay in the grand scheme of things.
Finally, note that the `chromium` build target was changed to no longer skip Babel translation, since the Chrome extension still supports version `49` of the browser (where native `async` support isn't available).
This moves/exposes the `URL` polyfill similarily to the existing `ReadableStream` polyfill, rather than exposing it globally, to avoid interfering with any "outside" code.
Both the `URL` and `ReadableStream` polyfills are now exposed on the `pdfjsLib` object, such that they are accessible to the viewer components.
Furthermore, the `no-restricted-globals` ESLint rule is also enabled to prevent accidental usage of the native `URL`/`ReadableStream` implementations directly in the `src/` and `web/` folders; see also https://eslint.org/docs/rules/no-restricted-globals
Addresses the remaining TODO in https://github.com/mozilla/pdf.js/projects/6
Please note that the standalone `pdf.image_decoders.js` file will be including the complete `src/shared/util.js` file, despite only using parts of it.[1] This was done *purposely*, to not negatively impact the readability/maintainability of the core PDF.js code.
Furthermore, to ensure that the compatibility is the same in the regular PDF.js library *and* in the the standalone image decoders, `src/shared/compatibility.js` was included as well.
To (hopefully) prevent future complaints about the size of the built `pdf.image_decoders.js` file, a few existing async-related polyfills are being skipped (since all of the image decoders are completely synchronous).
Obviously this required adding a couple of pre-processor statements, but given that these are all limited to "compatibility" code, I think this might be OK!?
---
[1] However, please note that previous commits moved `PageViewport` and `MessageHandler` out of `src/shared/util.js` which reduced its size.
Since the tests (currently) run with the `pdf.worker.js` file built, i.e. with `PRODUCTION = true` set, there's no simple way to add e.g. `assert` calls for both non-production *and* test-only builds without also affecting PRODUCTION builds.
To avoid having to manually update the "shared files" list in `gulpfile.js`, whenever any changes are made to the file structure in the `src/shared/` folder, let's simply list the files we do *not* want instead.
Given that the excluded files have been consistent ever since the `lib` build target was added, this patch should help avoid unnecessary churn in `gulpfile.js` in the future.
Jasmine had a major version bump and required a few minor changes in our
booting code. Most notably, using `pending` in a `describe` block is no
longer supported, so we can only return early there. On the positive
side, the unit tests now run in a random order by default, which
eliminates any dependencies between unit tests.
Note that upgrading to Webpack 4 is out of scope for this patch since
the bots cannot work well with the newly generated bundles (both
browsers on both bots do not react within 120 seconds). Webpack 4 is not
faster for us than Webpack 3, so for now there is no need to upgrade.
Please note that this build target, and the resulting `build/pdf.combined.js` file, is equivalent to setting the `PDFJS.disableWorker` option to `true` which is a performance footgun.
These were removed in PR 9170, since they were unused in the browsers that we'll support in PDF.js version `2.0`.
However looking at the output of Travis, where a subset of the unit-tests are run using Node.js, there's warnings about `btoa` being undefined. This doesn't appear to cause any errors, which probably explains why we didn't notice this before (despite PR 9201).
The `gulp-util` module is now deprecated and authors are asked to stop
using it (refer to https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5 for
more information).
PDF.js does not rely on it that much, fortunately, so it's relatively
easy for us to remove the dependency. This patch does that by making the
following changes:
- Require `gulp-zip` version 4.1.0 or higher since they already removed
their `gulp-util` dependency in that version.
- Replace `gulp-util.log` with the `fancylog` module as recommended in
the article above.
- Replace `gulp-util.File` with the `Vinyl` module as recommended in the
article above.
The only change I had to make for Vinyl is removing the `base` and `cwd`
lines since they may not be empty strings anymore. This way we fall back
to the defaults Vinyl provides, which for us doesn't matter since we
move the file afterwards anyway. Moreover, I used `vfs` for `vinyl-fs`
in the `Gulpfile` to avoid confusion with `vinyl` (which is also how the
documentation names the variable).
This is all we can do on our side; the other modules that still use
`gulp-util` must be updated upstream.
Nothing uses this option anymore, so setting it is a no-op now. We can
safely remove it.
Use `SKIP_BABEL` (instead of `PDFJS_NEXT`) now if you want to skip Babel
translation for a build.
To support this, the following changes have to be made as well:
- Ignore `package-lock.json` since NPM creates it, but we should not
have it in the repository.
- Switch from `babel-preset-es2015` to `babel-preset-env` to resolve
the deprecation warning in the test logs. The latter is more recent
and flexible, but should be the same functionality-wise.
- `transform` now needs to have the `utf-8` encoding option provided.
If not given, it will call the callback with a `Buffer` object,
which results in an unhandled promise rejection since what is
returned from the callback is a string, not a `Buffer`.
Rather than (basically) duplicating the `SimpleLinkService` in `test/driver.js`, with potential test failuires if you forget to update the test mock, it seems much nicer to just re-use the viewer component.
Note that `SimpleLinkService` is already bundled into the `build/components/pdf_viewer.js` file. Hence we only need to expose it similar to the other viewer components in that file, and make sure that the `gulp components` command runs as part of the test-setup.
By updating to uglify-es, rather than uglify-js, the minifier *itself* now supports ES6 code. This means that it's now possible to minify code built with `PDFJS_NEXT = true` set, i.e. with Babel transpilation disabled, which wasn't the case previously.
Note that uglify-es is based on the API of uglify-js v3, which differs from the one that we previously used.
Of particular importance is the fact that it's no longer possible to provide a path to a file for minification, but one must instead directly provide the source of the file.
For more information, please see https://github.com/mishoo/UglifyJS2/tree/harmony
Use the environment's zlib implementation if available to get
reasonably-sized SVG files when an XObject image is converted to PNG.
The generated PNG is not optimal because we do not use a PNG predictor.
Futher, when our SVG backend is run in a browser, the generated PNG
images will still be unnecessarily large (though the use of blob:-URLs
when available should reduce the impact on memory usage). If we want to
optimize PNG images in browsers too, we can either try to use a DEFLATE
library such as pako, or re-use our XObject image painting logic in
src/display/canvas.js. This potential improvement is not implemented by
this commit
Tested with:
- Node.js 8.1.3 (uses zlib)
- Node.js 0.11.12 (uses zlib)
- Node.js 0.10.48 (falls back to inferior existing implementation).
- Chrome 59.0.3071.86
- Firefox 54.0
Tests:
Unit test on Node.js:
```
$ gulp lib
$ JASMINE_CONFIG_PATH=test/unit/clitests.json node ./node_modules/.bin/jasmine --filter=SVG
```
Unit test in browser: Run `gulp server` and open
http://localhost:8888/test/unit/unit_test.html?spec=SVGGraphics
To verify that the patch works as desired,
```
$ node examples/node/pdf2svg.js test/pdfs/xobject-image.pdf
$ du -b svgdump/xobject-image-1.svg
# ^ Calculates the file size. Confirm that the size is small
# (784 instead of 80664 bytes).
```
`__pdfjsdev_webpack__` was used to skip evaluating part of an AST,
in order to not mangle some `require` symbols.
This commit removes `__pdfjsdev_webpack__`, and:
- Uses `__non_webpack_require__` when one wants the output to
contain `require` instead of `__webpack_require__`.
- Adds options to the webpack config to prevent "polyfills" for
some Node.js-specific APIs to be added.
- Use `// eslint-disable-next-line no-undef` instead of `/* globals ... */`
for variables that are not meant to be used globally.
http://eslint.org/docs/rules/comma-danglehttp://eslint.org/docs/rules/object-curly-spacing
*Please note:* This patch was created automatically, using the ESLint `--fix` command line option. In a couple of places this caused lines to become too long, and I've fixed those manually; please refer to the interdiff below for the only hand-edits in this patch.
```diff
diff --git a/gulpfile.js b/gulpfile.js
index d18b9c58..7d47fd8d 100644
--- a/gulpfile.js
+++ b/gulpfile.js
@@ -1247,7 +1247,8 @@ gulp.task('gh-pages-git', ['gh-pages-prepare', 'wintersmith'], function () {
var reason = process.env['PDFJS_UPDATE_REASON'];
safeSpawnSync('git', ['init'], { cwd: GH_PAGES_DIR, });
- safeSpawnSync('git', ['remote', 'add', 'origin', REPO], { cwd: GH_PAGES_DIR, });
+ safeSpawnSync('git', ['remote', 'add', 'origin', REPO],
+ { cwd: GH_PAGES_DIR, });
safeSpawnSync('git', ['add', '-A'], { cwd: GH_PAGES_DIR, });
safeSpawnSync('git', [
'commit', '-am', 'gh-pages site created via gulpfile.js script',
```
After PR 8459, the run-time of the various `gulp` test commands has regressed quite badly on Windows. For me, `gulp test` now takes approximately *twice* as long when run locally on Windows.
The problem seems to be the Babel transpilation step, which takes well over five minutes to run.[1]
For someone like me, who runs tests a lot locally, this slowdown is really hurting the overall development experience.
To get around this I tested setting `PDFJS_NEXT = true` in `gulpfile.js`, since the transpilation step isn't necessary when testing in a modern browser.
However, having to edit `gulpfile.js` every time that I need to run tests isn't very practical. Hence this patch, which adds an environment variable that allows you to disable the transpilation simply by using e.g. `PDFJS_NEXT=true gulp test`.
I hope that this can be considered an acceptable solution, such that I don't need to maintain this patch locally (or worse, edit `gulpfile.js` locally before testing).
---
[1] This can also be observed on the Windows bot, but it seems fine on Linux.
Also updates a `console.log` statement in the `gulp importl10n` command (since I forgot it in a previous patch, and it didn't seem necessary with a separate patch for it).
Added test for ReadableStream.
Adds ref-implementation license-header in streams-lib
and change gulp task to copy external/streams/ in build/
external/streams/ and build/dist/external/streams folder.
Adds README.md and LICENSE.md
Note that we have to use `fs.writeFileSync` since `.to()` is not
available anymore. Moreover, introduce `safeSpawnSync` to make sure that
we check the return codes of the spawned processes properly.
To run the regression tests, developers use `gulp browsertest` and the
bot uses `gulp bottest`. We're not passing the `noreftest` option
anywhere in the code (probably because the `bottest` command takes care
of this already), so we should remove this.
Please see http://eslint.org/docs/rules/object-shorthand.
Unfortunately, based on commit 9276d1dcd9, it seems that we still need to maintain compatibility with old Node.js versions, hence certain files/directories that are executed in Node.js are currently exempt from this rule.
Furthermore, since the files specific to the Chromium extension are not run through Babel, the `/extensions/chromium/` directory is also exempt from this rule.
With the exception of just one test-case, all the current `ui_utils` unit-tests can run successfully on Node.js (since most of them doesn't rely on the DOM).
To get this working, I had to first of all add a new `LIB` build flag such that `gulp lib` produces a `web/pdfjs.js` file that is able to load `pdf.js` successfully.
Second of all, since neither `document` nor `navigator` is available in Node.js, `web/ui_utils.js` was adjusted slightly to avoid errors.
