An old shortcoming of the `preprocessCSS`-function is its complete lack of support for our "normal" defines, which makes it very difficult to have build-specific CSS rules. Recently we've started using specially crafted comments to remove CSS rules from the MOZCENTRAL build, but (ab)using the `preprocessCSS`-function in this way really doesn't feel great.
However, it turns out to be surprisingly simple to instead use the "regular" `preprocess`-function for the CSS files as well. The only special-handling that's still necessary is the helper-function for dealing with CSS-imports, but apart from that everything seems to just work.
One reason, as far as I can tell, for having a separate `preprocessCSS`-function was likely that we originally used *lots* of vendor-prefixed CSS rules in our CSS files. With improvements over the years, especially thanks to Autoprefixer and PostCSS, we've been able to remove *almost* all non-standard CSS rules and the need for special-casing the CSS parsing has mostly vanished.
*Please note:* As part of testing this patch I've diffed the output of `gulp generic`, `gulp mozcentral`, and `gulp chromium` against the `master`-branch to check that there was no obvious breakage.
There are two notable changes here:
- `dommatrix` is a major version upgrade, but looking through the commit
history of their `package.json` file at https://github.com/thednp/dommatrix/commits/master/package.json
(due to the lack of a changelog) I couldn't find any breaking changes.
- `es-module-shims` is a regular update, but it was previously pinned
for causing intermittent breakage when running the unit tests in a
browser manually. Fortunately this cannot be reproduced anymore with
the most recent version, so we can also put the caret back now.
Note how both of the openFile-buttons are always hidden during viewer initialization in the MOZCENTRAL build, i.e. the *built-in* Firefox PDF Viewer. Despite that we still include HTML, CSS, and JavaScript code for these buttons in the build.
This patch *reduces* the size of the `gulp mozcentral` output by `1679` bytes, which isn't a lot but still cannot hurt.
Given that we're no longer, after PR 14560, bundling the `web-streams-polyfill`-code in the `legacy`-builds we shouldn't need to exclude it from Babel now.
Given that none of these CSS rules are used at all, unless debugging is enabled, it seems completely unnecessary to load them *unconditionally* for all users.[1]
Note that if *both* the `textLayer` and `pdfBug` debugging hash-parameters are specified simultaneously, we'll now load the `PDFBug`-file *twice* (since the code is simpler that way). However, given first of all that none of this is enabled by default and secondly that using those parameters together isn't helpful[2], potentially loading that file twice is hopefully not an issue.
For the `gulp mozcentral` target, the size of the *built* `viewer.css` file is reduced `> 3%` with this patch.
---
[1] For the Firefox built-in PDF Viewer, in order to even be able to access the `PDFBug` functionality, you need to first of all set `pdfjs.pdfBugEnabled = true` manually in `about:config`. Secondly, you then also need to append the `pdfBug=...` hash-parameter to the URL when *initially* loading the document.
[2] Note how the `textLayer`-settings are already, since essentially forever, overriding the highlighting-features of the "FontInspector"-tab.
The various functionality in `web/debugger.js` is currently *indirectly* added to the global scope, since that's how `var` works when it's used outside of any functions/closures.
Given how this functionality is being accessed/used, not just in the viewer but also in the API and textLayer, simply converting the entire file to a module isn't trivial[1]. However, we can at least export the `PDFBug`-part properly and then `import` that dynamically in the viewer.
Also, to improve the code a little bit, we're now *explicitly* exporting the necessary functionality globally.
According to MDN, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#browser_compatibility, all the browsers that we now support have dynamic `imports` implementations.
---
[1] We could probably pass around references to the necessary functionality, but given how this is being used I'm just not sure it's worth the effort. Also, adding *official* support for these viewer-specific debugging tools in the API feels both unnecessary and unfortunate.
Rather than *manually* specifying a "mode", we can simply use the regular `defines` directly instead. To improve consistency, in the `external/builder/builder.js` file, a couple of parameters are also re-named.
To allow using modern CSS features that currently only Mozilla Firefox supports[1], while still enabling development/testing in recent Google Chrome versions, we'll have to start building the `web/viewer.css` file with `gulp server` as well.
In my testing, building the development CSS (and copying the images) takes *less than* `200 ms` on average which is hopefully an acceptable overhead for this sort of feature.
---
[1] In particular `float`, with `inline-start`/`inline-end` values.
Every single call-site has always passed in `true` for this parameter, ever since the function was first added back in PR 8023. Hence the parameter appears to be completely unnecessary, which is why it's removed and the function is updated to *unconditionally* strip out any license headers (in the middle of the file).
According to the MDN compatibility data, see https://developer.mozilla.org/en-US/docs/Web/API/DOMMatrix/DOMMatrix#browser_compatibility, all browsers that we support have native `DOMMatrix` implementations (since quite some time too).
Hence Node.js is the only environment that lack `DOMMatrix` support, which probably isn't that surprising given that it's browser functionality.
While the `DOMMatrix` polyfill isn't that large, it nonetheless seems completely unnecessary to bundle it in the `legacy` builds when it's not needed in browsers. However, we can avoid that by simply listing `dommatrix` as a dependency for the `pdfjs-dist` library.
Since the Autoprefixer plugin indirectly depends on this, it seems like a good idea to add this as a direct dependency in the PDF.js project to hopefully avoid having to manually update `caniuse-lite` in the future; see https://github.com/browserslist/browserslist#browsers-data-updating
Also, slightly tweaks the Autoprefixer config for GENERIC-builds of the PDF.js library; note that this change doesn't affect the contents of the *built* `web/viewer.js` file.
The "External: Promise"-page in the JSDocs pre-dates the introduction of `Promise`s, as a generally available standard JS feature, by a number of years. Hence it now longer seems necessary, as far as I can tell, to include this "special" page in the documentation.
Also, while unrelated to the rest of the patch, updates the `test/`-folder description in the documentation.
According to the MDN compatibility data, see https://developer.mozilla.org/en-US/docs/Web/API/ReadableStream#browser_compatibility, all browsers that we support have native `ReadableStream` implementations (since quite some time too).
Hence only Node.js is now lagging behind w.r.t. `ReadableStream` support, and its experimental implementation doesn't really help us given the life-span of the LTS releases (see https://en.wikipedia.org/wiki/Node.js#Releases).
It seems quite unfortunate to bundle a `ReadableStream` polyfill in the `legacy` builds when it's unnecessary in browsers, given its overall size, but fortunately we can avoid that by simply listing `web-streams-polyfill` as a dependency for the `pdfjs-dist` library.
Please note that while we "support" some (by now) fairly old browsers, that essentially means that the library (and viewer) will load and that the basic functionality will work as intended.[1]
However, in older browsers, some functionality may not be available and generally we'll ask users to update to a modern browser when bugs (specific to old browsers) are reported.[2]
There's always a question of just how old browsers the PDF.js contributors can realistically support, and here I'm suggesting that we place the cut-off point at approximately *three* years.
With that in mind, this patch updates the *minimum* supported browsers (and environments) as follows:
- Chrome 73, which was released on 2019-03-12; see https://en.wikipedia.org/wiki/Google_Chrome_version_history
- Firefox ESR (as before); see https://wiki.mozilla.org/Release_Management/Calendar
- Safari 12.1, which was released on 2019-03-25; see https://en.wikipedia.org/wiki/Safari_version_history#Safari_12
- Node.js 12, which was release on 2019-04-23 (and will soon reach EOL); see https://en.wikipedia.org/wiki/Node.js#Releases
---
[1] Assuming a `legacy`-build is being used, of course.
[2] In general it's never a good idea to use an old/outdated browser, since those may contain *known* security vulnerabilities.
Either the latest Chromium update, the latest Puppeteer update, or a combination of them both are now causing the Windows bot to timeout during the browser-tests; please see PR 14392.
Either the latest Chromium update, the latest Puppeteer update, or a combination of them both are now causing the Windows bot to timeout during the browser-tests.
To unblock both the updates and other improvements (i.e. the `structuredClone` polyfill), let's simply disable the problematic configuration for now since this a Mozilla project after all.
With ESLint 8 we should now finally be able to start using modern `class` features, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Public_class_fields and https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Classes/Private_class_fields
However, while both ESLint and Acorn now support this, it unfortunately turns out that Escodegen (which we use during building) still lack the necessary support. Looking at https://github.com/estools/escodegen there's not been any updates since last year, and there's also open PRs adding support for these new `class` features.
To avoid blocking usage of these `class` features in the PDF.js code-base, in particular *private* fields/methods, this patch thus proposes that we (hopefully temporarily) switch to an `escodegen` fork that has the necessary support; please see https://www.npmjs.com/package/@javascript-obfuscator/escodegen
While I have no reason to doubt the security of the `escodegen` fork, this patch nonetheless pins the version number. Furthermore, I've also diffed the output of the two `.js`-files in this forked package against the original files without finding anything that looks immediately "dangerous".
This command was added all the way back when basic CI-support was first introduced (using Travis at the time), however it's never really intended to be used e.g. for local development.
By having a `npm test`-command listed in the `package.json` file, there's a very real risk that someone unfamiliar with the code-base would only run that one and thus miss all the other (more important) test-suites[1].
Hence this patch which removes the `npm test`-command, and instead simply calls the relevant gulp-task[2] directly in the GitHub Actions configuration.
---
[1] Which consist of the unit-tests (run in browsers), the font-tests (potentially), the reference-tests, and the integration-tests.
[2] Which is also renamed slightly, to better fit its current usage.
Currently a `TESTING = true` environment variable will *always* take precedence in the various build-tasks, and there's no way to explicitly disable it for a particular build. That's clearly an oversight on my part, however it's easy enough to fix this; sorry about breaking this!
The new command is a *variation* of the standard `gulp test` command and will run all unit/font/integration-tests just as normal, while *only* running ref-tests for XFA-documents to speed up development.
Given that we currently have (some) unit-tests for XFA-documents, and that we may also (in the future) want to add integration-tests, it thus makes sense to run all test-suites in my opinion.
*Please note:* Once this patch has landed, I'll submit a follow-up patch to https://github.com/mozilla/botio-files-pdfjs such that we can also run the new command on the bots.
Given that we've over time been reducing the number of `compatibilityParams` in use, there's now few enough left that I think it makes sense to simply inline them directly in the `web/app_options.js` file.
Note that we recently inlined/removed the separate `src/display/api_compatibility.js` file, see PR 13525, and that it (in my opinion) thus makes sense to do the same in the `web/`-folder. This patch will also slightly reduce the size of *built* `web/viewer.js` file, which cannot hurt.
Currently it's possible to run e.g. `gulp unittest`, `gulp fonttest`, and `gulp integrationtest` *separately* on the bots; see https://github.com/mozilla/botio-files-pdfjs
However, it's not possible to run *only* the `gulp browsertest` command on the bots without also running the full test-suite. In some cases, e.g. if the "browsertest" times out, having a way to only re-run those would thus save some time and resources.
If/when this patch lands, I'll follow-up with a patch adding a new `on_cmd_browsertest.js` file to the https://github.com/mozilla/botio-files-pdfjs repository.
Please note that while the `gulp types`/`gulp typestest` tasks (obviously) still work with this patch, I've got no idea if the output is first of all even useable and secondly if it's actually useful in practice.
However, in the interest of closing some (seemingly simple) issues, I suppose that this probably shouldn't hurt (and we'd need TypeScript users to help improve things here).
By adding basic linting of JSON files, we can ensure that they're actually valid and prevent e.g. test-failures caused by *accidental* errors when editing the `test/test_manifest.json` file (something that I've done *many* times myself).
For now this simply uses the `recommended` configuration, but we can obviously tweak this later if/when needed. Please find additional information at https://github.com/azeemba/eslint-plugin-json
Given that `DOMMatrix` is, unsurprisingly, not supported in Node.js the `createMatrix` helper function in `src/display/pattern_helper.js` is most likely broken in Node.js environments. It will obviously try to fallback to the `DOMSVGFactory`, however that isn't intended for Node.js usage and errors will be thrown.
Rather than trying to implement a `NodeSVGFactory`, this patch takes the easier route of just adding a `DOMMatrix` polyfill using: https://www.npmjs.com/package/dommatrix
This isn't done only for simplicity, but it'll become necessary anyway since the `createMatrix` helper function is only temporary and will be removed in the future.
Currently only the Foxit license-file is included, which is most likely just an oversight as far as I can tell.
Furthermore, to be able the tell the two license-files apart, the Foxit one is also renamed slightly.
- Some js files contain scale factors for each glyph in order to rescale Liberation to have a final font with the correct width.
- A lot of XFA have some containers where their dimensions are based on their text content, so using default font from browser can lead to an almost unreadable pdf.
Looking at the history of the code, this `gulp` task was *accidentally* broken in the Gulp 4 upgrade in PR 10266 (all the way back in 2018).
Given that no one has either noticed and/or complained about this since, it's safe to assume that it's completely unused, hence why I suggest that we just remove this `gulp` task (rather than fixing it).
Most of the `gulp` tasks with a "-pre" suffix, especially those related to the main build-targets, are no longer needed and can simply be inlined in their corresponding tasks. This is likely a left-over from older Gulp-versions, but currently it only adds unnecessary indirection.
Furthermore, a number of the `gulp` tasks which are only used *internally* are converted to standard functions instead. This prevents them from being exposed "globally" and e.g. listed by `gulp default`.
With the changes in PR 13361, we're now using the `CanvasPattern.setTransform()` method when rendering certain Shadings/Patterns.
Note that while `CanvasPattern` itself has been supported since basically "forever", its `setTransform` method is a slightly newer addition to the specification; please refer to https://developer.mozilla.org/en-US/docs/Web/API/CanvasPattern#browser_compatibility
Rather than trying to re-write PR 13361 to not use, or possibly spending time/effort (if possible) polyfilling, `CanvasPattern.setTransform()` this patch thus suggests that we simply update the *minimum* supported browser versions instead.
According to the compatibility data linked above, the *minimum* supported browser versions in the PDF.js library are now as follows:
- Chrome >= 68, which was released on 2018-07-24.[1]
- Firefox ESR, see https://wiki.mozilla.org/Release_Management/Calendar.
- Safari >= 11.1, which was release on 2018-03-29.[2]
(Given that the PDF.js contributors cannot realistically test a bunch of old browsers, it's not unimaginable that some older browser versions are already not working with the PDF.js library.)
Based on these changes, which we should ensure are reflected in the Wiki as well, we can also remove a number of now redundant polyfills. Furthermore we'll no longer "claim" to support Windows XP, note the `gulpfile.js` changes, which should definitely *not* be an issue given that it's no longer officially supported.[3]
---
[1] According to https://en.wikipedia.org/wiki/Google_Chrome_version_history
[2] According to https://en.wikipedia.org/wiki/Safari_version_history#Safari_11
[3] According to https://en.wikipedia.org/wiki/Windows_XP#End_of_support
With the changes made in the previous patch, we can now list "disableTelemetry" in the `AppOptions` only for the `CHROME`-builds and thus remove the special-casing in the `checkChromePreferencesFile` helper function.
Currently there's a lot of duplication in the `buildLib` and `buildDefaultPreferences` functions, which seem quite unfortunate. Hence this patch extracts the common functionality in a new `buildLibHelper` function instead.
Originally the default preferences where simply placed in a JSON-file, checked into the repository, which over time became impractical, annoying, and error-prone to maintain; please see PR 10548.
While that improved the overall situation a fair bit, it however inherited one quite unfortunate property of the old JSON-based solution[1]: It's still not possible for *different* build targets to specify their *own* default preference values.
With some preferences, such as e.g. `enableScripting`, it's not inconceivable that you'd want to (at least) support build-specific default preference values. Currently that's not really possible, which is why this PR re-factors the default preferences generation to support this.
---
[1] This fact isn't really clear from the `AppOptions` implementation, unless you're familiar with the `gulpfile.js` code, which could lead to some confusion for those new to this part of the code-base.
A significant portion of the code-base has now been converted to use `let`/`const`, rather than `var`, hence it should be possible to simply enable the ESLint `no-var` rule globally.
This way we can ensure that new code won't accidentally use `var`, and it also removes the need to manually enable the rule in various folders.
Obviously it makes sense to continue the efforts to replace `var`, but that should probably happen on a file and/or folder basis.
Please note that this patch excludes the following code:
- The `extensions/` folder, since that seemed easiest for now (and I don't know exactly what the support situation is for the Chromium-extension).
- The entire `external/` folder is ignored, since most of it's currently excluded from linting.
For the code that isn't imported from elsewhere (and should be ignored), we should probably (at some point) bring the code up to the same linting/formatting standard as the rest of the code-base.
- Various files in the `test/` folder are ignored, as necessary, since the way that a lot of this code is loaded will require some care (or perhaps larger re-factoring) when removing `var` usage.
