With the removal of the (standalone) Firefox building code in PR 9566 (a year and a half ago), these files are now completely unused in the GitHub repository[1].
Hence it doesn't really seem necessary to keep fetching them with `gulp importl10n`, and the existing files in the `l10n` folder can also be removed (thanks to version control, they're easy enough to restore should the need ever arise).
The patch also allows an additional simplification, for the `gulp locale` and `gulp mozcentral` commands, since it's now possible to stop writing `l10n` files to the `extensions/firefox/` folder and instead just copy them similar to other build targets.
---
[1] They're obviously still used in `mozilla-central`, for fallback messages displayed through `PdfStreamConverter.jsm`, but that doesn't make it necessary to keep them *here* as far as I'm concerned.
Rather than having the script remove locales automatically, which seems like a heavy-handed approach at least initially, listing these for manual checking seems nice though.
Rather than having to manually maintain a static list of language codes, it's much easier to simply fetch the active ones from `mozilla-central` instead.
As part of this the code in `external/importL10n/locales.js` was modernized slightly, by using Promises/async functions to get rid of a bunch of annoying callbacks (which shouldn't be a problem for reasonably modern Node.js versions).
Compared to the recently replaced `URL` polyfill, the new `ReadableStream` polyfill isn't being exported globally for two reasons:
- We're currently checking for the existence of a global `ReadableStream` implementation when determining if the Fetch API will be used; please see `isFetchSupported` in the src/display/display_utils.js file.
- Given that it's much newer functionality (compared to `URL`) and that not all browsers may implement all parts of the specification yet, not exposing the `ReadableStream` globally seems safer for now.
Currently this leads to confusing behaviour with `eventBusDispatchToDOM = true` set, since then there's *two* different "localized" events being fired.
In many cases in the code you don't actually care about the index itself, but rather just want to know if something exists in a String/Array or if a String starts in a particular way. With modern JavaScript functionality, it's thus possible to remove a number of existing `indexOf` cases.
For proof-of-concept, this patch converts a couple of `Promise` returning methods to use `async` instead.
Please note that the `generic` build, based on this patch, has been successfully testing in IE11 (i.e. the viewer loads and nothing is obviously broken).
Being able to use modern JavaScript features like `async`/`await` is a huge plus, but there's one (obvious) side-effect: The size of the built files will increase slightly (unless `SKIP_BABEL == true`). That's unavoidable, but seems like a small price to pay in the grand scheme of things.
Finally, note that the `chromium` build target was changed to no longer skip Babel translation, since the Chrome extension still supports version `49` of the browser (where native `async` support isn't available).
Without providing useful (custom) error messages for the `no-restricted-globals` rule, see https://eslint.org/docs/rules/no-restricted-globals, it's quite likely that the rule will be incorrectly disabled rather than the required globals being imported as intended.
To reduced duplication of the `no-restricted-globals` rule in multiple `.eslintrc` files, it's instead moved to the top-level `.eslintrc` file and disabled as needed on a folder/file basis outside of `/src` and `/web`.
It gives warnings when running `gulp locale`, in a previous patch it was
explicitly asked to remove it and there is quite a bit of discussion
about the validity of the locale at
https://bugzilla.mozilla.org/show_bug.cgi?id=418485. In short, let's
remove it to prevent these issues.
This moves/exposes the `URL` polyfill similarily to the existing `ReadableStream` polyfill, rather than exposing it globally, to avoid interfering with any "outside" code.
Both the `URL` and `ReadableStream` polyfills are now exposed on the `pdfjsLib` object, such that they are accessible to the viewer components.
Furthermore, the `no-restricted-globals` ESLint rule is also enabled to prevent accidental usage of the native `URL`/`ReadableStream` implementations directly in the `src/` and `web/` folders; see also https://eslint.org/docs/rules/no-restricted-globals
Addresses the remaining TODO in https://github.com/mozilla/pdf.js/projects/6
This required changing the import script in two ways:
- we should use the `default` branch and not the `tip` tag since the
latter may refer to another branch than `default` (this is the case for
the `vi` locale, which caused in the files to be overwritten with
incorrect contents since `tip` referred to the
`THUNDERBIRD600b1_2018031614_RELBRANCH` branch);
- we should check if the response code is indeed 200 because recently a
script removed all empty localization files upstream (refer to
https://bugzilla.mozilla.org/show_bug.cgi?id=1443175).
This rule is available from https://www.npmjs.com/package/eslint-plugin-mozilla, and is enforced in mozilla-central. Note that we have the necessary `Array`/`String` polyfills and that most cases have already been fixed, see PRs 9032 and 9434.
Despite this patch removing the `disableWorker` option itself, please note that we'll still fallback to loading the worker file(s) on the main-thread when running in environments without proper Web Worker support.
Furthermore it's still possible, even with this patch, to force the use of fake workers by manually loading the necessary file using a `<script>` tag on the main-thread.[1]
That way, the functionality of the now removed `SINGLE_FILE` build target and the resulting `build/pdf.combined.js` file can still be achieved simply by adding e.g. `<script src="build/pdf.worker.js"></script>` to the HTML (obviously with the path adjusted as needed).
Finally note that the `disableWorker` option is a performance footgun, and unfortunately many existing third-party examples actually use it without providing any sort of warning/justification.
---
[1] This approach is used in the default viewer, since certain kind of debugging may be easier if the code is running directly on the main-thread.
`__pdfjsdev_webpack__` was used to skip evaluating part of an AST,
in order to not mangle some `require` symbols.
This commit removes `__pdfjsdev_webpack__`, and:
- Uses `__non_webpack_require__` when one wants the output to
contain `require` instead of `__webpack_require__`.
- Adds options to the webpack config to prevent "polyfills" for
some Node.js-specific APIs to be added.
- Use `// eslint-disable-next-line no-undef` instead of `/* globals ... */`
for variables that are not meant to be used globally.
Adds functionality to accept Queueing Strategy in
sendWithStream method. Using Queueing Strategy we
can control the data that is enqueued into the sink,
and hence regulated the flow of chunks from worker
to main thread.
Adds capability in pull and cancel methods.
Adds ready and desiredSize property in streamSink.
Adds unit test for ReadableStream and sendWithStream.
http://eslint.org/docs/rules/comma-danglehttp://eslint.org/docs/rules/object-curly-spacing
*Please note:* This patch was created automatically, using the ESLint `--fix` command line option. In a couple of places this caused lines to become too long, and I've fixed those manually; please refer to the interdiff below for the only hand-edits in this patch.
```diff
diff --git a/gulpfile.js b/gulpfile.js
index d18b9c58..7d47fd8d 100644
--- a/gulpfile.js
+++ b/gulpfile.js
@@ -1247,7 +1247,8 @@ gulp.task('gh-pages-git', ['gh-pages-prepare', 'wintersmith'], function () {
var reason = process.env['PDFJS_UPDATE_REASON'];
safeSpawnSync('git', ['init'], { cwd: GH_PAGES_DIR, });
- safeSpawnSync('git', ['remote', 'add', 'origin', REPO], { cwd: GH_PAGES_DIR, });
+ safeSpawnSync('git', ['remote', 'add', 'origin', REPO],
+ { cwd: GH_PAGES_DIR, });
safeSpawnSync('git', ['add', '-A'], { cwd: GH_PAGES_DIR, });
safeSpawnSync('git', [
'commit', '-am', 'gh-pages site created via gulpfile.js script',
```
Also updates a `console.log` statement in the `gulp importl10n` command (since I forgot it in a previous patch, and it didn't seem necessary with a separate patch for it).
Added test for ReadableStream.
Adds ref-implementation license-header in streams-lib
and change gulp task to copy external/streams/ in build/
external/streams/ and build/dist/external/streams folder.
Adds README.md and LICENSE.md
Please see http://eslint.org/docs/rules/object-shorthand.
Unfortunately, based on commit 9276d1dcd9, it seems that we still need to maintain compatibility with old Node.js versions, hence certain files/directories that are executed in Node.js are currently exempt from this rule.
Furthermore, since the files specific to the Chromium extension are not run through Babel, the `/extensions/chromium/` directory is also exempt from this rule.
This patch is another step towards enabling Babel. Since we're moving
towards ES6 modules, we will not be using UMD headers anymore, so we can
remove the validation.
Please see http://eslint.org/docs/rules/no-unused-vars; note that this patch purposely uses the same rule options as in `mozilla-central`, such that it fixes part of issue 7957.
It wasn't, in my opinion, entirely straightforward to enable this rule compared to the already existing rules. In many cases a `var descriptiveName = ...` format was used (more or less) to document the code, and I choose to place the old variable name in a trailing comment to not lose that information.
I welcome feedback on these changes, since it wasn't always entirely easy to know what changes made the most sense in every situation.
Currently you have to open the files to be certain which tests each one will run, which we can avoid by appending the directory names to the file names of the tests themselves.
*Please note that most of the necessary code adjustments were made in PR 7890.*
ESLint has a number of advantageous properties, compared to JSHint. Among those are:
- The ability to find subtle bugs, thanks to more rules (e.g. PR 7881).
- Much more customizable in general, and many rules allow fine-tuned behaviour rather than the just the on/off rules in JSHint.
- Many more rules that can help developers avoid bugs, and a lot of rules that can be used to enforce a consistent coding style. The latter should be particularily useful for new contributors (and reduce the amount of stylistic review comments necessary).
- The ability to easily specify exactly what rules to use/not to use, as opposed to JSHint which has a default set. *Note:* in future JSHint version some of the rules we depend on will be removed, according to warnings in http://jshint.com/docs/options/, so we wouldn't be able to update without losing lint coverage.
- More easily disable one, or more, rules temporarily. In JSHint this requires using a numeric code, which isn't very user friendly, whereas in ESLint the rule name is simply used instead.
By default there's no rules enabled in ESLint, but there are some default rule sets available. However, to prevent linting failures if we update ESLint in the future, it seemed easier to just explicitly specify what rules we want.
Obviously this makes the ESLint config file somewhat bigger than the old JSHint config file, but given how rarely that one has been updated over the years I don't think that matters too much.
I've tried, to the best of my ability, to ensure that we enable the same rules for ESLint that we had for JSHint. Furthermore, I've also enabled a number of rules that seemed to make sense, both to catch possible errors *and* various style guide violations.
Despite the ESLint README claiming that it's slower that JSHint, https://github.com/eslint/eslint#how-does-eslint-performance-compare-to-jshint, locally this patch actually reduces the runtime for `gulp` lint (by approximately 20-25%).
A couple of stylistic rules that would have been nice to enable, but where our code currently differs to much to make it feasible:
- `comma-dangle`, controls trailing commas in Objects and Arrays (among others).
- `object-curly-spacing`, controls spacing inside of Objects.
- `spaced-comment`, used to enforce spaces after `//` and `/*. (This is made difficult by the fact that there's still some usage of the old preprocessor left.)
Rules that I indend to look into possibly enabling in follow-ups, if it seems to make sense: `no-else-return`, `no-lonely-if`, `brace-style` with the `allowSingleLine` parameter removed.
Useful links:
- http://eslint.org/docs/user-guide/configuring
- http://eslint.org/docs/rules/
Currently `importl10n` fails, since mxr.mozilla.org has been down for a while. Based on discussions in #developers, it isn't clear to me if/when it'll be available again.
Furthermore, I'm not sure why we're even getting the l10n files through a code search tool, rather than just using the official Mozilla repository directly.
This is regression from PR 7063, causing `gulp importl10n` to fail:
```
$ gulp importl10n
[19:45:02] Using gulpfile c:\Users\Jonas\Git\pdfjs\gulpfile.js
[19:45:02] Starting 'importl10n'...
Downloading ach...
[19:45:02] 'importl10n' errored after 4.42 ms
[19:45:02] Error: EEXIST, file already exists 'c:\Users\Jonas\Git\pdfjs\l10n\ach
'
at Error (native)
at Object.fs.mkdirSync (fs.js:747:18)
at downloadLanguageFiles (c:\Users\Jonas\Git\pdfjs\external\importL10n\local
es.js:59:8)
at next (c:\Users\Jonas\Git\pdfjs\external\importL10n\locales.js:90:5)
at Object.downloadL10n (c:\Users\Jonas\Git\pdfjs\external\importL10n\locales
.js:91:5)
at Gulp.<anonymous> (c:\Users\Jonas\Git\pdfjs\gulpfile.js:92:11)
at module.exports (c:\Users\Jonas\Git\pdfjs\node_modules\gulp\node_modules\o
rchestrator\lib\runTask.js:34:7)
at Gulp.Orchestrator._runTask (c:\Users\Jonas\Git\pdfjs\node_modules\gulp\no
de_modules\orchestrator\index.js:273:3)
at Gulp.Orchestrator._runStep (c:\Users\Jonas\Git\pdfjs\node_modules\gulp\no
de_modules\orchestrator\index.js:214:10)
at Gulp.Orchestrator.start (c:\Users\Jonas\Git\pdfjs\node_modules\gulp\node_
modules\orchestrator\index.js:134:8)
```
Features / bug fixes in the preprocessor:
- Add word boundary after regex for preprocessor token matching.
Previously, when you mistakenly used "#ifdef" instead of "#if", the
line would be parsed as a preprocessor directive (because "#ifdef"
starts with "#if"), but without condition (because "def" does not
start with a space). Consequently, the condition would always be false
and anything between "#ifdef" and "#endif" would not be included.
- Add validation and error reporting everywhere, to aid debugging.
- Support nested comments (by accounting for the whole stack of
conditions, instead of only the current one).
- Add #elif preprocessor command. Could be used as follows:
//#if !FEATURE_ENABLED
//#error FEATURE_ENABLED must be set
//#endif
- Add #error preprocessor command.
- Add end-of-line word boundary after "-->" in the comment trimmer.
Otherwise the pattern would also match "-->" in the middle of a line,
and incorrectly convert something like "while(i-->0)" to "while(i0)".
Code health:
- Add unit tests for the preprocessor (run external/builder/test.js).
- Fix broken link to MDN (resolved to DXR).
- Refactor to use STATE_* names instead of magic numbers (the original
meaning of the numbers is preserved, with one exception).
- State 3 has been split in two states, to distinguish between being in
an #if and #else. This is needed to ensure that #else cannot be
started without an #if.
It took a while to figure out why adding comments in worker_loader.js
caused the build to fail, because getWorkerSrcFiles did not print an
error message when it failed to parse the file. These issues have been
resolved as follows:
- Leading comments are stripped.
- The trailing comma is removed from the array.
- Errors are detected and useful error messages are printed.
There is no need to have a copy of ShellJS in the repository as it is also available on NPM. The NPM version is also much newer. This way we do not have to update this anymore and let NPM do that automatically.
See "Case-insensitive language comparisons per RFC 4646"
https://github.com/fabi1cazenave/webL10n/pull/51
Removed the "patch" in compatibility.js and fixed the bug in webL10n,
because the logic belongs to webL10n instead of PDF.js.
The immediate motivation for this patch is that Chrome 40 converts
navigator.language to lower case: https://crbug.com/454331
This refactors getData to be more readable and extracts all the color
conversion algorithms to their own functions. The resulting code was then
cleaned up.
This also introduces a flag `forceRGBoutput` to getData, that allows to always
get the data as a `width * height * 3` bytes long RGB buffer
Based on http://pastebin.mozilla.org/3061694
1. Remove "debug helpers" and use console.log/console.warn directly (top).
2. Remove page load initialization (middle).
3. Remove window._ alias (bottom)
The original diff contained an extra entry
"Adds fallback argument to the getL10nData;",
but this was already implemented in the rebased webL10n, so it's no
longer PDF.js-specific.
