[Security] Update yarn dependencies to fix security issues

@xjm was asking for this issue in the #d9readiness meeting today.

Current results:

9.1.x

22 vulnerabilities found - Packages audited: 938
Severity: 21 Low | 1 Moderate

9.0.x

22 vulnerabilities found - Packages audited: 938
Severity: 21 Low | 1 Moderate

8.9.x

299 vulnerabilities found - Packages audited: 870
Severity: 296 Low | 1 Moderate | 2 High

high

% yarn audit --level high
yarn audit v1.22.4
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ high          │ Machine-In-The-Middle                                        │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ https-proxy-agent                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=2.2.3                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ nightwatch                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ nightwatch > proxy-agent > https-proxy-agent                 │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1184                        │
└───────────────┴──────────────────────────────────────────────────────────────┘
┌───────────────┬──────────────────────────────────────────────────────────────┐
│ high          │ Machine-In-The-Middle                                        │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ https-proxy-agent                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=2.2.3                                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ nightwatch                                                   │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ nightwatch > proxy-agent > pac-proxy-agent >                 │
│               │ https-proxy-agent                                            │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1184                        │
└───────────────┴──────────────────────────────────────────────────────────────┘

Moderate

┌───────────────┬──────────────────────────────────────────────────────────────┐
│ moderate      │ Regular Expression Denial of Service                         │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Package       │ acorn                                                        │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Patched in    │ >=5.7.4 <6.0.0 || >=6.4.1 <7.0.0 || >=7.1.1                  │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Dependency of │ eslint                                                       │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ Path          │ eslint > espree > acorn                                      │
├───────────────┼──────────────────────────────────────────────────────────────┤
│ More info     │ https://www.npmjs.com/advisories/1488                        │
└───────────────┴──────────────────────────────────────────────────────────────┘

8.8.x

299 vulnerabilities found - Packages audited: 870
Severity: 296 Low | 1 Moderate | 2 High

(Same as 8.9.x)

I believe the 9.y.x parts of this are blocked on #3107926: Update stylelint to ^13.0.0

Not at all sure this is the right approach, but here's a start. ;)

I can't seem to upgrade just https-proxy-agent nor proxy-agent:

% yarn upgrade https-proxy-agent
yarn upgrade v1.22.4
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Rebuilding all packages...
success Saved lockfile.
success Saved 0 new dependencies.
✨  Done in 5.01s.

The only thing that seems to work is updating nightwatch itself. :( See attached for full command output.
% yarn upgrade nightwatch
yarn upgrade v1.22.4
...
success Saved lockfile.
success Saved 95 new dependencies.
...
✨ Done in 14.39s

After that yarn audit --level high is coming back empty. Let's see what the bot thinks of this.

Note, to resolve the 'Moderate' issue, it seems I have to use 'yarn upgrade eslint'. Perhaps that should be better handled as an 8.9.x backport of #3107926: Update stylelint to ^13.0.0?

Re: #6: Sorry, I misread. That's about "stylelint" not "eslint", which aren't the same. ;) So, would like input on if this issue should handle both eslint and nightwatch for 8.9.x or if we should split out eslint to a separate issue.

The 9.1.x "moderate" is the same vulnerability as 8.9.x: eslint > espree > acorn

So perhaps it makes sense to have a separate issue for "Update eslint yarn dependency" that can be for all branches?

Per Slack w/ @xjm, here's an 8.9.x patch that includes both nightwatch and eslint upgrades.

Now that #3107926: Update stylelint to ^13.0.0 landed, here's a patch for D9 (applies to both 9.1.x and 9.0.x) to upgrade eslint to fix the moderate vuln there.

yarn-lock-diff of #9:

package name	old version(s)	new version(s)
@types/node	[...; 8.10.51]	[...]
acorn	5.7.3	5.7.4
ansi-regex	[...]	[...; 5.0.0]
ansi-styles	[...]	[...; 4.2.1]
ast-types	0.13.2	0.13.3
camelcase	[...]	[...; 5.3.1]
chai-nightwatch	0.3.0	0.4.0
chalk	[...]	[...; 3.0.0]
cli-cursor	[...]	[...; 3.1.0]
cli-width	2.2.0	2.2.1
color-convert	[...]	[...; 2.0.1]
commander	[...; 2.20.0]; 2.15.1	[...]; 2.20.0
data-uri-to-buffer	2.0.1	1.2.0
ejs	2.6.2	2.7.4
es-abstract	[...]	[...; 1.17.5]
es-to-primitive	[...]	[...; 1.2.1]
escodegen	1.12.0	1.14.1
esquery	1.0.1	1.3.1
estraverse	[...]	[...; 5.1.0]
fast-json-stable-stringify	2.0.0	2.1.0
find-up	[...]	[...; 3.0.0]
get-uri	2.0.3	2.0.4
glob	[...]; 7.1.2	[...; 7.1.6]; 7.1.3
graceful-fs	[...]	[...; 4.2.4]
has-flag	[...]	[...; 4.0.0]
has-symbols	[...]	[...; 1.0.1]
he	1.1.1	1.2.0
https-proxy-agent	2.2.2	3.0.1
is-callable	[...]	[...; 1.1.5]
is-promise	2.1.0	-
is-regex	[...]	[...; 1.0.5]
js-yaml	[...]	[...; 3.14.0]
locate-path	[...]	[...; 3.0.0]
log-symbols	[...]	[...; 3.0.0]
lru-cache	[...]	[...; 5.1.1]
mimic-fn	[...]	[...; 2.1.0]
mkdirp	[...]	[...; 0.5.4]
mocha	5.2.0	6.2.3
ms	[...]; 2.1.2	[...; 2.1.2]; 2.1.1
mute-stream	[...]	[...; 0.0.8]
nightwatch	1.2.1	1.3.6
onetime	[...]	[...; 5.1.0]
optimist	0.6.1	-
optionator	0.8.2	0.8.3
p-limit	[...]	[...; 2.3.0]
p-locate	[...]	[...; 3.0.0]
p-try	[...]	[...; 2.2.0]
pac-proxy-agent	3.0.0	3.0.1
proxy-agent	3.1.0	3.1.1
proxy-from-env	1.0.0	1.1.0
psl	[...]	[...; 1.8.0]
readable-stream	[...]; 3.4.0	[...; 3.4.0]; 2.3.7
request	[...]	[...; 2.88.2]
restore-cursor	[...]	[...; 3.1.0]
run-async	2.3.0	2.4.1
semver	[...]	[...; 6.3.0]
signal-exit	[...]	[...; 3.0.3]
smart-buffer	4.0.2	4.1.0
strip-ansi	[...]	[...; 6.0.0]
supports-color	[...]; 5.5.0; 5.4.0	[...; 7.1.0]; 6.0.0; 5.5.0
yallist	[...]	[...; 3.1.1]
yargs-parser	[...]	[...; 13.1.2]
@types/color-name	-	1.1.1
ansi-colors	-	3.2.3
bluebird	-	3.7.2
ci-info	-	2.0.0
cli-spinners	-	2.3.0
cliui	-	5.0.0
clone	-	1.0.4
defaults	-	1.0.3
envinfo	-	7.5.1
flat	-	4.1.0
get-caller-file	-	2.0.5
is-interactive	-	1.0.0
node-environment-flags	-	1.0.5
object-inspect	-	1.7.0
object.getownpropertydescri…	-	2.1.0
ora	-	4.0.4
request-promise-core	-	1.1.3
request-promise	-	4.2.5
require-directory	-	2.1.1
require-main-filename	-	2.0.0
set-blocking	-	2.0.0
stealthy-require	-	1.1.1
string.prototype.trimend	-	1.0.1
string.prototype.trimleft	-	2.1.2
string.prototype.trimright	-	2.1.2
string.prototype.trimstart	-	1.0.1
strip-json-comments	-	2.0.1
tough-cookie	-	2.5.0
wcwidth	-	1.0.1
which-module	-	2.0.0
wrap-ansi	-	5.1.0
y18n	-	4.0.0
yargs-unparser	-	1.6.0
yargs	-	13.3.2
defaults	-	1.0.3
envinfo	-	7.5.1
flat	-	4.1.0
get-caller-file	-	2.0.5
is-interactive	-	1.0.0
node-environment-flags	-	1.0.5
object-inspect	-	1.7.0
object.getownpropertydescri…	-	2.1.0
ora	-	4.0.4
request-promise-core	-	1.1.3
request-promise	-	4.2.5
require-directory	-	2.1.1
require-main-filename	-	2.0.0
set-blocking	-	2.0.0
stealthy-require	-	1.1.1
string.prototype.trimend	-	1.0.1
string.prototype.trimleft	-	2.1.2
string.prototype.trimright	-	2.1.2
string.prototype.trimstart	-	1.0.1
strip-json-comments	-	2.0.1
tough-cookie	-	2.5.0
wcwidth	-	1.0.1
which-module	-	2.0.0
wrap-ansi	-	5.1.0
y18n	-	4.0.0
yargs-unparser	-	1.6.0
yargs	-	13.3.2

Edit: Tried to make it readable by macro-ing it into tsv, with limited success.

Here's a spreadsheet since that was less work to format:
https://docs.google.com/spreadsheets/d/1hA6DA_kWDynzziNOduKxirq_2vbK0jd9...

I don't think yarn has a built in command for only updating security releases. To make our lives easier, should we just run yarn upgrade and update all packages within the version constrains we've set? If there's something breaking, we can update the version constrain to not update that package until we have resolved the issue.

Started implementing based on #15. After this there's only two cases to handle on 8.9.x, and one on 9.0.x.

The only remaining package to update is yargs. We should probably open a follow-up for that since it seems like updating it might require changes to stylelint-no-browser-hacks.

Opened #3144854: Remove stylelint-no-browser-hacks as a follow-up.

In order to verify the patches in #16 I ran yarn upgrade && yarn build.

Note if I run yarn build on HEAD of either 8.9.x on 9.0.x there are no changes.

9.x review

There's a difference after running yarn build

diff --git a/core/misc/polyfills/array.find.es6.js b/core/misc/polyfills/array.find.es6.js
index 1dc9e3055a..e24f3a3c68 100644
--- a/core/misc/polyfills/array.find.es6.js
+++ b/core/misc/polyfills/array.find.es6.js
@@ -12,7 +12,7 @@
  */
 if (!Array.prototype.find) {
   Object.defineProperty(Array.prototype, 'find', {
-    value: function(predicate) {
+    value: function (predicate) {
       // 1. Let O be ? ToObject(this value).
       if (this == null) {
         throw TypeError('"this" is null or not defined');
diff --git a/core/misc/polyfills/object.assign.es6.js b/core/misc/polyfills/object.assign.es6.js
index 1dbb8bb5e9..399bdb8c35 100644
--- a/core/misc/polyfills/object.assign.es6.js
+++ b/core/misc/polyfills/object.assign.es6.js
@@ -12,9 +12,8 @@
  */
 if (typeof Object.assign !== 'function') {
   // Must be writable: true, enumerable: false, configurable: true
-  Object.defineProperty(Object, 'assign', {
-    value: function assign(target, varArgs) {
-      // .length of function is 2
+  Object.defineProperty(Object, "assign", {
+    value: function assign(target, varArgs) { // .length of function is 2
       'use strict';
       if (target === null || target === undefined) {
         throw new TypeError('Cannot convert undefined or null to object');
@@ -37,6 +36,6 @@ if (typeof Object.assign !== 'function') {
       return to;
     },
     writable: true,
-    configurable: true,
+    configurable: true
   });
 }
diff --git a/core/misc/polyfills/object.assign.js b/core/misc/polyfills/object.assign.js
index 265eb629a3..289c59ad31 100644
--- a/core/misc/polyfills/object.assign.js
+++ b/core/misc/polyfills/object.assign.js
@@ -6,7 +6,7 @@
 **/
 
 if (typeof Object.assign !== 'function') {
-  Object.defineProperty(Object, 'assign', {
+  Object.defineProperty(Object, "assign", {
     value: function assign(target, varArgs) {
       'use strict';
 

8.9.x review

I tried to do the same updates locally and found that I couldn't repeat the 8.9.x one. And I get errors when I try to commit it:

/Users/alex/dev/drupal/core/misc/ajax.es6.js
  103:34  error  Replace `⏎········'An·AJAX·HTTP·error·occurred.',⏎······)}\n${Drupal.t('HTTP·Result·Code:·!status',` with `'An·AJAX·HTTP·error·occurred.')}\n${Drupal.t(⏎········'HTTP·Result·Code:·!status',⏎·······`  prettier/prettier
  106:1   error  Replace `········` with `··········`                                                                                                                                                                    prettier/prettier
  107:7   error  Replace `}` with `··},⏎······`                                                                                                                                                                          prettier/prettier

✖ 3 problems (3 errors, 0 warnings)
  3 errors, 0 warnings potentially fixable with the `--fix` option.

yarn run v1.19.1
$ cross-env BABEL_ENV=legacy node ./scripts/js/babel-es6-build.js --check --file /Users/alex/dev/drupal/core/modules/quickedit/js/models/EntityModel.es6.js
[17:41:53] '/Users/alex/dev/drupal/core/modules/quickedit/js/models/EntityModel.es6.js' is being checked.
✨  Done in 0.53s.

/Users/alex/dev/drupal/core/modules/quickedit/js/models/EntityModel.es6.js
  246:32  error  Insert `⏎·············`  prettier/prettier

✖ 1 problem (1 error, 0 warnings)
  1 error, 0 warnings potentially fixable with the `--fix` option.

The yarn.lock file is exactly the same. This happens with Yarn 1.19.1 and Yarn 1.22.4

Looking at @alexpott's review in #19

1. For 9.0.x I don't get any changes after yarn build

   I am using yarn version 1.22.4.

   yarn upgrade
   does get 1 change to is-callable https://classic.yarnpkg.com/en/package/is-callable
   Because is-callable 1.2.0 was released today. No uploading patch because I think if we did every time somebody reviewed there would be a new update.

2. For me for 8.9.x
   running

   yarn upgrade
   yarn build
   

   The only difference still in is-callable
   and I don't get an error with
   BABEL_ENV=legacy node ./scripts/js/babel-es6-build.js --check --file /Users/ted.bowman/sites/d8/core/modules/quickedit/js/models/EntityModel.es6.js

Hmmm... I can't explain this. I've tried multiple versions of node (12 & 14) and yarn and still end up with differences.

Actually - yeah upgrading node from 12 to 14 fixed what I was seeing. I now can apply #16 and run yarn build and nothing changes.

Sorry for the noise. Given that @tedbow has verified @lauriii's work and now I have I think we're good to go here.

tried #16 for 9.x and same as tedbow. No diff in the JS, after a yarn build I have is-callable updated, electron-to-chromium updated.

yarn 1.22.4
node 12.17.0

so yeah RTBC+1

I postponed #3143289: Upgrade babel dependencies (yarn run build:js broken) on this. I think everyone from that issue should also get credit here

Committed and pushed 045a00c129 to 9.1.x and ac4f426e32 to 9.0.x. Thanks!

Committed 0aea9b9 and pushed to 8.9.x. Thanks!

Fwiw on 8.9.x you need to run yarn run lint:core-js-passing --fix

diff --git a/core/misc/ajax.es6.js b/core/misc/ajax.es6.js
index 26a551fcfd..6eeddf3df7 100644
--- a/core/misc/ajax.es6.js
+++ b/core/misc/ajax.es6.js
@@ -100,11 +100,12 @@
     let statusText;
     let responseText;
     if (xmlhttp.status) {
-      statusCode = `\n${Drupal.t(
-        'An AJAX HTTP error occurred.',
-      )}\n${Drupal.t('HTTP Result Code: !status', {
-        '!status': xmlhttp.status,
-      })}`;
+      statusCode = `\n${Drupal.t('An AJAX HTTP error occurred.')}\n${Drupal.t(
+        'HTTP Result Code: !status',
+        {
+          '!status': xmlhttp.status,
+        },
+      )}`;
     } else {
       statusCode = `\n${Drupal.t(
         'An AJAX HTTP request terminated abnormally.',
diff --git a/core/modules/quickedit/js/models/EntityModel.es6.js b/core/modules/quickedit/js/models/EntityModel.es6.js
index 8c92aeb084..2984425318 100644
--- a/core/modules/quickedit/js/models/EntityModel.es6.js
+++ b/core/modules/quickedit/js/models/EntityModel.es6.js
@@ -243,7 +243,8 @@
             //   proceed to set the fields to candidate state.
             if (
               (changedFields.length || this.get('fieldsInTempStore').length) &&
-              !options.saved && !options.confirmed
+              !options.saved &&
+              !options.confirmed
             ) {
               // Cancel deactivation until the user confirms save or discard.
               this.set('state', 'opened', { confirming: true });

Fixed the js coding standards on commit.

Confirmed that 8.9.x and 9.0.x can run yarn lint:css && yarn run lint:core-js-passing wthout any fails.

This is also supposed to address 8.8.x sec issues.

Oops, I thought there was an 8.8.x patch earlier on the issue but there wasn't.

I tried to use yarn upgrade-interactive to do a minimal upgrade to fix the 299!!! warnings from yarn audit. Was not possible. So did yarn upgrade - that means we're now down to 2 audit messages about yargs-parser - same as 8.9.x and 9.0.x.

yarn upgrade
yarn build
yarn lint:core-js-passing --fix

And checked yarn lint:css is not reporting any fails.

Tested on 8.8, running the commands no diff on my end with the patch #35.

Committed a2c5f7b and pushed to 8.8.x. Thanks!