Add ES6 to ES5 build process

The process strips all comments so the filesize is not impacted by our jsdoc. The source map prevent this from being an issue while debugging.

When logged in on the homepage this saves 50KB of gzipped javascript (which ends up as 200KB uncompressed). Pretty significant.

1. we need NODEJS script instead of bash script to max the compatible with Windows platform.
2. add a watcher for development. we need not reload them all. (Default to ES6 in development would help for performance. Generating SourceMap is taken times.)
3. Personally, I disagreed the compiled scripts provided by Patcher. It makes the patch harder to review online (& offline also actually). It's also not safe for our codebase. A little NODEJS config in their computer system could make a big diff :)

1. Agreed, bash was easier for a first patch. If you know/want to take of making the js script, feel free :) I always use bash for this type of things
2. Agreed, phpstorm can be configured for that.
3. We have to have the compiled ES5 version in the git repository. The testbot can't compile and commit the js. If we need to split patches to make the review easy that's fine with me.

If all developers (especially PHP) needs to have nodejs installed to compile the js, this whole issue is dead. Maybe have the js compiled by the core committer jst before commit? I don't think they'll like another step to get something comitted.

Maybe have the js compiled by the core committer jst before commit? I don't think they'll like another step to get something comitted.

Safety is my real concern. After ES6, it implied not to review (each line of) ES5 version. In some cases, the Babel will generate ugly code that we can't review it easily. We also can't verify our reviewers doing the hash comparison between files.

Ideally, adding a hook in GIT server to reject the ES5 version in patch and re-compiling it with D.org automatically.

--

There's one more thing I just thought....

Do we think Drupal NON-CORE Developers may read the ES5 version?

If Yes, can we strip the comments?
If No, why don't minify it also?

I see what you mean. It should be fine though, in the package.json we specify the version of our deps as well as any babel configuration so the result should be the same everywhere. That's a good point though, would help to have something take care of it consistently

We could definitely minify it at the same time. For now we didn't want to change all the libraries.yml files (need to add {minified: true}) so we left that for later.

Actually, I'm worried the modified configs or virus from patchers side. We should not specify the NODEJS & NPM version. Generally speaking, there're no compatible issues with the versions. In Windows, it's also unable (not easy) to use nvm. (I'm still running NodeJS 5.x, hehe)

I don't know how well the latest gaze is. chokidar is a better choice.
(Maybe split watcher to a standalone issue?)

One last thing..

.es6.js or .es6

.es6 is well-supported in the most popular editor also. And a little benefit in PHPStorm:

Initially made for brunch (an ultra-swift web app build tool), it is now used in gulp, karma, PM2, browserify, webpack, BrowserSync, Microsoft's Visual Studio Code, and many others. It has proven itself in production environments.
https://github.com/paulmillr/chokidar

chokidar is more widely used. In the old day, gaze has performance issues.

Pretty sure frontend developers will use BrowserSync. Webpack or Browserify good for project level development. If we go with Mocha test engine, Karma always our friend to let it run on real devices.

- Changed: babel.transformFile does not write SourceMap to file. I converted it to babel-cli. (the simplest way to apply sourcemap)
- Changed: Default to `ignoreInitial: true`. We need not compile the full source every time.
- Changed: Using chokidar `ready` event to tell process monitor states.
- Changed: Changed the build command to `build:js` & `watch:js` (Just prepare for SCSS..etc one day..)
- Added: `log` logging changes
- Added: Add back the build-javascript.sh (It's more like only used once in our migration.)

Thanks All.

Does what it should. We'll make the patch with the file changes in another issue and keep this one only about tooling. We can always tweak options later.

@droplet: looks like it's ok but can you confirm this works on windows also?

+++ b/core/scripts/migrate-js.sh
index 0000000..65d54a2
--- /dev/null

--- /dev/null
+++ b/package.json

What about existing projects? Shouldn't this go in ./core - is there anything like the merge plugin in composer?

@nod_,

@droplet: looks like it's ok but can you confirm this works on windows also?

Yup, that should work. I've developed it in Windows.

nothing like merge plugin for packages.json. Existing projects may have package.json but I'm pretty sure it won't be in the root, might be in their theme or a module or even the libraries folder.

I guess we could put it in core/ but then it's harder for regular people to use it for their stuff.

started the change record

per alexpott request moved package.json in core subfolder. Adding it in the root means lots of documentation changes.

Looks like this still needs windows testing? There are at least two core developers who use windows and write js that I know of.

Before commit, I have a general question.

How possible to keep these files up-to-date (marked as @internal API..etc)?

JS world has very short release cycles. And there're many new cool stuff coming out every week or so, developers will adopt to use it immediately. It's very diff to PHP and most languages. And I believed this commit will be affected how the themers and Drupal educators on any frontend workflow also. We love Productivity way and won't want to follow/learn something old-fashioned. (and it may stopping great frontend developers contribute to CORE)

Let's said today, here's a new rockstar Yarn from Facebook. Luckily, this is seamless. But if it's not, how possible to make a change to support it? Timing is important.

Looking back to the Frontend issues, here're two 4 years old issues. How can we stop it and get it done within 1 Drupal minor (major) release cycles? Is it possible? What can we learn from it?

#1310698: Add a CSS preprocessor library to core
#1778828: [policy, no patch] Update JS coding standards

I think we're safe and we can do whatever we want as long as scripts declared in libraries.yml files are in git and can be run as-is in all supported browsers. We're not actively encouraging people to use this for contrib yet

We do need a way to tag it @internal or something, maybe add a explicit text in package.json description?

I think adding an @internal comment to the files is a really good idea. The lack of support for comments in json is annoying.

Added a bunch on @internal. Not sure it makes a ton of sense for bash script but anyway. No changes to package.json, we already have the private: true key here. Not totally what we mean but good enough for now.

Renamed the files to be a bit more meaningful.

Interdiff comes out useless, only comment changes and file name changes.

@nod I think the @internal should explain why. Something like "This file is part of the core javascript build process and is only meant to be used as part of that."

+++ b/core/package.json
@@ -0,0 +1,21 @@
+  "version": "8.3.0",

Should we remove the version entry?

Yes, that works. It's not supposed to be published anywhere and there are no errors when running any npm run command or npm install.

(we should open a follow-up to commit the yarn.lock file too).

So can anyone answer #24?

Shouldn't let Windows Review blocking the progress. Dive into babel-cli source code. Adding sourcemap is simpler than I thought.
https://github.com/babel/babel/blob/v6.17.0/packages/babel-cli/src/babel...

Go back to @drpal code and adding improve. It's much faster in compile actually!!!

It should be 100% Windows compatible then.

Thanks Yarn, speed up testing & patching

Removed the stray active-link.js.map file from previous patch.

Given that all of this is intended for core development workflow only and is non-runtime I'm going to commit to unblock the far larger amount of work in converting to es6. ANy windows problems or other issues we can work out along the way.

Well I would apart from eslint revealed...

/Volumes/devdisk/dev/drupal/core/scripts/js/babel-es6-watch.js
  19:3  error  Unexpected console statement  no-console

✖ 1 problem (1 error, 0 warnings)

Bit of an egg in the face moment :D

Committed 46c6327 and pushed to 8.3.x. Thanks!

I think we should actually publish the CR once this is necessary.

Speed up compile files also. Just realized we needed many re-builds for testing on other ES6 issues before ES5 in GIT.
#2818409: Convert Babel ES6 Compile script to NODE Script

I know it's late since this was closed 6 months ago, but regarding Chokidar, we had issues with its use of fsevents on macOS, causing constant disk scans (on webpack meteor projects, not Drupal, though).

@fgm,

It's never late. I don't think we have packages frozen problem in our build script. We able to swap it when it has a problem I think.

Can you post a package or something so we able to test? It's better reported to upstream/with rel also, so we able to track and understand more the background info :)

@droplet

• https://github.com/thereactivestack-legacy/meteor-webpack/issues/67
• https://forums.meteor.com/t/meteor-constantly-polling-filesystem-120-loa...