We don't have an helpful way of structuring JS in Drupal core. We have Drupal.behaviors and drupal_add_js. This makes code reuse hard and it's an issue when a script has dependencies. Usually you have to play with the weight option of drupal_add_js to make sure the script is loaded at the right place. This is not a good way to manage dependencies. See at the end what this means for maintainers and contrib.

Using AMD would allow us and contrib to rely on an open and well tested specification https://github.com/amdjs/amdjs-api/wiki/AMD. Several implementations exists, we/contrib won't be locked in with a particular library. Having pluggable JS processing will make it possible if not easy to change the library implementing AMD if someone don't like what is used by core.

AMD loaders have scripts to bundle all the dependencies into one single file removing the problem of having a lot of small files to load. This might need some work since most of them work with node.js/rhino and not PHP.

Having an AMD loader means we'll be loading a 5kb-ish (and in some cases, much less) file to bootstrap the loading process of the rest of the files async. This reduces the time it takes to parse and load the page by a very significant %. Loading everything on page load is also an option if there is a need to support that.

  • The AMD spec has options making some level of sandboxing available (it's possible to use several versions on jQuery on the same page for example).
  • Dependencies can be overriten by config to point to another file, providing a clean way to do the Drupal.behaviors monkey-patching we currently have.
  • Will replace most if not all of Drupal.settings by targeting settings for specific JS modules (or not).
  • There is a way to turn non-AMD library into AMD modules to help with the transition. It's the use! plugin.

Drupal needs to be more involved in the way JS is managed to make JS fast by default for core and especially for contrib. AMD is the only solution we have right now to have Javascript modules that are not bound to a specific JS library or implementation, only to a spec.

Some examples

This is what it looks like:
MyModule.js

define(["jquery", "drupal"], function ($, Drupal) {
  return {
    "run": function () {
      /* do something */
    }
  };
});

SomeOtherFile.js

// using the previous module somewhere else
require(["MyModule"], function (MyModule) {
  MyModule.run();
});

You can see that there is no need to file-level closure anymore.

Using an AMD-compatible loader makes on-demand loading easy and could look something like this:

require(['require', 'jquery'], function (require, $) {

  $('#ajax-link').click(function (e) {
    e.preventDefault();
    var href = this.href;
    
    require(['ui.modal'], function (modal) {
      modal.open(href);
    });
    
  });
  
});

This particular example loads the modal component and all it's dependencies (it can be CSS, JS, JSON or some TXT file for templating) when the link is clicked, not on page load.

Some slides that can help grasp the issue: http://unscriptable.com/code/Using-AMD-loaders/

Changes for Contrib

This will make contrib life easier.

  • Instead of declaring a library on the PHP side all the dependencies will be declared inside the JS file.
  • There is no more need for file-level closures.
  • There will be no need to define a library but there will be a need to define a path to a module, to declare that the module "drupal" refers to the file at "core/misc/drupal.js". We'll have to introduce a hook in PHP to make that work.
  • There is an automatic way to transform non-AMD js files into AMD modules, see the use! plugin linked to earlier.
  • There are ways to override dependencies of other modules with config options.
  • You won't have to play with the weight option to make you JS execute at the right time, just declare a dependency in your module and you code will be executed after the dependency has loaded.

Example of contrib js:

(function ($, Drupal) {

  Drupal.behaviors.myCustomBehavior = {
    attach: function (context, settings) {  
      
    },
    detach: function (context, settings, trigger) {
    
    }
  };
  
}(jQuery, Drupal));

What it will look like with AMD:

define(["jquery", "drupal"], function ($, Drupal) {

  Drupal.behaviors.myCustomBehavior = {
    attach: function (context, settings) {
            
    },
    detach: function (context, settings, trigger) {
    
    }
  };

});

You're replacing the closure with a function. That's about the only change it will involve on your javascript files.

This change is a proposal to use AMD as it was intended. This is not another NIH issue. We chose a JS loader implementing AMD among the good number of already existing/working/having a test suite AMD loaders, and use that.

Here is the sandbox: https://drupal.org/sandbox/nod_/1545078

The current state of things

The sandbox is working and all JS is turned into AMD. like mfer noticed, there are a couple of non-Drupal JS that has been modified to declare them as AMD modules. Aggregation is not possible yet. Currently using 10 js files will load 10 js files, that's no good.

TODO

  • Tweak/extend the library hook to have enough information about a non-AMD JS library to use! them properly with the loader.
  • Once we have this working, remove the AMD wrapping of non-Drupal libs (fabtarstic, cookie.jquery, form.jquery, etc.).
  • Look up some way to minimize as much as possible the work non-AMD js will have to do to be used properly (ideally to a point where a script line in an info file would suffice).
  • Work on making aggregation possible. So we have a simple PHP version of r.js that does "on the fly" optimization. See RobLoach link in #7 for a possible starting point.
CommentFileSizeAuthor
#40 amd.patch73 KBSebCorbin
#14 core-js-AMD-1542344-14.patch65.21 KBnod_
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Ralt’s picture

I can only +1 this. Having AMD in Drupal would definitely be a huge win.

Now, where do we get started?

nod_’s picture

The main one is here #1033392: Script loader support in core (LABjs etc.) and links to other relevant issues.

For now I'd like to get feedback on the proposal and see if I'm missing something. I'll post something in the core group to get some attention.

nod_’s picture

Issue summary: View changes

use! plugin

nod_’s picture

Issue summary: View changes

info for contrib

mfer’s picture

I have a handful of questions/thoughts on this topic.

  1. There are alternatives to AMD we should consider. For example, some that use lazy evaluation. Not everyone agrees AMD is the answer we should use. Any thoughts on this?
  2. Loading all the JS in one file isn't efficient in newer browsers due to how they handle networking, async fetching (not execution), tcp slow-start, etc. At velocity 2011 there was a presentation that showed how script loaders could be slower than native browser. Drupal currently aggregates into several files per region. Reverting to a single file would be a performance slow down. Any thoughts on how to handle this?
  3. In practice a lot of Drupal modules just drop in external jQuery libraries (e.g., jcarousel). A lot of these aren't written for AMD. How would these be handled under an AMD centered approach?
  4. Similar to the previous one, a lot of Drupal code that integrates these libraries for slideshows, carousels, and other display options are written by non-JavaScript develops. They drop in a library, use some tutorial or limited documentation, and get it up and running. How would this new setup be useful to them and/or taught to them without driving them away?

My largest concern with switching to AMD is the developer experience and how this would raise the bar to entry for many of the people who currently drop in JavaScript. Any thoughts on how we can address these non-technical issues?

mfer’s picture

Issue summary: View changes

ortho

RobLoach’s picture

Definitely would love to have a modular system for our JavaScript management. Would be neat if we could scan the files and find the dependencies for hook_library, or something.

nod_’s picture

  1. I'll refer you to the answer from James Burke: http://tagneto.blogspot.fr/2012/01/reply-to-tom-on-amd.html. To have the ability to make and manage JS modules it's either using AMD or using YUI. And YUI comes with strings attached. I know there are others but this is what really works right now.
  2. We're going to have some kind of build step, we just need to take that into account. Shouldn't be a big problem to split the dependencies in several files. Also we can put scripts with the loader script so that it doesn't have to fetch them.
  3. see the updated issue, there is a part for contrib and maintainers.
  4. same

My guess is doc will be enough, I mean a lot of people don't get why we need closures, at least now it'll make a bit more sense.

nod_’s picture

@Rob loach, that's what the build tool is for, RequireJS has r.js that runs on node/rhino we'll need to find/make a PHP one, then we'll have the dependency tree and we'll be able to split things up to account for mfer comments.

RobLoach’s picture

RequireJS integration for Symfony2 is interesting, and adds some RequireJS compilation to Assetic. Pretty interesting, but not sure it's the correct route for us. Still would love to work with Assetic as we'd gain a huge amount of developers.

mfer’s picture

@nod_ In this whole thing I find one point that makes me see a potential problem. It's where you said "My guess is doc will be enough". Let me explain.

There is a certain group of devs who aren't experts or heavy devs. They are people who build sites for others, like restaurants. Instead of being expert devs they dabble in a lot of things including running a business, client interaction, understanding a clients needs, dev (and sometimes design), and more. There are a lot of Drupal devs like this. It's the very long tail of developers (and I know quite a number of people in this group).

When we typically make a change like this and we document it in the upgrade path I hear lots of complaining and foul language while devs struggle through trying to make the change. It doesn't matter if the change is a good one for the community it's a matter of people learning something new that doesn't necessarily translate to meeting their needs or that of their clients. And with Drupal this happens every few years in major ways.

If we document it like we typically do this will not be sufficient. It won't teach them about AMD (what it is, how it works, why people should do it, etc). It will raise the barrier to entry for updating Drupal versions and annoy developers. When I ask for a solution here I'm looking for something to ease the burden on the developers in this situation.

nod_’s picture

Yes I understand, our JS doc is not great so it'll need to be improved a lot and quite a lot of material should be explained.

Also the use! plugin turning non-amd script into AMD is something that James Burke is considering for inclusion in requirejs, so that's very much on the radar to help people make the transition.

It means that providing the script dependencies are defined in PHP in some hook (that people should already be used to) we could be turning a plain file (without the closure and direcly using $) into an AMD. The developer won't notice, it could actually feel better than before.

Now i'm afraid this will turn into abuse and people will write sloppy code just because they can. But if push come to shove, it's possible.

Does something like this sounds better?

mfer’s picture

@_nod if we make it difficult for people to write sloppy code we will make it difficult for many current drupal devs to continue to use drupal. Instead of making it difficult to write sloppy code how about something that makes it easier for them to output good code? The subtle difference is important and I'm not sure what that is.

nod_’s picture

That's an important goal but I'm not quite ready to add a drupalism on top of AMD to achieve that. That is, until we tried all the other available solutions.

I'm not sure what the right balance is either but I'm willing to get my hands dirty and find out. If others feels the same, please let yourselves known :)

cosmicdreams’s picture

To see how far we can take this idea and how it will look like for module developers (as that is what the purpose of this effort should be targetted, IMO) how do we get started?

What would be a good "base case". A full implementation for a subset of the functionality that would demonstrate what we're talking about.

_nod can you setup a sandbox to test this idea out and provide the proper access to others so we can help in the effort?

nod_’s picture

yes, working on it. Will be up soon.

also, just forget about the underscore in my pseudo people. That'll be simpler.

nod_’s picture

Issue summary: View changes

add config and order

nod_’s picture

FileSize
65.21 KB

Actually a patch will do to show what's going on. Core is all AMD and working just like before, except dashboard couldn't be bothered to make all the jquery ui module declarations (like it's done in jquery.drupal.js).

Now that'll need a few things on the PHP side to make overriding easier but that's the idea. I'm not putting as NR since there is a lot of cruft left, need to remove quite a number of things that are not needed anymore and aggregation is broken now. I'll make a sandbox out of that.

(edit) sandbox is up: https://drupal.org/sandbox/nod_/1545078

(edit2) patch outdated, check the sandbox for up to date code. Principle is the same but moved a lot of things out of drupal_add_library in the sandbox.

mfer’s picture

I noticed that your patch alters external libraries we are including in Drupal such as bbq and farbtastic. Is the expectation going to be that libraries used by code need to be altered and not just included?

nod_’s picture

Well at the same time I'm trying to get all that out of core, you could say I'm a little bias.

But no, like I said, this will not be necessary once everything is in place. It's just way easier to do it like that for now.

Actually for this kind of libs, the people maintaining the AMD spec keep forks of AMD-ified libraries. So there is choice here.

cosmicdreams’s picture

Noteable:

http://blog.jquery.com/2011/11/03/jquery-1-7-released/
https://github.com/amdjs/amdjs-api/wiki/jQuery-and-AMD

Read the part where jquery now supports AMD (but everyone here probably already knew that)

mfer’s picture

@cosmicdreams This isn't about jquery or another specific JavaScript project. It's about a development with drupal use case. Right now a developer/site builder can go drop in an external JS library that works with jQuery (or even stands alone). Then in their own scripts file initialize it to run. It's fairly easy to do and lots of people who don't know JS do this all the time. The barrier to entry is low. When a change is made how do you keep that low barrier to entry.

If the answer is to get every library on the internet to drop in work with AMD that's a non-starter.

@nod_ Drupal is a product in addition to having an underlying framework. No matter how we slice it this is the case and it's not changing. Getting all these libs out of core is really unlikely. I won't bank on it or think of this issue being anything other than supporting the status quo of JS.

There are also way more libraries that Drupalers use without AMD forked versions. You can't really count on them.

If AMD is going to go in the low barrier to entry for JS inclusion needs to be maintained. Not for me or other JavaScript devs who might comment here. But for the long tail of Drupal users, site builders, and developers.

idflood’s picture

As stated in the first post, the best solution with non-AMD libraries is to load them with the requrie.js use! plugin. The amd incompatible libraries can stay untouched but the plugin require additional configuration that may be difficult to hide/automate.

Here is what the code would look like with use! plugin:

require.config({
  paths: {
    jquery: 'libs/jquery-1.7.2',
    jQueryUi: 'libs/jquery-ui/js/jquery-ui-1.9m6',
    'someAMDlibrary': 'libs/test.js',
    use: "libs/require/use",
    order: "libs/require/order"
  },
  use: {
    'jQueryUi': {
      deps: ['jquery'],
      attach: "jquery"
    }
  }
});

// then notice the required use! prefix for jqueryUi but not jquery or someAMDlibrary
require(['someAMDlibrary', 'jquery', 'use!jQueryUi'], function(someLibrary, $) {
  ...
}

This is clearly not ideal but still better than modifying libraries or requiring amd-ified versions of those. Moving to amd is a good thing but adding custom javascript should remain simple.

@nod_: Any idea of how to simplify the !use configuration step? And maybe you could add me to the sandbox project, I'm working on a "large" project based on require.js and the !use plugin. Hopefully I will be able to help push this forward.

JohnAlbin’s picture

Issue tags: +frontend performance

Tagging so it shows in the mobile issue queue

nod_’s picture

@idflood thanks for the offer, i will definitely take you up on that once a few other things have been ironed out, starting with #352951: Make JS & CSS Preprocessing Pluggable. It looks like requirejs 2 will be a better fit, it'll have use! built in as well as a way to target very specifically the dependencies you might want to replace. Both are really important for Drupal so if you'd like to help this issue moving along please consider working directly there: https://github.com/jrburke/requirejs/wiki/Requirejs-2.0-draft

The current state of things

The sandbox is working and all JS is turned into AMD. like mfer noticed, there are a couple of non-Drupal JS that has been modified to declare them as AMD modules. Aggregation is not possible yet. Currently using 10 js files will load 10 js files, that's no good.

TODO

  • Tweak/extend the library hook to have enough information about a non-AMD JS library to use! them properly with the loader.
  • Once we have this working, remove the AMD wrapping of non-Drupal libs (fabtarstic, cookie.jquery, form.jquery, etc.).
  • Look up some way to minimize as much as possible the work non-AMD js will have to do to be used properly (ideally to a point where a script line in an info file would suffice).
  • Work on making aggregation possible. So we have a simple PHP version of r.js that does "on the fly" optimization. See RobLoach link in #7 for a possible starting point.

I'm out of free time for a little while but please let me know if you want to get this going and need info. I'll make time.

nod_’s picture

Issue summary: View changes

Adding the sandbox link.

nod_’s picture

Very good news, Require JS 2.0 is out.

There is no plugin nor manual edit needed to wrap non-AMD and load them like any other AMD script.
There is a module specific configuration (this means the end of Drupal.settings).
Better error handling when loading dependencies.
Delayed module evaluation (only execute the init function of a module when there is an explicit require that needs it (or is one of the dependencies of a "require"d script).
And more!

I'll update the sandbox once I get a few of those things integrated. If you'd like to help I can give access to the sandbox.

cosmicdreams’s picture

@nod_ may i have access to the sandbox?

nod_’s picture

Done. idflood, you're still interested?

idflood’s picture

@nod_: Oh yes. I'm not sure if I will have much free time to work on this the next two weeks but you can still add me.

nod_’s picture

Done. ping me on irc/pm when you have time to start something.

mfer’s picture

@_nod (and whom ever else) what is your plan for performance testing this change? For core and what the use would be with some contrib in the loop?

nod_’s picture

https://drupal.org/sandbox/nod_/1498772 got a whole lot of data already, works really well.
If you want you can help get this thing rtbc :D #1568052: Navigation Timing It's been like 3 weeks without a proper review.

There is a lot of ui and tools code I removed from the module to speed up review. Will clean that up and add it back little by little once it's a full project. As it is now I can get the individual perfs of a "RAW", JS, CSS, JS+CSS page over 160+ pages of the default drupal admin.

mfer’s picture

@_nod What about the other parts of timing? For example, using a simulated network connection to test the network implications. You can't use a real network connection to do multiple tests easily due to variance in your connection. Instead using something like Charles Proxy to simulate network delay before and after a change. Then looking at a waterfall to understand the differences and what's going on.

The reason I ask about contrib with regard to this approach is because of how browsers only open 6 parallel requests to the same domain. If we load a lot of small JS files individually the network issues will become a factor in page loading. I'm curious to how this plays out when contrib starts adding a bunch of scripts to a page.

nod_’s picture

Well navigation timing is for spotting big issues, so that we know where to gear up and look closely with the tools you're thinking about, getting the waterfall and all. First things first.

Then again the waterfall shouldn't be very surprising (I've been looking at a lot for a couple of months now) and we'll have aggregation so no 20+ single js files to load on production websites and it will still be possible to load everything in script tags and not rely on script loading.

(edit) you can drop the underscore in my pseudo :)

mfer’s picture

@nod bah... sorry about the _ slips.

klonos’s picture

Wim Leers’s picture

RobLoach’s picture

Issue tags: +JavaScript

Somehow this didn't have the JavaScript tag.

Owen Barton’s picture

Another benefit of this, that I haven't seen noted yet, is that it enables some nice potential aggregation strategies because we can preload (by including in an aggregate) "likely to be needed" JS without actually executing it if it is not needed on the users current page.

Wim Leers’s picture

#35++++++++++++++++++++++++

It enables contrib modules that analyze web log statistics and decide on not only optimal bundling, but optimal preloading.

Owen Barton’s picture

gagarine’s picture

We can perhaps directly use the http://requirejs.org implementation

nod_’s picture

Check the sandbox, awesome stuff has been happening.

SebCorbin’s picture

Status: Active » Needs review
FileSize
73 KB

Fresh patch from the sandbox, just throwing this to see what the testbot is going to report

Status: Needs review » Needs work

The last submitted patch, amd.patch, failed testing.

nod_’s picture

Possible to split the patch in several ones?

First one with all the explicit jquery and drupal_add_library calls, then the one with RequireJS? it'll be easiser to review for other people and will have better chances of going in.

nod_’s picture

Possible to split the patch in several ones?

First one with all the explicit jquery dependencies and drupal_add_library calls, then the one with RequireJS? it'll be easiser to review for other people and will have better chances of going in.

Wim Leers’s picture

You'll want to update require.js to 2.0.5 because anything older will break in Safari 6.

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+  // Force drupal to be the first script (needed as

Is "drupal" a script? You mean drupal.js, right?j I think that's what it should say here then.

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+    if($group['type'] != 'file' || !isset($group['data'])) {

There should be a space after "if".

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+    // Add previously included libraries

Missing trailing period.

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+          $path = $files[0];

Shouldn't this one also be routed through file_create_url()?

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+          $config['paths'][$name] = str_replace('.js', '', $path);

A comment to explain why this is necessary would be helpful.

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+        if (!empty($library['shim'])) {
+          $config['shim'][$name] = $library['shim'];
+        }
+        elseif (empty($library['type']) || $library['type'] !== 'amd') {
+          foreach ($library['dependencies'] as $dep) {
+            $config['shim'][$name]['deps'][] = $dep[1];
           }

Another comment would be nice.

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+    // Allow altering configuration

Trailing period.

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+    drupal_alter('js_config', $config);

This will allow me to e.g. not load the html5shiv, or even don't load ANY JavaScript, correct?

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+        'data-main' => base_path() . 'core/misc/',

This is the root where require.js will go and look for JS files of Drupal core AFAICT.

Two concerns:
1) It should not be impossible to aggregate require.js with other JS.
2) What if we want to serve core's JS files from a CDN, with unique file URLs?

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+        'src' => base_path() . 'core/misc/require.js',

Needs to use file_create_url().

+++ b/core/includes/common.incundefined
@@ -3999,103 +3982,126 @@ function drupal_get_js($scope = 'header', $javascript = NULL, $skip_alter = FALS
+          // TODO remove that and use events instead.

s/TODO/@todo/

(This is a core convention I also only recently learned about.)

+++ b/core/includes/common.incundefined
@@ -4495,19 +4505,31 @@ function drupal_process_states(&$elements) {
+  // Return libraries
+  if(!$module && !$name) {

Trailing period, space after "if".

+++ b/core/includes/common.incundefined
@@ -4495,19 +4505,31 @@ function drupal_process_states(&$elements) {
+      // Remember which library we are treating

Trailing period. s/treating/processing/.

+++ b/core/includes/common.incundefined
@@ -4748,15 +4770,28 @@ function drupal_build_js_cache($files) {
+          // wrap it with a define() call

TP. (Trailing period.)

+++ b/core/includes/common.incundefined
@@ -4748,15 +4770,28 @@ function drupal_build_js_cache($files) {
+          // Retain this library to put the aggregated file path in the shim config later

TP

+++ b/core/modules/locale/locale.moduleundefined
@@ -430,6 +430,7 @@ function locale_library_info_alter(&$libraries, $module) {
+    // TODO transform to library

s/TODO/@todo/

+++ b/core/modules/system/system.moduleundefined
@@ -1212,6 +1212,19 @@ function _system_batch_theme() {
+  // Drupal specific javascript.

Drupal-specific JavaScript.

+++ b/core/modules/taxonomy/taxonomy.moduleundefined
@@ -1703,3 +1703,22 @@ function taxonomy_entity_query_alter($query) {
+  /**
+   * Implements hook_library_info().

Indentation issue?

nod_’s picture

Split the dependencies declarations to a new issue #1737148: Explicitly declare all JS dependencies, don't use drupal_add_js hopefully that'll make things easier to commit.

nod_’s picture

Issue summary: View changes

add status to summary

DamienMcKenna’s picture

Issue summary: View changes

Fixed a tiny typo ("durpal").

DamienMcKenna’s picture

Just to mention it, #35 is a pretty mind-blowing idea.

That said, would having a recommendation to enable gzip compression ("deflate" Apache HTTPd module) be sufficient for 8.0, or is there a requirement to have some form of aggregation available OOTB?

nod_’s picture

Status: Needs work » Active

not quite ready yet.

Wim Leers’s picture

#47: more explanation please? :)

nod_’s picture

the dependency has to go in first, then we have to make the aggregation work properly then we can review.

Wim Leers’s picture

Okay, so #1737148: Explicitly declare all JS dependencies, don't use drupal_add_js is now a dependency for this patch. I gave it a review :)

nod_’s picture

Just found out about this, nice example. https://github.com/requirejs/example-multipage-shim

Wim Leers’s picture

#51: Very interesting indeed :) What they achieve there is more or less what Facebook does; with the difference that Facebook also takes usage patterns into account.

RobLoach’s picture

effulgentsia’s picture

Why do we need to remove jQuery UI to make progress on this? We should assume that some contrib module somewhere will want to add a library that isn't compatible with AMD. Therefore, our AMD solution needs to allow for non-AMD-compatible libraries to exist.

RobW’s picture

How will this will interact with/ conflict with/ replace/ augment #1784774: Remove Assetic component from core?

Wim Leers’s picture

#55:

@wimleers @SebCorbin @jessebeach we should get the Assetic thing going before and implement AMD as a plugin /cc @seutje
— @nod_

Source: https://twitter.com/nod_/status/253904833084534784

Fabianx’s picture

Assetic is in #1784774: Remove Assetic component from core. How will this continue now? Just really curious.

nod_’s picture

We're working on Assetic right now, the work on AMD is held up by that since the current API is too poor to do anything 'clean'. Once we have assetic integrated we can go crazy :)

RobLoach’s picture

Issue tags: +Assetic

Although not directly related to Assetic, might be handy to have it in the queue.

Wim Leers’s picture

#1737148: Explicitly declare all JS dependencies, don't use drupal_add_js landed a long time ago. As per #56 and #58, this depends on Assetic, which won't happen in D8 anymore.

What's the plan for this issue?

nod_’s picture

It's probably going to end up being the basis for the issue "Use JS modules for JS architecture" once ES6 gets out. There'll be some discussions about it next week I believe.

I'd just leave it open. Nothing really actionable for D8 at this point we're not changing the loading system. I mean except if it turns out that we need to know when a ajax-loaded JS file has been loaded before executing code that depends on it (the concenr came up a few times in other issues, see how opening a modal link works for example). But short of that it's doomed to contrib/D9

Wim Leers’s picture

So… 8.x -> 9.x? :)

nod_’s picture

Version: 8.x-dev » 9.x-dev

The jury is still out on us needed to know if a dependency loaded (I think it'll be needed soon enough and that a contrib module providing it will be used pretty much everywhere).

but yeah d9

adnasa’s picture

Pushed to D9: I can't wait!
Please post back to this issue when you're ready!

adnasa’s picture

Issue summary: View changes

Fixed another occurrence of "durpal".

jibran’s picture

Status: Active » Needs review

40: amd.patch queued for re-testing.

rfay’s picture

Issue summary: View changes
Status: Needs review » Needs work

Sorry, can't be testing against 9.x. Please set this to 8.x if you want to test it.

rfay’s picture

Version: 9.x-dev » 8.x-dev

Changing to 8.x to try to recover testbot.

The last submitted patch, 40: amd.patch, failed testing.

jibran’s picture

Version: 8.x-dev » 9.x-dev

@rfay sorry for the inconvenience. I haven't check the issue version before hitting retest link. Back to D9 then.

LewisNyman’s picture

Issue tags: +frontend

So at the moment we require theme developers to create a libraries.yml file to add dependencies to their JS. Is it possible to use the AMD spec and parse JS files to build those dependancies instead of requiring a separate file?

Anonymous’s picture

+1 to the idea, I'd definitely back getting JS dependency management out of PHP land.

Re: #70, I can see what you're aiming for, but my understanding of this seems like over-engineering things a bit; why would we need the js dependencies in theme.libraries.yml at all if we have implemented AMD? I think we'd best wait and see where we are down the line.

nod_’s picture

@Lewis, we've been down that road during brainstorming. Short story, no. Has to be declared in PHP/YML so that Drupal doesn't need to come up with the deps. The way to see this is that our aggregation/processing system will generate the amd/commonjs definition.

@James, we need it for aggregation/processing. Otherwise unless you're using spdy or http2 It's a performance catastrophe.

cosmicdreams’s picture

From my reading of this issue it sounds like this isn't going to get done for Drupal 8.0.0. Should we postpone this and target to Drupal 8.1 ? Drupal 9?

nod_’s picture

see status already forr D9

Chi’s picture

frob’s picture

Is there any hope for getting this into contrib for 8?

nod_’s picture

yes, there is.

nod_’s picture

Status: Needs work » Closed (won't fix)

But for core: nope. If we use something it'll be straight up ES6 modules.

RobLoach’s picture

AMD is good, but it is limited to require.js. Node.js, and even ES6 to some extent, has adopted CommonJS.

UMD is the pattern to allow use of Globals, AMD (RequireJS), or CommonJS (Node.js). jqueryPluginCommonjs.js is the likely pattern we'd want to use.

ES6 modules are what we want in the future, when it's adopted across all browsers, but UMD is what we'd use if we want a moduled-design now.

frob’s picture

We really don't need to go into a AMD/Commonjs debate here.
Reminder that this is a closed issue. For debate, open a new issue.

As a reference for ES6 module syntax http://www.2ality.com/2014/09/es6-modules-final.html

nod_’s picture

I hate umd, it's really ugly. Makes sense for a library designed to be used anywhere, doesn't makes sense to standardize a whole codebase on it. We would chose one and write an adapter of some kind if another format is used.

And stuff like babel would allow us to use ES6 and transform it down to ES5.

RobLoach’s picture

jQuery 2 and 3 has adopted a factory method similar to UMD, which allows loading dependencies through Globals, CommonJS or RequireJS. While UMD can be ugly, it doesn't limit you to one tool-chain. We're using it on jQuery Once to allow its use across anything:
https://github.com/RobLoach/jquery-once/blob/master/jquery.once.js#L8-L32

When documented, it's not ugly, dependency declaration becomes clear, and you can use the JavaScript in RequireJS, BabelJS, through globals, anywhere.

Reminder that this is a closed issue. For debate, open a new issue.

Understand that, just sticking a few additional links in here. As a side note, ES2016 is going the CommonJS route with require() and exports. That's the way we should go.

Version: 9.x-dev » 9.0.x-dev

The 9.0.x branch will open for development soon, and the placeholder 9.x branch should no longer be used. Only issues that require a new major version should be filed against 9.0.x (for example, removing deprecated code or updating dependency major versions). New developments and disruptive changes that are allowed in a minor version should be filed against 8.9.x, and significant new features will be moved to 9.1.x at committer discretion. For more information see the Allowed changes during the Drupal 8 and 9 release cycles and the Drupal 9.0.0 release plan.

andypost’s picture

Issue tags: -JavaScript +JavaScript

As core moved to es6 probably now we need to render nomodule as default to script tags https://developer.mozilla.org/en-US/docs/Web/HTML/Element/script#module_...