Problem/Motivation

As it stands, Drupal 8 contrib modules that add a JS minifier would have to GUESS based on heuristics (file name, first 100 bytes, …) whether a JS file is already minified or not. The only reason it's not a problem in HEAD is because we still don't have a JS minifier!

(Noticed thanks to #2258313-24: Add license information to aggregated assets.)

Proposed resolution

There are valid reasons to have minified JS assets (see #1, #4, #7). In fact, we already have minified JS assets in core.

Therefore the solution is simple: make Drupal aware of minified JS assets, so that it doesn't have to guess, and doesn't have to reminify already minified JS.

This issue/patch proposes to add a minified flag for JS assets. This flag would default to false (i.e. Drupal assumes JS assets are unminified).

Remaining tasks

None.

User interface changes

None.

API changes

Introduced a minified flag for JS assets. This flag defaults to false (i.e. Drupal assumes JS assets are unminified).

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

RainbowArray’s picture

It seems strange to decide that themes are unable to use minification in their build process. Maybe D8 ships with a great minification system. What if something better becomes available in the next 3-5 years after D8 ships?

If core wants to ship with no unminified JS, that seems to make sense. But it should be possible for themes to add in minified JS. It's possible somebody downloads a library for use in a theme, and only a minified version is available.

There are others who know far more about this than me, but it seems like there could be use cases where it would be preferable to be able to declare that JS has already been minified.

Damien Tournoud’s picture

Is this really a problem? What bad would it do to re-minify something already minified?

Wim Leers’s picture

Issue summary: View changes

Cleaned up the issue summary.


#1:

  1. Maybe D8 ships with a great minification system. What if something better becomes available in the next 3-5 years after D8 ships?
    D8 already ships with an abstraction that allows you to replace the current JS minifier (which is: none) with any other one.
  2. I've never encountered a library for which you can only download an unminified version?
  3. What's the use case for themes wanting to include minified JS?
  4. I could see a possible rule "everything in core is unminified, contrib does what it wants" to be sensible. But I've yet to see a reason why you'd want, let alone need that?

(attiks has expressed his +1 for this comment via Twitter.)


#2: I agree with you. I don't think it's a problem at all. hass thinks it is — from #2258313-22: Add license information to aggregated assets: In case a file is already minified it is a waste of cpu to double minify them.. I do think that it's preferable to ship with unminified JS files for debugging purposes.

RainbowArray’s picture

Most libraries provide unminified js, but in theory is it possible that one wouldn't? Probably.

I think there may be cases where one is linking to an external library hosted on a CDN, but that shouldn't be affected by aggregation I would think.

A workflow that's becoming more common for frontend work is to use Grunt or Gulp to do build work before moving to production. Those commonly have minification as part of the default build.

Setting up D8 in a way that you can't have minified assets in your themes seems pretty pushy on the part of Drupal. Maybe Drupal has an awesome minification system, or you can plug in a minification system that works wonders. That may even be the best way to do things.

But I get leery of anything that limits what front end techniques can be used when who knows what the heck might happen during the lifecycle of Drupal 8, which could be 5-10 years.

Wim Leers’s picture

#4: I can see that. So perhaps this would be the appropriate policy then:

Drupal encourages you to give it unminified assets, since this makes it easy to toggle between the minified assets (for production) and unminified assets (for development and debugging). But you can feed it minified assets, Drupal just might minify it again, but it will continue to work just fine.

This is conceptually the simplest (Drupal assumes every asset is unminified), the least restrictive (you still can use minified assets if you want), and the least demanding (no need for a special flag on libraries that have minified assets).

What do you think?

RainbowArray’s picture

I have no problem with that.

If people are really concerned with minifying minified assets, I don't think it's the end of the world to tag an asset as already minified in libraries.yml. But probably would be bad if things broke if it wasn't tagged.

So maybe just minifying everything is the way to go.

hass’s picture

We have a lot of minified code in core... Just one example ckeditor. There are more...

nod_’s picture

The flag needs to be at the file level, not library level and I'd go with a "minified: true/false" on the file properties.

If we look at a higher level that means asset plugins would define and use their own option on files properties to know when to process it (or not). Or at least we should make sure we're thinking it through making a precedent with this.

We can't disallow the use of minified file in core/contrib so 2) is not an option.

From the other issue: double escaping. I had problems years ago with yui compressor but i'm pretty sure it's not a problem anymore. I'd like to avoid having processing already minified files but it's probably not the end of the world.

catch’s picture

This is a feature addition at worst, unless we're going to require this is set (which seems unlikely, it should just default one way or the other), so no need to be beta deadline.

catch’s picture

Also we require that external libraries used in core are minified, i.e. #1341792: [meta] Ship minified versions of external JavaScript libraries. The first and only time we've had unminified external libraries in core was this release cycle to make development easier, all previous releases used the minified versions throughout. So it's not a policy change to allow it if we already require it in core.

hass’s picture

EDIT: Forget it. Answered in the linked case I had not read yet.

sun’s picture

How reliable are .min.js file suffixes in 2014?

(…as well as /min/ subdirectories?)

if (strpos($pathname, '.min.') || strpos($pathname, '/min/')) {
  // Already minified; aggregate as-is and move to next.
  $aggregate .= file_get_contents($pathname);
  continue;
}

re: #10: I think we should commit both original sources and minified versions, if both variants are normally part of the upstream dist archive/package. Unless we find a smart logic like the above (which must be reliable), we should think about declaring variants in libraries.yml files.

Wim Leers’s picture

Title: [Policy, no patch] Do we allow minified JS assets to be committed? If we do, then we must add a flag to *.libraries.yml » Inform Drupal in *.libraries.yml via a new per-JS file "optimized" flag whether a JS file has been minified
Status: Active » Needs review
Issue tags: +JavaScript, +Needs tests
FileSize
4.27 KB

Unless we find a smart logic like the above (which must be reliable), we should think about declaring variants in libraries.yml files.

Agreed. But that's for a separate issue.


So, now that we have all this feedback, I think the following are clear:

  1. my proposal in the issue summary was flawed, because it proposed to add a asset library-level flag, while we need an asset-level flag (#8)
  2. there are valid use cases for committing minified JS (see #1, #4)
  3. we already have lots of minified JS in core (see #7), which made my issue title simply pretentious (Do we allow minified JS assets to be committed? […] ) — perhaps in the future we won't have minified JS in core, but it's not a reality today. And given that e.g. minifying CKEditor efficiently is a non-trivial task (it's 505 KB of compressed JS, which means megabytes of uncompressed JS), it seems like it will stay that way.

Therefore, I propose to simply add a new optimized flag to JS assets. ("optimized", not "minified", because that's in sync with AssetOptimizerInterface, but if people prefer minified adamantly, then I'll change it.)

What do you guys think about a patch like this?

Wim Leers’s picture

Note that #13 also uses this new flag to prevent double minification.

Status: Needs review » Needs work

The last submitted patch, 13: 2307717-13.patch, failed testing.

nod_’s picture

the problem with optimized is that it doesn't say what "optimized" means. So far optimized === minified but will that stay true? what if we have people putting asm'ed files? If we go with optimized we need to define it and say it's only about minification, in which case, "minified" is a better key to be using.

Wim Leers’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
FileSize
6.29 KB
6.68 KB

Alright, so let's change to minified. Done. Also added tests.

Any remaining concerns?

Wim Leers’s picture

Title: Inform Drupal in *.libraries.yml via a new per-JS file "optimized" flag whether a JS file has been minified » Inform Drupal in *.libraries.yml via a new per-JS file "minified" flag whether a JS file has been minified

Status: Needs review » Needs work

The last submitted patch, 17: 2307717-17.patch, failed testing.

RainbowArray’s picture

This looks nice and simple. +1 from me.

Wim Leers’s picture

Status: Needs work » Needs review
FileSize
1.4 KB
6.14 KB

Should be green now.

Thanks for the review, mdrummond!

RainbowArray’s picture

Status: Needs review » Needs work

Pretty minor doc fix needed. Somebody with more expertise than me might want to review the tests. Rest looks fine. Any thoughts on who may need to review this so we can RTBC?

+++ b/core/lib/Drupal/Core/Asset/LibraryDiscoveryParser.php
@@ -177,6 +177,11 @@ public function buildByExtension($extension) {
+          // Set the 'minified' flag is on JS file assets, default to FALSE.

Change to: Set the 'minified' flag on JS file assets, default to FALSE.

micbar’s picture

Status: Needs work » Needs review
FileSize
5.76 KB
735 bytes

Made the suggested changes.

moshe weitzman’s picture

Status: Needs review » Reviewed & tested by the community

Code and tests are good to go.

alexpott’s picture

Afaics there is absolutely no documentation anywhere about the format of libraries.yml file. Can someone file an issue to create some documentation? Could go in core.api.php

Also we need an issue summary update to document which solution we've go for and why.

Wim Leers’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

Afaics there is absolutely no documentation anywhere about the format of libraries.yml file. Can someone file an issue to create some documentation? Could go in core.api.php

Indeed :( Filed an issue: #2323895: [Meta] Document format/content of various YML files.

Also we need an issue summary update to document which solution we've go for and why.

Done.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Core/Asset/JsCollectionOptimizer.php
@@ -112,7 +112,13 @@ public function optimize(array $js_assets) {
+                // Optimize this JS file, but only if it's not yet minified.
+                if (isset($js_asset['minified']) && $js_asset['minified']) {
+                  $data .= file_get_contents($js_asset['data']);
+                }

The isset() here is unnecessary since we ensure it is present in LibraryDiscoveryParser

Wim Leers’s picture

Status: Needs work » Reviewed & tested by the community

#27: it is in fact necessary, because you can still use #attached without a library to attach JS assets:

$build['#attached']['js'][] = array('data' => drupal_get_path('module', 'user') . '/user.permissions.js' => array());

(Hopefully we can get to a point where we *only* have libraries, but whether that's feasible to achieve within this cycle remains to be seen.)

  • alexpott committed dd4fd1a on 8.0.x
    Issue #2307717 by Wim Leers, micbar: Fixed Inform Drupal in *.libraries....
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed dd4fd1a and pushed to 8.0.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.