This is aimed to solve #352951: Make JS & CSS Preprocessing Pluggable
As well as making it trivial for contrib to implement something like this #1310698: Add a CSS preprocessor library to core

Assetic is a symfony component written by Kris Wallsmith, one of the main symfony contributors.

This is used by several other big PHP projects and has a neat Twig integration.

It will also make life easier for the other initiatives by providing a sane way to interact with assets. The current process is pretty bad and complicated.

Files: 

Comments

DamienMcKenna’s picture

From the documentation:

The core provides the following filters in the Assetic\Filter namespace:

  • CoffeeScriptFilter: compiles CoffeeScript into Javascript
  • CssEmbedFilter: embeds image data in your stylesheets
  • CssImportFilter: inlines imported stylesheets
  • CssMinFilter: minifies CSS
  • CssRewriteFilter: fixes relative URLs in CSS assets when moving to a new URL
  • GoogleClosure\CompilerApiFilter: compiles Javascript using the Google Closure Compiler API
  • GoogleClosure\CompilerJarFilter: compiles Javascript using the Google Closure Compiler JAR
  • JpegoptimFilter: optimize your JPEGs
  • JpegtranFilter: optimize your JPEGs
  • LessFilter: parses LESS into CSS (using less.js with node.js)
  • LessphpFilter: parses LESS into CSS (using lessphp)
  • OptiPngFilter: optimize your PNGs
  • PackerFilter: compresses Javascript using Dean Edwards's Packer
  • PngoutFilter: optimize your PNGs
  • CompassFilter: Compass CSS authoring framework
  • Sass\SassFilter: parses SASS into CSS
  • Sass\ScssFilter: parses SCSS into CSS
  • SprocketsFilter: Sprockets Javascript dependency management
  • StylusFilter: parses STYL into CSS
  • Yui\CssCompressorFilter: compresses CSS using the YUI compressor
  • Yui\JsCompressorFilter: compresses Javascript using the YUI compressor

That's pretty darned nice!

Kiphaas7’s picture

So what would be the general approach to implementing this?

nod_’s picture

1) get the lib in core with composer and all
2) make drupal_add_js use Assetic to add stuff and get rid of all the processing that exists currently (aggregation is broken at this point).
3) make Assetic take care of the actual processing/aggregation.

I think. Tried and failed 1).

jessebeach’s picture

Did 1) fail for technical or social reasons? I mean, do you just need support to get this in place or technical input as well?

nod_’s picture

oh just because I have no idea how to use composer to add the Assetic lib to core. No politics or anything actually. I just fail at composer :p

cosmicdreams’s picture

In order to start building up a patch, we don't need to wait for core to include assetic. Just modify the composer.json and update.

In order to test this for core we could have a large patch, a patch with just the changes, and (if so desired) a patch with just the tests.

schmittjoh’s picture

Hey all,

Kris has asked me to chime in here on his behalf as his wife passed away yesterday. If you want to support him during these times, I think he would appreciate it: http://indiegogo.com/franyaberkman.

Unfortunately, I have never used Drupal, so I am totally unfamiliar with how your Asset management works at the moment. So, I will just give you a quick introduction how Assetic works, and I can answer any questions that you might have along the way.

As you mentioned, Assetic has a very good integration with Twig. Currently, we rely on static analysis to parse your templates and extract all assets that are defined in them. Then, we apply the different filters that you have configured (can be different per asset), and dump them in a location you have specified, or for the development environment we also offer the option to route asset requests through a controller action which compiles the assets on-the-fly.

Also, it is possible to compile the same asset for different languages (if you have an i18n page), or to compile optimized assets for each browser. Assetic would then choose the correct asset to include at runtime, and for example only send the German version of your asset to German users.

I hope that is good for a start, if you have any questions, keep them coming.

nod_’s picture

Thanks for reaching out to us, I'm deeply sorry about the situation. I hope he has the support he needs to have and my thoughts goes to him and his kids.

I'm uncomfortable going from this to lowly technical questions but here goes:

I've looked at the API and it seems there is always a need to dump the files somewhere, is that correct? I'm not sure we can really support the "put the php script as the script src and dump from there" workflow but we can try it out. The "dump the file and serve it" is totally fine, that's what we're already doing.

The l18n thing does looks nice, we have our own way of handling it so we might need to look into that and see how we can approach the problem best.

Thanks a lot for reaching out to us, at this point we're mostly running into drupal-related issues more than Assetic ones but once we're up to speed we'll be definitely need you input on how we do things.

Thanks, and take care.

mcjim’s picture

Assigned: Unassigned » mcjim

I've been given time to work on this issue by my employer (yay!): I'll see how far I can get.

Kiphaas7’s picture

mcjim: yay indeed!!!

mcjim’s picture

Slowly (very!) getting my head around Assetic.
Here's some example code I've got working which gathers together CSS files located in one place and servers them as one file from another.
I'm still very far from a useful patch, but learning lots :-)

  $factory = new AssetFactory(DRUPAL_ROOT . '/sites/default/files/css');
  $system_all = $factory->createAsset(
    array(
      // This isn't actually a useful example, as you'd never do this.
      DRUPAL_ROOT . '/core/modules/system/*.css'
    ),
    array(
      // Filters go here.
    ),
    array(
      'output' => 'system-all.css?*',
    )
  );

  $am = new AssetManager();
  $am->set('system_all', $system_all);
  
  $writer = new AssetWriter(DRUPAL_ROOT . '/sites/default/files/css');
  $writer->writeManagerAssets($am);

  var_dump($system_all->getTargetPath());
JohnAlbin’s picture

I've been given time to work on this issue by my employer (yay!): I'll see how far I can get.

Woo-hoo! :-)

catch’s picture

I'm pretty sure Assetic doesn't have anything in it to help with 'bundling'. In #352951: Make JS & CSS Preprocessing Pluggable we discussed making the actual processing of the files, vs. the bundling separately pluggable, and I think that still stands. So the question is how to cleanly separate those two things out into separate subsystems (one of which might be raw assetic or we might need to extend it). If any aggregation logic needs to change to allow Assetic to handle concatenating the file, please take into account #1014086: Race conditions, stampedes and cold cache performance issues with css/js aggregation which has a lot of discussion on how to improve what we currently do as well as various pitfalls.

Also I'm not sure why this is a new issue rather than #352951: Make JS & CSS Preprocessing Pluggable, can we mark it duplicate and do the work there? Or at least make it clear why there's two issues?

JohnAlbin’s picture

Title: Use Assetic to handle JS and CSS files » Use Assetic to package JS and CSS files

@catch Over on comment #46 in #352951-46: Make JS & CSS Preprocessing Pluggable, I think there's a fairly good reason to do "grouping" as a separate plugin from "packaging". So this issue is simply to explore using Assetic to do the packaging, I think.

Maybe mcjim can jump in with what he's learned from the past week working on Assetic.

mcjim’s picture

+1 to title change for clarity.

@catch You're entirely right, Assetic has no logic itself to handle bundling together of assets, it needs to be told what to do.
What Assetic is fantastic at is packaging those bundles into discreet files which can be served from anywhere in the web root, and that's what I've been exploring.

Attached is an experimental patch (which depends on Assetic being in core: see patch at http://drupal.org/node/1784774#comment-6481486. You'll need to apply this first). It simply shows Assetic being used: an actual implementation would need much more work.

Existing Drupal core functions are used to group together CSS files which are then filtered and packaged by Assetic so they can be served from sites/default/files/css. Currently, one filter is being applied, which rewrites image URLs to the new paths, but there is commented-out code in there which can be used to apply the YUI compressor filter too (just as an example, it could be anything).

JohnAlbin’s picture

Status: Active » Needs review

Just to let the bot have at it.

Kiphaas7’s picture

@17, won't happen, the patch is marked as do-not-test, and it depends on assetic being in core. So it would fail regardless.

Kiphaas7’s picture

@mcjim: since you're now the guy who has hands on experience with assetic, what is needed to move this forward? Do you have an idea what/where is needed to create a working patch? So other followers are able to chip-in :).

(assumed the #15 patch was a non-functional, proof of concept patch)

nod_’s picture

step 0 is this #1784774: Remove Assetic component from core then we can talk about patches.

issues should be tacked in this order:

  1. #1762204: Introduce Assetic compatibility layer for core's internal handling of assets
  2. then this one
mcjim’s picture

@Kiphaas7 Give the patch in #15 a whirl, see what you think. it only works for CSS at the moment, so If you have time to experiment with making it work with JS, that would be great. It could be really useful to show the benefit of using Assetic.

I'm working on http://drupal.org/node/352951 at the moment, as bringing in Assetic will be a lot easier once we've made the whole asset system pluggable. I'm reworking the patch in http://drupal.org/node/352951#comment-6537684 but it's still worth looking at, in case you have any suggestions to improve the abstract class I've introduced. Ping me at any time in IRC to discuss :-)

nikkubhai’s picture

RobLoach’s picture

Issue tags: +JavaScript, +Assetic
mcjim’s picture

fubhy’s picture

Did you make any further progress on this? Is there any way we can help?

sdboyer’s picture

i think this, and a number of other issues, are a sorta on hold in favor of the more exhaustive approach we're attempting in #1871596: Create a PartialResponse class to encapsulate html-level data [Moving to sandbox].

robinvdvleuten’s picture

I have written a module to add the base Assetic functionality in D7. Maybe I can help getting the same functionality in core D8 :)

catch’s picture

#352951: Make JS & CSS Preprocessing Pluggable just went in.

This should make an 8.x contrib project possible without jumping through hoops, and it should be possible to use Assetic in core now without any API changes in case someone wants to work on that. I'd consider backporting that from 9.x if it's a transparent change.

catch’s picture

Issue summary: View changes

Fix typo.

LewisNyman’s picture

Issue summary: View changes
Issue tags: +frontend
jhedstrom’s picture

Version: 8.0.x-dev » 8.1.x-dev
Issue tags: +Needs issue summary update

Issue summary is out of date. Bumping to 8.1.x as this is a feature request.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

andypost’s picture

Version: 8.2.x-dev » 8.3.x-dev
Assigned: mcjim » Unassigned
nod_’s picture

Project: Drupal core » Drupal core ideas
Version: 8.3.x-dev »
Component: base system » Idea

Not sure it's still needed, we have pluggable processing and better aggregating on the way #1014086: Race conditions, stampedes and cold cache performance issues with css/js aggregation.

yoroy’s picture

Who can chime in on the need for this? If we don't need it we should mark this idea won't fix. Which can be a very respectable way to end an idea issue :)

catch’s picture

It should be postponed on #1014086: Race conditions, stampedes and cold cache performance issues with css/js aggregation and there are non-assetic js/css preprocessors that Wim Leers was looking at a few months ago, so I'd tend towards won't fix here.

Wim Leers’s picture

so I'd tend towards won't fix here.

Same here. From long discussions with @sdboyer, I remember that Assetic made certain assumptions that Drupal can't make (because Drupal is modular and click-configurable, not all custom code or code-configurable like most Symfony apps). Which is why @sdboyer had to go through so much trouble in defining alternative implementations for Drupal, that in the end we were using rather little of Assetic.

https://github.com/kriswallsmith/assetic shows that it's at version 1.4. No new major features in 1.3 or 1.4. And the changelog file for 1.2 is 3 years old. So I think it's safe to assume that the architecture is unchanged, and hence the problems in Drupal adopting it remain the same.

Summoning @sdboyer to get his confirmation. https://twitter.com/wimleers/status/855106833773719552

Wim Leers’s picture

Status: Active » Closed (works as designed)