Port CDN to Drupal 8

Standardized the title.

I've begun porting yesterday. I'm making good progress. But with all my other responsibilities, and much higher-impact things that should happen first (https://www.drupal.org/project/big_pipe), I won't have a release available at the same time as 8.0.0. Expect a release before 8.0.1, within a week at most.

Is there any more news on this? 8.0.1 has been out for a while now, and haven't seen any work on this in the git repo.

^ Was just about to ask the same thing.

At least could the maintainers update the Project page with a more accurate assessment given the current reality?

From project page:

A Drupal 8 port will be done around 8.0.0, but before 8.0.1.

I'm not advanced enough to help with the port but would like to express my appreciation for all you've done. Thanks so much for your time and hard work!
Is there anyway of getting an update on this as I've got a new build and would love to use Drupal 8, and a CDN is pretty important for all public files.
Thanks again!

Apologies for the delay. I expected to find time to port the CDN module sooner. I've literally been looking forward to port it for more than a year.

In the mean time, I've helped land #1494670: References to CSS, JS, and similar files should be root-relative URLs: avoids mixed content warnings & fewer bytes to send. This will be very important for the Drupal 8 port of the CDN module also.

My current intent is to move the Far Future expiration logic into a http://stackphp.com/ component, so that it can be reused outside Drupal too. I also intend to remove a lot of the more complex aspects (File Conveyor because obsolete, HTTPS support in favor of protocol-relative URLs). The brittle/complex syntax-based configuration that currently exists for CDN mappings and unique file identifier generation will be made less error-prone and easier to deploy thanks to all of that moving out of variables (D7) and into configuration (yay for D8's configuration system). Thanks to configuration schemas (yay again), it'll be much easier to understand what that configuration actually means.

And finally, I am strongly contemplating removing the UI, in favor of requiring a cdn.settings.yml file to be created manually. Just like Drupal 8 core already requires for the REST module (rest.settings.yml). Thoughts?

I am strongly contemplating removing the UI, in favor of requiring a cdn.settings.yml file to be created manually. Just like Drupal 8 core already requires for the REST module (rest.settings.yml). Thoughts?

I don't think a minimal CDN UI would hurt. This way you'd have cdn.settings.yml with recommended settings on module enablement, and normal CM-based workflows taking it from there. Having to create the file would probably be less obvious for many, despite this information being available, say, in INSTALL.txt

I wouldnt let the creation of the UI delay an alpha or beta level port. UI can always be worked in later. !!Wild assumption warning here!!: Most anyone implementing CDNs must already have a pretty advanced knowledge of site building... they've most likely already learned the D7 syntax and are storing environment specific CDN configs in code. Thus requiring a yml file based config to be written by hand shouldn't be a blocker.

#8: that matches my thinking. Thanks for your feedback!

I'm fine with not having a UI for the settings file as long as all of the options and keys are well documented.

#10 Of course, that speaks for itself :) Thanks for your feedback!

I've just tagged the 7.x-2.7 release of the CDN module: https://www.drupal.org/node/2666056.

Now that D7 maintenance will be done for quite some time to come, I can focus on the D8 port.

It's been crazy around here. Acquia Build Week. We have to move, so have to look for a new apartment. Sorry for the delay.

I've been working on the config schema today. After a lot of different attempts, this is what I've settled on. I'd like your feedback.

Let's use the examples in the documentation (at help/admin-details-mode-pull-cdn-mapping.html in the D7 CDN module) as a comparison:

• This would serve all files from a single CDN:
  

  http://cdn-a.com

• This would serve CSS files and images from CDN A, .zip files from CDN B and everything else from CDN C:
  

  http://cdn-a.com|.css .jpg .jpeg .png
  http://cdn-b.com|.zip
  http://cdn-c.com

• This would serve CSS and JS files from CDN A, images from either CDN B or C and everything else from CDN C:
  

  http://cdn-a.com|.css .js
  http://cdn-b.com http://cdn-c.com|.jpg .jpeg .png
  http://cdn-c.com

The three corresponding configurations in the cdn.settings.yml file would be:

1. This would serve all files from a single CDN:
   

   cdn:
     type: simple
     host: cdn-a.com
   

2. This would serve CSS files and images from CDN A, .zip files from CDN B and everything else from CDN C:
   

   cdn:
     type: per_file_extension_with_fallback
     host: cdn-c.com
     per_file_extension:
       single:css_and_images:
         host: cdn-a.com
         extensions: [css, jpg, jpeg]
       single:downloads:
         host: cdn-b.com
         extensions: [zip]
   

3. This would serve CSS and JS files from CDN A, images from either CDN B or C and everything else from CDN C:
   

   cdn:
     type: per_file_extension_with_fallback
     host: cdn-c.com
     per_file_extension:
       single:css_and_js:
         host: cdn-a.com
         extensions: [css, js]
       balanced:images:
         hosts:
           - cdn-b.com
           - cdn-c.com
         extensions: [jpg, jpeg, png]
   

Thoughts?

Looks straightforward to me, only possible improvement would be to remove the single: and balanced: prefixes, although I suppose without that designation the config schema won't validate as hosts is different for each. Would it be simpler if hosts were always a sequence?

Also : cannot be within a key, the key would need to be escaped, no?

I based that on this example from your cheat sheet:

messages:
 ‘single:1’: ’Hello!’
 ‘single:2’: ‘Hi!’
 ‘multiple:1’:
 - ‘Good morning!’
 - ‘Good night!’

Yup, if you wrap that in quotes, it should be fine :) your code sample above did not use quotes.

Also you don't need to base it on a prefix, you can include a type key within the data: single or balanced. That would likely be clearer to read vs. encoding it in the key.

Woot, with Gábor's help (in chat), I managed to get this all working! :) I verified all possible setups using https://www.drupal.org/project/config_inspector.

The 3 examples in #13's final syntax:

1. mapping:
     type: simple
     domain: cdn-a.com
     conditions:
       extensions: [jpg, jpeg, png]
   

2. mapping:
     type: complex
     fallback_domain: cdn-c.com
     domains:
       -
         domain: cdn-a.com
         conditions:
           extensions: [css, jpg, jpeg]
       -
         domain: cdn-b.com
         conditions:
           extensions: [zip]
   

3. mapping:
     type: complex
     fallback_domain: null
     domains:
       -
         type: simple
         domain: cdn-a.com
         conditions:
           extensions: [css, js]
       -
         type: auto_balanced
         domains:
           - cdn-b.com
           - cdn-c.com
         conditions:
           extensions: [zip]
   

As a bonus, now you can also do:

mapping:
  type: auto_balanced
  domains:
    - cdn-b.com
    - cdn-c.com
  conditions:
    extensions: [jpg, jpeg, png]

And more importantly, you can add your own types of conditions! People who only want to serve files in certain directories from a CDN for example, will now be able to write a plugin that does just that, and the config schema allows for it.

You can even define new CDN mapping types (besides the three current ones: simple, auto_balanced and complex).

Perhaps the type: complex mapping type should be called nested (or even recursive, though that sounds too scary)?

(Corresponding config schema attached.)

It's very late now, so I won't be pushing this today, but now that this is settled, I'll be able to make very fast progress at last :)

Please provide feedback! Is this simple/clean enough?

Great work, it all looks really good. I have a question about:

mapping:
  type: auto_balanced
  domains:
    - cdn-b.com
    - cdn-c.com
  conditions:
    extensions: [jpg, jpeg, png]

How can we consistently ensure assets are going to be served from both cdn-b.com and cdn-c.com and not only from cdn-b.com as it comes first in the list?

Naming things is hard. If we keep simple, then I think complex is the correct opposite. If we want to try a few alternatives, here's what I just had in mind:

• simple / advanced (my personal fav.)
• single / advanced
• single / multiple
• direct / multiple

#20: This is just the same auto-balancing feature that already exists in D7. See help/admin-details-mode-pull-cdn-mapping.html in D7, it was introduced in #1452092: Automatically distribute certain filetypes over multiple CDNs.

I also contemplated advanced. Perhaps single + auto_balanced + nested make more sense?

While working on a super minimal UI, ran into #2595613: '#title_display' => 'invisible' does not work for composite form elements.

A simple CDN (pull origin only) module: https://github.com/chapter-three/cdn_pull_origin The code might help with CDN Drupal 8 port.

Ran into another issue while working on the UI: #2677414: VerticalTabs' #default_tab broken due to wrong/mismatching class name.

Wow, both of those got committed already! Incredible that @alex.pott already committed them! :O

I finished writing the CDN UI submodule. Here's screenshots.

First I wanted to limit the UI to literally just one case: the one where you use a CDN for all files, and all we need to know is the CDN domain. That would have covered at least 50% of use cases. But only serving certain files from a CDN (e.g. only images, or only videos, or only CSS/JS, or …) is a fairly common use case too, and if the UI supported that, then 90% of sites could just stick to this UI. So I added support for that too.

This is what the UI looks like now:

#26 made sense to do as part of this port, and then also solves #1784432: Version 3.0: move UI of CDN module into a cdn_ui module.

I started porting this on November 18, 2015. One day before the Drupal 8 release.

I'm now cleaning up my commit history, and am adding scaffolding for the things that still remain to be done, so that people who want to help out with this port (I've received several offers, thank you very much!) can actually do so. I just wanted to have the fundamentals outlined: the config schema, the removal of File Conveyor support, a CDN UI submodule.

Should this be 8.x-2.x or 8.x-3.x? It will offer similar functionality, but with File Conveyor gone, and e.g. #1784432: Version 3.0: move UI of CDN module into a cdn_ui module finally done, it feels like 8.x-3.x is appropriate. Thoughts?

Agreed on 8.x-3.x.

Re: #21 I understand the decision better, thanks. I don't see the urge to file a new issue to have perfect balancing unless someone can back this up with real-life data.

Glad to see #1784432: Version 3.0: move UI of CDN module into a cdn_ui module which echoes my suggestion in #7 - Having a minimal UI to account for 50% or more of all CDN module installs is great. If you're willing to have more advanced use cases, then surely you can configure advanced settings via YAML.

I also agree on 8.x-3.x which brings substantial changes and will make sense better in a new branch.

Per #28, I worked yesterday night on cleaning up my commit history so that it is more helpful for the future, and am now working on the scaffolding for the things that remain to be done. I also want to make sure that the utmost basic case (alter all file URLs to use the same CDN domain) is at least working, so that there is a working basis.

I'll try to wrap that up tonight, but it could be later this week.

Anymore movement on this? great work so far and understand you're super busy but is there anywhere we can look at/ mess about with the code you've already got?

Following!

You need to update the module page "A Drupal 8 port will be done around 8.0.0, but before 8.0.6."

Any chance we could get a source link?

Sorry for having taken so long. But it's finally here :)

• 8.x-3.x-dev snapshot: https://www.drupal.org/node/2708759
• 8.x-3.0-alpha1: https://www.drupal.org/node/2708767

all the fundamentals

• the ability to use simple CDN mappings, including conditional ones depending on file extensions, auto-balancing, and complex combinations of all of the above
• DNS prefetching
• a simple UI to set it up — in fact, much simpler than before!

changed/improved

1. the CDN module now always uses protocol-relative URLs, which means there's no more need to distinguish between HTTP and HTTPS, which simplifies a lot
2. the UI is now a separate module (fixes #1784432: Version 3.0: move UI of CDN module into a cdn_ui module)
3. the UI is optional: for power users there is a sensible configuration structure with strict config schema validation
4. complete unit test coverage of the heart of the CDN module, thanks to D8's improved architecture
5. DNS prefetching using headers rather than tags in <head>, which allows me to use a much simpler/cleaner Symfony response subscriber
6. tours instead of advanced help (fixes #1753174: Dependency for advanced_help module), which very often was ignored

removed

1. File Conveyor support
2. separate HTTPS mapping (also mentioned above)
3. all the exceptions (blacklist, whitelist, based on Drupal path, file path…) — all of them are a maintenance/debugging/cacheability nightmare

TODO

1. the 'Access files on the CDN when in testing mode' permission is not yet operational… but should it even be? Is this something people actually use? Should we keep it or remove it?
2. integration tests: #2708777: Write/migrate integration tests
3. Far Future expiration support: #2708787: Port Far Future expiration to 8.x-3.x
4. SEO/duplicate content prevention: #2708771: Port SEO (duplicate content prevention) to 8.x-3.x
5. #2666382: EditorFileReference filter: generate URLs for inline images (core issue) to allow the CDN module to rewrite file URLs in user-generated content
6. #1783278: Scheme-relative URL rejected by validation (core issue) to allow the CDN module to work in tandem with file upload widgets, which currently cause errors due to this core bug

Looking forward to your feedback!