Add a progressive web app module to core

My concern with service workers is that you're limited to one at a time. That means if core provides a generic one to do caching, an individual site or module will not be able to provide one. That means we need to either be able to easily turn it off, or it needs to be a complex framework that modules can plug into. The latter option scares me greatly.

Do we really want core to pw0n the service worker user case? Many sites won't even want an offline cached site, especially if doing heavily personalization.

I did see @nod_ sessions at IronCamp today, and I really liked this service workers stuff. However reading Crell's comment that there can be only one service worker at the time. That should not be a big problem for 'simple' caching purposes. However.. Once the use case is asking for more than the caching layer, the fun really starts. Being a core thing, people expect it to handle all things. So like Crell says, it would create a very complex hook/plugin system.

And then there is the fact that Webkit aka Safari is not on board YET. Once again, people except core stuff to work everywhere.

So for those two reasons alone I do think Serviceworkers are better of in contrib space, allowing the technology to grow without the stress of it being a core 'problem'

If we can make sure it stays that way, as just an "easy offline cache" module that is trivially disableable, I'm on board with that. However, it likely will need at least some configuration; I may not want to offline every page, and I may want to control which pages I pre-fetch. A basic pattern match (similar to the block visibility control) is probably sufficient for that, and the limit of how far we'd want to make it configurable.

#5 sounds very valuable. I think it's worth exploring. It definitely has the potential to make many (smaller) Drupal sites into handy references that are available offline.

I agree with #5, it would be great that there is some support for basic caching and offline serving of basic assets, as soon as you get comfortable and want to provide with actual content and get more advanced instead of just providing the basics to the offline mode then you start having to roll your own SW. But just being able to get rid of the dinosaur for basic sites its already a plus I would say.

Contrib can always fill in these gaps with more things of course, but for the basics I would argue that it would be something extremely valuable and yet another incredible selling point for people to choose drupal.

I agree with #5.
I believe it's worth investing time into the module/core implementation and the community will definitely get involved and shape the functionality to help most of the use cases. Also, in the current technology environment PWA is the way to go forward.

Can scopes overlap?

This all sounds *really* good stuff. I can think of lots of uses for this already...

This is not an area I'm particularly familiar with so had to do some reading! Am I right in understanding that iOS doesn't have ServiceWorker and, therefore, needs some sort of polyfill to make this PWA thing work?

If so, we need to include that. If this is going to be used anywhere, it's on mobile and that means Safari is a big chunk of the market. (You've probably already thought of this!)

I like the approach in #5 - an optional core module that focuses on a really simple and common use case: allow a configurable set of resources/assets to be served from a cache where this makes sense and is possible. For many (but not all) users it will make sites faster and more reliable under poor network conditions. However it makes no guarantees about offline support, nor does it adversely affect anyone on a non-SW capable browser.

There are many other use cases, including the more advanced app-like UIs, push notifications etc. It isn't possible to write something for every use case (you'd end up with AppCache). Contrib modules can do those things, and by having a basic implementation in core it should make it easier for those modules to follow. I think it should also be possible to write a completely standalone service worker script and add it to a Drupal site. Hopefully this addresses the concerns made in #2.

This is a similar approach to that taken by the media initiative - basic image/youtube support in core as a reference implementation, whilst making it easy for contrib to support other providers.

FWIW I find the term progressive web app a bit unhelpful in this context. It puts the focus on rebuilding things as "apps", and away from making small incremental improvements to what we already have.

@rachel_norfolk that's correct - Safari (both desktop and iOS) does not support this at the time of writing. This extends to other browsers on iOS too - they must all use Apple's webview under the hood.

This site is a great resource for SW browser support: https://jakearchibald.github.io/isserviceworkerready/

This is also not about providing a perfect or comprehensive solution. It's about providing a generic default that works great for the 80% use case. It's about lowering the barrier to have a website that also works offline.

(At least that's how I interpret it. I'd love to hear whether @nod_ agrees with that or not.)

I believe that there should be two modules.

1. An API layer so that multiple js files aren't being loaded on the client. It could also provide a common UI for configuration so the site admin can easily see which serviceworkers are doing what and to find conflicting scopes.
2. A simple implementation of the API so module creators have an example of how it all works.

Both of which could be disabled when not needed by the site, but implementation is made easier for developers.

The API could also look after the manifest.json (or whatever Apple are doing) and it may be possible to merge together serviceworkers with the same scope.

Yes, more APIs is not what we want. Such an API could never capture all the possible implementation details anyway. If you want something very advanced, write your own service worker. Don't rely on this default one.

Ideally, this module is like the BigPipe module: zero configuration. And if you don't like what it does, then you can still choose to extend what it does in a contrib/custom module.

EDIT: lol, @nod_ also used the BigPipe module as an example. Cool :D

Important question from a marketing/PR standpoint: Is the end-game here "offline sites" (vis, all or most of the site works offline) or "faster sites" (vast swaths of the site are available offline so there's zero network traffic to get them, not even to get a 304.)

My concern with the former is that even one missing asset on a page could render it technically broken. But if we simply offline the entire site period as it's browsed, that could really eat up someone's disk space. (And many mobile devices have limited storage capacity.) Try applying something like that to Drupal.org and I'd very quickly have a gig of cached site and yet 98% of what I do on Drupal.org would still not actually work. :-) (I'm sure there's some disk usage cap involved but you get the idea.) I just want to make sure that what we market it as is actually viable, and desireable. *Most* sites I visit I don't actually *want* to be permanently offlined.

(Now, where it would be REALLY useful is camp/con schedules, which have absolutely no justification in not being fully offline-able in 2016. Lookin' at you, DrupalCon website...)

Adding the "idea template" to the issue summary. A short answer to those 4 questions will help us compare and weigh ideas for their relative impact. Thank you!

Great arguments, but none of them matter.

Google has spoken, and it went all in with Progressive Web Apps merging the best of Android into js based webpages:

• https://developers.google.com/web/fundamentals/
• https://developers.google.com/web/

So, game over.

Drupal should become one :) Or perish.

Thank you @nod_ for outlining the idea like this. It sounds very relevant to me to explore this further.
@giorgio79 your comment is somewhat cryptic, what's the point you're trying to make? :)

@giorgio79 is perhaps a bit too dramatic because as far as I know, having your site work as a PWA has no effect on your ranking (at least for now).

He does have a point in that Google includes a whole section for PWAs on its web fundamentals page:

• https://developers.google.com/web/progressive-web-apps/
• https://developers.google.com/web/progressive-web-apps/checklist

They've also provided some tooling around them:

• Workbox ("a set of tools and libraries to help you simplify your Progressive Web App")
• lighthouse ("automated tool for improving the quality of web pages"), which rates your pwa as well as other things.

So they are pushing this approach, including it as part of good practices at least :)

Hehe, yeah sry for the dramatic / poetic approach.

Google did not just include a section on PWAs but dedicated the entire web developer section to the topic :) https://developers.google.com/web/

If Google will rank higher pages that are PWA then that is where the traffic and eventually the money will go.

PS: It now considers PWAs as fundamental to the web https://developers.google.com/web/fundamentals/getting-started/

Reading this: https://www.aaron-gustafson.com/notebook/your-site-should-be-a-pwa/

The basic isn't new. Just different terms of word and plus one new thing in the modern browsers, which is "Service Worker API"

Google is the game player, I only trust them. Even other links (and who invented the word "PWA") said if you have that few items, that is. I also won't count it as PWA. :p. The first step is to define what we actually wanted.

pwa module + RefreshLess module + bigpipe module = A half-baked PWA. (Not a perfect one, even not 80% IMO. But this is what Drupal can do until this moment and in the near future without a big change --- #2645250: [META] Start using reactive declarative JS programming for some new core admin UIs)

Having PWA in core doesn't mean we need to support all use cases and make it a framework. I can just be a good enough solution that most will find useful. If they want more, replace it with something from contrib.

Big +1 on the approach outlined in #5

We've been working on implementing push notifications in our D8 distribution, for use cases like that we definitely need contrib modules. All the extra libraries and their requirements and installation instructions, let contrib just deal with that :).

How can we then arrive at the "good enough" initial scope for this? What could be the rules, guidelines, common sense guidelines that help define a first take on this?

For those who are eagerly waiting for this to happen. I have something that's tangentially related: HTTP/2 Server Push support. I created https://www.drupal.org/project/http2_server_push for this, it had its first stable release 6 months ago. No known bugs. It pushes CSS + JS assets. Zero configuration.

It's a small thing/aspect on the way to PWAs, but you can use it today. Try it, see if it makes a difference, suggest changes, post patches.

Magento is becoming a Progressive Web Application platform.

https://community.magento.com/t5/Magento-DevBlog/Introducing-Magento-PWA...

#40 😀 I'll take that as a compliment 🙃

Workbox:A library that makes PWAs much easier.

https://www.youtube.com/watch?v=DtuJ55tmjps

I don't know if this library could work with a module outside core.

I have a couple of accessibility concerns with the manifest.json properties. I'll expand on these in #2698127: Extend site information page to generate a manifest.json file, but it deserves a note in the overall plan.

Orientation: We must address WCAG 2.1 "SC 1.3.4 Orientation". Basically, site builders should not be locking orientation. Very few sites will qualify for a WCAG exemption on this. I don't think we should include the orientation property in core's manifest generator at all. A manifest alter hook will suffice for the rare occasions where it can be justified.

Display: When a user views a website in a browser, they have lots of useful tools built in to the browser: bookmarking a URL, find in page, share a URL to another app, Firefox add-ons for dictionary look-ups, reader mode, etc.. But when they save a "page" to the homescreen, the PWA manifest "display" property kicks in, and the user may loose access to some browser tools which they depend on - e.g. with "minimal-ui", PWAs on Android/Chrome do not offer bookmarking URLs. I'm highly skeptical of why many web sites would need to do this at all. I'm not sure what happens if users add a PWA to their homescreen from Android/Firefox, but I'm concerned users will lose the ability to use Firefox addons which they rely on (like dictionary look-ups). I think use of the "display" property could present problems for users with a large range of cognitive/learning impairments. I'll keep looking for articles which can provide more info on this. I don't think we'll find an outright WCAG failure here, but it's something we can provide guidance about for site builders, in the manifest settings form and help topics.

D8 has an official release of the pwa module, which was originally Nod's serviceworker module. I'm working on a document that maps out the history of the pwa module and a roadmap of where it can go. Feel free to join in the issue queue there.