Fully working D8 version based on D7 Serviceworker

I haven't had a chance to troubleshoot this, but I am getting a 500 error when it tries to register the service worker script via the `/serviceworker-pwa` path. Also, the module supports development on non-https at localhost, right? I will follow-up later tonight.

The website encountered an unexpected error. Please try again later.GuzzleHttp\Exception\ConnectException: cURL error 7: Failed to connect to localhost port 32855: Connection refused (see http://curl.haxx.se/libcurl/c/libcurl-errors.html) in GuzzleHttp\Handler\CurlFactory::createRejection() (line 185 of /app/vendor/guzzlehttp/guzzle/src/Handler/CurlFactory.php).

GuzzleHttp\Handler\CurlFactory::finishError(Object, Object, Object) (Line: 102)

Not sure about HTTP support, we work strictly under HTTPS. @AlexBorsody can illuminate this more.
I'll see if I can reproduce the 500 error.

Hmm… OK well, I think it's probably not related to SSL. I used the --ignore-certificate-errors trick documented in this blog post by Dean Hume to run it on https locally with a self-signed certificate. Note that according to the Chromium devs, localhost is considered a secure origin and can server over http and the spec agrees (as long as it meets the requirements).

I'll step through the code with a debugger next to see if I can pin this down. It's probably something stupid on my end but worth documenting.

Upon first saving the PWA config at /admin/config/system/pwa I am seeing a couple of errors.

Warning: unlink(/Users/paul/Sites/schoolpwa/public_html/web): Operation not permitted in Drupal\pwa\Manifest->deleteImage() (line 79 of modules/contrib/pwa/src/Manifest.php).

Warning: unlink(/Users/paul/Sites/schoolpwa/public_html/webcopy.png): No such file or directory in Drupal\pwa\Manifest->deleteImage() (line 81 of modules/contrib/pwa/src/Manifest.php).

These errors clear themselves after saving the config again or clearing caches.

EDIT
----
I am not seeing this with a clean install, so obviously something in my current setup. I have been using D8 PWA dev version for some time, with a custom manifest file. Be good to know if this could be a problem.

I am seeing serviceworker problems on a clean install.
3 x screenshots attached

Thank you. I am travelling and can only test on my Android phone currently, but I now see an install banner so guess this must be working now.

Can we use wildcards for cachable URLs?

/about* for example

The last patch did not apply

Could not apply patch! Skipping. The error was: Cannot apply patch https://www.drupal.org/files/issues/2019-06-20/3060759-15.patch

Thank you for the revised patch.

I am seeing a couple of Lighthouse failures;

8. Is not configured for a custom splash screen
Failures: Manifest does not have a PNG icon of at least 512px.

9. Does not set an address-bar theme color
Failures: No `<meta name="theme-color">` tag found.

In the case of 8. This is because of incorrect size values for the large image. I don't have much experience of creating patches so I just edited 3060759-17. Please find attached

As for 9. I am not a coder and could not see how to resolve this.

Yes it does require a metatag. And if you do decide on a metatag sub-module, it might also be worthwhile providing some optional iOS optimisations for PWAs

<meta name="apple-mobile-web-app-capable" content="yes">

This allows iOS devices to run PWAs in full screen mode. The reason I say this should be optional, is because iphones do not have a back button, and it is therefore necessary to incorporate some UI in an app to enable backward navigation. The latest version of iOS Safari supports forward and backward swipe gestures, an indication perhaps that Apple are starting to recognise PWA's as a thing.

Another iOS metatag controls the appearance of the status bar. I am still experimenting with this and have yet to form an opinion about how important it is.

<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">

I currently have these tweaks in my theme, but they might be good inclusions for the the PWA module. Especially if evaluation tools such as Lighthouse call them out.

References:

https://developers.google.com/web/updates/2015/08/using-manifest-to-set-...

https://developer.apple.com/library/archive/documentation/AppleApplicati...

https://medium.com/appscope/designing-native-like-progressive-web-apps-f...

For other feedback. I'd like to see a sub-module which shows a block which can trigger an installation action. I have started a separate issue about this.

Provide a PWA installation button block

I think metatag output might better be left to the metatag module, but am open to other opinions. Either way, it would be better to open a new feature request issue, rather than getting side tracked here. Mind you, I do believe the mentioned metatags should be output somewhere, and we should also have full documentation how this is all set up. I plan to work on a proper doc section for this module soon, linked into the left sidebar of the module landing page as it should be.

Back to the task at hand: What I am deeply interested in here in this issue is whether the latest patch throws showstopper errors for anyone. Keep in mind that for those who have used previous versions of the D8 module, your phone (esp. Android) will hang on to the old service worker and cause possibly unrelated issues. In fact, even your desktop browser leaves the old service worker hanging around, but there you can at least get rid of it via developer tools.

Thanks for the feedback and testing thus far, much appreciated!

More feedback if I may.

Firstly, using chrome usb debugging I deleted all service workers from my Android phone. I have since performed countless installs, making sure to remove any Chrome history and files on each occasion.

I am seeing inconsistent results for caching. On some occasions with the expected outcome of URL's being cached as expected and seeing the offline message where not. But on other occasions, finding that things have not been cached and not seeing an offline message - broken pages essentially.

The main question that comes to mind is when does the caching actually take place and how long should it take? Is it during the initial installation or after the app is launched for the first time?

For testing purposes, I think it would be helpful if the maintainers could define what is the 'works as designed' behaviour.

Briefly, my method of Android testing was;

- Configure the module with caching for about 6 paths using relative urls

- Clear any Chrome history and files for the https domain
- Reload the site. Observe cookie compliance banner confirming no history
- Select add to home screen install banner
- Enable airplane mode for offline testing (in different ways*)
- Launch the app and test various paths
- Uninstall the app and repeat

* By enabling airplane mode in different ways, I mean that I tested a number of scenarios;

- Before launching the app for the first time
- Immediately after launching the app for first time
- After launching the app and viewing the home page for a short while
- After browsing a couple of paths, closing the app, then re-opening
- A few other variations of the above

// Aside.
After first deleting the service workers I saw a number uncached image placeholders for the first time and probable confirmation that the old service worker was fighting with the new one. This will be problematic for some sites. Can it be mitigated?

Of the placeholder image; being a very dark gray, these made one page look like a heavily redacted document. Perhaps consider a much lighter color that would be easier on the eye. A scalable svg with a missing image symbol would be neat.

In the D7 branch of the module, you can choose which event to wait for before registering the Service Worker, and the default is window.load. I also had options for "immediate" and DOMContentLoaded. If these options aren't present then I'd recommend setting to window.load, as the caching requests by SW can interfere with normal page loads due to competing requests for CSS/JS/img assets.

For testing: it's a tad confusing docs-wise because we've closed so many active issues and consolidated them into one meta-issue. While it makes the coding easier, many of the old D7 issues had specific criteria for testing, and that muddles the ability to leverage my past experience and advice I'd been dispensing for two years as people tested other D7 patches. I won't reproduce everything I wrote in those individual issues but there were steps to reproduce bugs, reproduce/confirm fixes, etc. You might want to read through them even though they were marked outdated. (For the record Chris/Alex spoke with me and we agreed to close the issues in order to make development of their D8 version more manageable, instead of splitting all their great work on D8 into several patches)

As for the issue of two SW files "fighting" with each other, yes that is totally possible, but please do use devtools and confirm this with screenshots before reporting. The majority of bug reports that roll in on SW-related issues are just support requests due to not understanding the detailed specified behavior of the SW lifecycle, including that it won't activate itself until the old one shuts off (and the criteria that triggers it is not straightforward).

Documenting all of the above gotchas in the module is undesireable, because browser behavior changes constantly and it's more responsible to link to externally-maintained docs for respective browsers.

Hi everyone, This is my first drupal build since v2 or 3.

I have installed the PWA module and I have patched it with 3060759-18.patch but once I set up my Manifest in /admin/config/system/pwa nothing seems to happen. If I do an audit on the site it says "Does not register a service worker that controls page and start_url" + "Web app manifest does not meet the installability requirements"

I have a feeling I have missed something simple but I wanted to check to see if anyone could be of any help.

Thanks so much for helping!
I do not see anything to do with pwa in /admin/reports/dblog I had checked there before thinking I would see something as well.

Here are my pwa settings

@chrisdotbm You might want to double check those 'URLs to cache on install'

I spotted a similar problem on a production site yesterday and could not figure out what the problem was as everything was okay with my dev.

I had changed the url alias of a node from /groups to /classes on production but had failed to update this is the PWA config. The service worker was failing and the error log was returning the following;

GuzzleHttp\Exception\ClientException: Client error: `GET https://www.kildwickceschool.org.uk/groups` resulted in a `404 Not Found` response: <!DOCTYPE html> <html lang="en" dir="ltr" prefix="content: http://purl.org/rss/1.0/modules/content/ dc: http://purl.or (truncated...) in GuzzleHttp\Exception\RequestException::create() (line 113 of /home/162921.cloudwaysapps.com/vrazewjpzq/public_html/vendor/guzzlehttp/guzzle/src/Exception/RequestException.php).

After correcting the URL normal service resumed.

@AlexBorsody Is there a case for some error checking that presents a message for roles that have permission to administer the PWA? And also for such mistakes to fail gracefully so that incorrect URLs do not compromise the service worker?

In my case had I used the node ID path instead of it's alias, this would not have happened. But I guess this needs supporting anyway. Moreover, there is often a disconnect between site editor and administrator, and the module should not have reason to fail because an editor deletes or renames a node.

Would converting the 'URLs to cache' input into an entity reference field help with this?

Look like our replies just crossed :-)

I've got a strange thing going with the home screen icon, domain name url being duplicated. What could being causing this?

Error while trying to use the following icon from the Manifest: https://www.kildwickceschool.org.uk/kildwickceschool.org.uk/sites/default/files/pwa/512.pngcopy.png (Download error or resource isn't a valid image)

Just a quick follow-up on my 500 error at /serviceworker-pwa, it's because I was running the site locally via Docker (Lando specifically) at http://localhost:32855, and when Guzzle tries to get the offline pages in _pwa_fetch_offline_page_resources($pages) it fails because that site should be accessed at port 80 from within the container. I confirmed this by SSHing into the container and manually curling http://localhost:80/serviceworker-pwa and there is a response with a generated service worker file.

I am seeing that my iOS device does not install the icon on the home screen, but a screenshot icon instead.

Tested with Safari on iOS 12.3.1 - iPad mini 2

Just a quick partial review of patch #33

+++ b/src/Controller/PWAController.php
@@ -1,26 +1,202 @@
+use Drupal\Core\Controller;
...
+use Drupal\Core\Ajax\AjaxResponse;
+use Drupal\Core\Ajax\AlertCommand;
+use Symfony\Component\HttpFoundation\JsonResponse;

These classes are not used and can be removed.

First of all thanks for al the work you did on for this module.

I have some minor nitpicks for the ConfigurationForm.php:

1. +++ b/src/Form/ConfigurationForm.php
   @@ -0,0 +1,409 @@
   +use Drupal\Component\Utility\UrlHelper;
   

   Unused class.

2. +++ b/src/Form/ConfigurationForm.php
   @@ -0,0 +1,409 @@
   +         if (!$url->isRouted()) {
   +           $form_state->setErrorByName('urls_to_cache', $this->t('Error "' . $page . '" URL to Cache is a 404.'));
   +       }
   +       else {}
   

   Remove else and fix indentation.

Oh, and I found a minor typo:

+++ b/src/Manifest.php
@@ -0,0 +1,146 @@
+   * @var \Drupal\Core\Language\LangaugeManagerInterface

Should be: LanguageManagerInterface

Fixed previous mentioned issues ( see interdiff )

Lets move this patch along :)

Ref #37 Take a look at Responsive Favicons module. They have home screen icons working for iOS.

I don't remember whether this came from their manifest file or some metatags.

Sorry. No testing against D7

Did you have to install the Shortcuts App that is mentioned your linked article?

EDIT
----
Seems a bit of a palaver and not something we could expect a site visitor to endure.

So, a bit of reading up confirms that add to home screen is supported by iOS (and always was). But iOS does not use the icon referenced in the manifest file. A workaround is needed.

https://www.google.com/search?q=progressive+web+app+apple+touch+icon

It is necessary to insert the code below in the head and provide a suitable png file.

<link rel="apple-touch-icon" sizes="180x180" href="/apple-touch-icon.png">

A bit messy as different sized icons were needed by different devices, but I think the 180 pixel size covers all. Perhaps someone can confirm this?

The code would need to go in the theme or otherwise be inserted with a module such as Metatag Favicons or Responsive Favicons. If using the later, important to remove references to the bundled manifest file so it does not compete with PWA module.

I tend to agree. Just writing up my findings to assist the discussion and help with basic documentation.

Review of #42

Great spotting the missed binary flag. I learned something :)

+++ b/src/Form/ConfigurationForm.php
@@ -0,0 +1,411 @@
+      $image_size = getimagesize($theme_image);

In my installation this is trying to get /themes/customtheme/logo.svg

That is not where it is located.
In my installation it is located at ./themes/customtheme/logo.svg or ./themes/customtheme/logo.png

Can we change this to $image_size = getimagesize('.' . $theme_image);

Or is this something that is wrong with my theme settings?

I guess we should use
$theme_image = theme_get_setting('logo.path', $nameOfDefaultTheme);
instead of
$theme_image = theme_get_setting('logo', $nameOfDefaultTheme)['url'];
Also in the Manifest.php

Regarding #48 yes I would use a similar meta tag but opt for 192x192 instead of 180:

<link rel="apple-touch-icon" sizes="192x192" href="/img/icons/app-icon-192.png">

This tends to work for me in my personal projects that incorporate iOS Homescreen + true PWA w/ manifest support.

As for how to implement this, I personally think injecting our own meta tag is perfectly fine. The D7 branch already injects the meta tag for the theme_color:

<!-- example from pwa-7.x -->
<meta name="theme-color" content="#ffffff" />

Whoops! After submitting I saw the comment in #49 about waiting for another issue to address meta tags. Fine by me :)

Getting the correct theme path for the theme logo.
Hmm this is actually comment #55

Thanks, guys for your really great job!
#58 works perfectly for me.

Based on #59 (and previous optimistic comments, and internal testing) I am setting the issue to RTBC, with the aim to commit what we have and roll a RC release unless issues surface.

Exciting! Great work to everyone involved with development, testing, and feedback!

At Alex's request I tested this with a larger number of urls for caching, and did so with 65 or so. There were no problems.

I see that an additional icon of 144x144 pixels is being generated, but I am still seeing a basic screenshot icon on iOS.

Reading back through the posts is not entirely clear whether it was agreed Apple Touch Icon support would be part of a separate metatag issue; or to be dealt with via documentation or whether such functionality should be provided by this issue as hinted at in #57. Some clarification would be helpful.

My personal feeling is that despite Apple's less progressive approach to PWAs, we should have a proper icon for iOS, given that a well established method using a metatag already exists and has done for many years.

Forgot to mention also; full marks from Lighthouse.

YEAHH! I applied patch in #58 and I was able to successfully register the service worker at http://localhost:32813. That's right, non-HTTPS local development and no guzzle issues with the port.

I noticed the new default language attribute since I am using the multi-lingual Umami demo, excellent.

Saving and received a notice on URLs to cache on install for a missing URL, very nice.

I tried iterating the cache version as well and observed the new SW queue up, Waiting, Install after reloading the tab, and remove previous cache instance. Glorious!

RTBC from me.

Confirm that the service worker uninstalls itself after the module is removed.

Should the pieces of this that originated from other issues be split off and posted back at those issues so that contributors can get their issue/commit credit? Or maybe just adding the credits at those issues and being sure they're included as authors when this gets committed would be enough?

hi alex, i have tested the patch in drupal 8 and it is working fine for anonymous users, am trying to make it work after i log in, is it possible?, i had an error saying not permissions for manifest.json file, i gave permissions from anonymous to authenticated users but is the same error, can you help me out please, thanks

@alexBorsody i have attached the images for the error, i have configured the permissions for authenticated users, then i LOG IN, after that i get the console log errors.

The screenshots in #72 are a result of HTTP Basic Auth being used on a dev/staging site. See some discussion here: https://stackoverflow.com/a/51157352/175551

The manifest.json is fetched as CORS by default, since it's supposed to be a public manifest of your app (to be included in aggregators and app stores and such).

The SO I linked to above mentions adding the crossorigin="use-credentials" attribute to ensure any credentials are sent along with the manifest request: https://developer.mozilla.org/en-US/docs/Web/HTML/CORS_settings_attributes

I'm not sure if the module should fix this by default, but that's the problem (and solution).

rupl thanks for the answer, the question is why is it working fine for anonymous users then?

When I load https://tigoselfcareregional-stg-bo.tigocloud.net/ and look at the HTML response, I do not see a manifest nor a reference to the serviceworker-load.js script tag. It doesn't seem like it's loading the PWA.

When I curl https://tigoselfcareregional-stg-bo.tigocloud.net/manifest.json I do see the manifest as an anonymous user. No auth was needed.

It seems my explanation in #73 is wrong.

I have been checking out with a partner and she realize the problem is on permissions, i am attaching the image where i have to change to make the module works for authenticated users, i had to change it from "access pwa" to "access content". for those 2 urls, should it be changed in this patch? or am i missing something i can do without modify the code?

As Alex is rolling our first beta I have added all contributors to other D8 issues to properly credit everyone who stuck it out on this long journey.
Looking forward to a solid 8.x-1.0 release soon.

alex could you help me please making the change about the permissions to "access content", i haven't done yet a patch update, so i would appreciate if you do the change in #76, thanks.

We will definitely check into that @lfjimenez. Alex and I just decided to roll the beta now and deal with remaining glitches as we near a 1.0 release.

Ok folks, we have a beta release out, please use that going forward and let us know what you find in fresh issues. I am closing this one now.
Thanks to everyone who contributed!

@lfjimenez, I opened a fresh issue for you: #3069828: Permission issue

Fantastic work all around! thanks Christoph for adding credit to all the previous issue contributors.

@AlexBorsody just a little housekeeping: could you please make sure the 8.x-1.x git branch is updated with your latest work? I noticed that no commits were pushed to that branch and when I pull from git, the last commit is:

commit f4178f7870baf11eb12a1f4bd1cb0828d9ed7c2a (HEAD -> 8.x-1.x, origin/8.x-1.x)
Author: git <git@3589438.no-reply.drupal.org>
Date:   Thu Oct 4 15:03:47 2018 +0200

    Issue #2842739 by mikael_ek, rupl, attiks, cafuego, jcmartinez: Failed to load resource - problem loading SW in Nginx

This is also confirmed by the main project page, which still reports that the last commit to the 8.x dev branch is 4 Oct 2018 (even though the new release is dated 23 July 2019)

It would be best to only commit to the 8.x-1.x branch and do local merges before pushing code. If others want to make new patches they should be able to do so from the 8.x-1.x branch at all times.

Edit: I'm not 100% sure but this might also require a re-release of the beta, as I see on our commits page that the 8.x-1.0-beta1 tag is on that same commit: https://www.drupal.org/commitlog/commit/85315/f4178f7870baf11eb12a1f4bd1...

Good call on branching and housekeeping @rupl. Will do today if possible.