Problem/Motivation
jquery.cookie is no longer a maintained library. A replacement should be found.
> git grep core/jquery.cookie core\core.libraries.yml | wc -l
3
> git grep -l cookie -- "*.js"
core/assets/vendor/jquery-joyride/jquery.joyride-2.1.min.js
core/assets/vendor/jquery.cookie/jquery.cookie.min.js
This is used in jquery.joyride and the only real usage I have found. Also, jquery.joyride is not hard dependency on cookies.
https://github.com/zurb/joyride/blob/v2.1.0/jquery.joyride-2.1.js#L32
Proposed resolution
Replace jquery.cookie with js-cookie library adding a backwards-compatibility layer. We can model the BC layer off of js-cookie v1.5.1 as well as the js-cookie v2.0.0 release notes (#26).
This approach was originally reviewed by droplet and confirmed by _nod in 2017. There is now a major version 3 in js-cookie that has a beta release. This is being evaluated in #3118726: Upgrade to js.cookie 3.
jquery.cookie, js-cookie, and the object.assign polyfill are correctly loaded when using Internet Explorer 11. See #153 and #154.
jquery.joyride will no longer depend on jquery.cookie because tour module does not use the functionality that uses cookies.
Remaining tasks
- Add more tests
- Review approach by JavaScript subsystem maintainers
API changes
Yes, with backwards-compatibility layer.
Dependency evaluation
Maintainership of the package: Maintained by the excellent carhartl, who also maintained jQuery cookie. It is actively maintained and the issue queue is very clean. I reviewed closed issues over the past several months and the response time is quite fast.
Security policies of the package: A documented security policy is not available online. Since this library is essentially a successor to jquery.cookie, the security approval granted to that library may extend to this one, but this is a judgement best suited to someone from the security team. In particular I'm not sure if 2.x will be supported once 3.x is out of beta. Issue to ask the maintainer opened at: https://github.com/js-cookie/js-cookie/issues/614
Expected release and support cycles: The release schedule is irregular based on the maintainer's availability and need, but there tend to be a few releases a year. The releases are available at https://github.com/js-cookie/js-cookie/releases. The maintainer follows semver strictly insofar as one can with a JavaScript library. (Dots are used in tags for pre-release versions which differs from Drupal but is valid semver.
Code quality: > 2800 dependents, available in all popular package managers. It's trusted by many and any concerns will likely be assuaged by quickly reviewing the 163 lines (including whitespace) .
Other dependencies it would add, if any: no dependencies, only dev dependencies that Drupal never pulls in.
Release notes snippet
jquery.cookie has been replaced with js-cookie version 2. The core/js-cookie library is introduced, and a backwards-compatible shim is provided as core/jquery.cookie for Drupal 9. We may upgrade to js-cookie 3 if it is available before 9.0.0-rc1.
| Comment | File | Size | Author |
|---|---|---|---|
| #198 | interdiff_89X_190-198.txt | 527 bytes | bnjmnm |
| #198 | 2550717-198-89X.patch | 2.38 KB | bnjmnm |
| #197 | interdiff_90X_194-197.txt | 323 bytes | bnjmnm |
| #197 | 2550717-197-90x.patch | 2.28 KB | bnjmnm |
| #194 | 2550717-194-90x.patch | 1.96 KB | bnjmnm |
Comments
Comment #3
silesky commentedI'm working at this issue at DrupalCon Nola in the sprint room!
Comment #4
silesky commentedThis looks like a good idea!
ckeditor might have a reference here: /core/assets/vendor/ckeditor ... it's minified code so I'm going to see if it's implementation is--in fact--a jquery one.
I'm tagging this 'need subsystem review' because I'm not sure if we can just pull out this javascript library (i.e. modules might depend on it)
Comment #5
droplet commentedIt was created before D8 release. Yeah, it's a bit late now.
Comment #6
silesky commentedThere is a CKEditor reference to a cookies object in the codebase, but on examination of the un-minified CKEditor code, it is a native (i.e. non-jquery) implementation. The other reference was in the joyride library, but on examination of the code (as droplet said), it appears to be a soft dependency-- as it should be, since the JQuery cookies plugin has been deprecated. In my testing of the tour module, the jquery cookies function was not called, and the tour module appears to work perfectly fine.
Comment #7
silesky commentedComment #8
chi commentedjquery.cookie library is no longer maintained.
https://github.com/carhartl/jquery-cookie
Comment #9
dawehnerMh, are you sure this can still be removed? Technically this would be a BC break and well, contrib modules / custom modules might easily rely on its existence.
Comment #10
droplet commentedNo. It's too late.
Upgrading to JS-cookie seems a more right direction for now and future.
Both jQuery.cookie & JS-cookie shared same API design. we can add BC layer:
to
Usage Stats:
https://github.com/search?l=javascript&p=1&q=org%3Adrupalprojects+jquery...
Comment #11
dawehner@droplet
Is that a better title?
Comment #12
droplet commentedComment #13
cilefen commentedA ticket for 7: #2742789: Update Cookie plugin 1.0
Comment #16
nod_Once again, agreed with @droplet in #10
Comment #17
manuel garcia commentedHere is a patch swapping it for js-cookie.
Still to do adding BC layer - though I'll leave that to someone with more in depth JS knowledge... unless you're happy to walk me through it of course.
In my opinion however I don't see why core would have to provide this library if itself is not using it. We could provide this in contrib, and other modules depending on
jquery.cookiecould move to that... perhaps in 9.x?Comment #19
manuel garcia commentedOops, thanks tests!
Comment #24
marassa commented@droplet re #10:
It's probably worth noting that while sharing the same syntax, jquery.cookie and js-cookie have different defaults:
$.cookie('name', 'value')sets a cookie for the current path butCookies.set('name', 'value')sets the cookie for the root.Comment #25
cilefen commentedComment #26
johnwebdev commentedWe can use js-cookie v1.5.1 as reference for the BC layer as it was back compatible with jquery-cookie at that point.
https://github.com/js-cookie/js-cookie/blob/v1.5.1/src/js.cookie.js
The release notes v2.0.0 also contains some great notes on what we need to do on our BC layer.
https://github.com/js-cookie/js-cookie/releases/tag/v2.0.0
There was 2 years ago however a subsystem maintainer signed of on this, probably should revisit if this library is the correct replacement :)
Comment #28
mradcliffeBased on #17 and #26 this is at Needs work to add a backwards-compatibility layer.
I updated the issue summary and changed the priority to Major.
Comment #29
mradcliffeThe patch no longer applied so I re-rolled and then tried to implement a backwards-compatible "core/jquery.cookie" library. I also added the "needs frontend framework manager review" based on @johndevman's comment regarding it being 2 years since the last subsystem maintainer review.
$.cookieand$.removeCookiewrappers.npm run lint:core-jsandnpm run build.I didn't add a backwards-compatible way to set path based on the 2.0.0 release notes.
To manually test, I applied the "with-joyride" version of the patch, enabled the Tour module, which still has a reference to jquery.cookie, and then went to the View Edit page. In my console (Firefox), I was able to run the various commands to test basic functionality. The "with-joyride" version should not be used because joyride does not have a hard dependency on jquery.cookie.
jQuery.cookie("blah");returnsundefined.jQuery.cookie("blah", "blah");returns"blah=blah; path=/".jQuery.cookie("blah");returns"blah".jQuery.cookie("blah", "halb");returns"blah=halb; path=/".jQuery.removeCookie("blah");returnstrue.Cookies.get("blah");returnsundefined.Cookies.set("blah", "blah");returns"blah=blah; path=/".Cookies.get("blah");returns"blah".Cookies.set("blah", "halb");returns"blah=halb; path=/".Cookies.remove("blah");returnsundefined.Comment #30
mradcliffeI added a question in the proposed resolution to clarify the the needs frontend framework manager review tag.
Comment #31
lauriiiThank you @mradcliffe! This seems like great progress!
Since we don't have any test coverage for this ourselves, I was thinking if we could run the tests from the upstream package. I think I figured steps needed to be able to do that, but it seems like there's some failures that we might want to look into.
Steps I used for running tests:
git clone git@github.com:carhartl/jquery-cookie.gitgit apply --patch jquery-cookie-test-only.patchyarn install./node_modules/.bin/grunt connect:testsI'm tagging this for subsystem maintainer review because JavaScript maintainers might want to review this too. 😎✌️
Comment #32
tedbowI am not sure how we can remove the dependencies here and no tests fail. I am probably missing something but since these libraries depend on
core/jquery.cookieand we are removing it without a replacement for a dependency shouldn't functionality be lost? Is it just because we don't have test coverage?Could we actually name this with "shim" or something in the file name. We should provide a BC layer for someone using the library not the file directly. This seems confusing.
I think we should also be calling
Drupal.deprecationError()for all of these methods. see #3070521: Trigger deprecation notifications on JavaScript deprecations to notify developers that deprecated code is being usedComment #33
tedbowForgot to upload patch
Comment #34
mradcliffeBased on the research earlier in the issue, it was determined that these were not hard dependencies. This _could_ affect contrib/custom if someone depended on having the cookie plugin added to JQuery, but only depended on drupal/form or drupal/tabledrag. There isn't any use of the dependencies in those two.
This makes sense to me.
+1. Would this go in each function in the shim or called once when the immediately-invoked function expression (IIFE) is invoked?
Updated the issue summary.
Comment #35
mradcliffeForgot to change to Needs Work (NW).
Comment #36
tedbow@mradcliffe
I think in each function in the shim. This would be the first non-test to use this but you can take a look at
core/modules/system/tests/modules/js_deprecation_test/js/js_deprecation_test.es6.jsthanks for the explanation. I wonder if there is any way we could deprecate this dependency. It shouldn't be in this issue but it seems like we should only have dependencies for what core needs and if someone need to set a cookie they should add the library explicitly. Not sure how we could deprecate something like and be able to warn sites.
Comment #37
tedbowComment #38
mradcliffeI renamed the shim to include shim, added the Drupal.deprecationError calls instead of @deprecated, and then added a cute test module and Nightwach test for both the shim and js-cookie. But I probably have screwed up my Drupal JavaScript behaviors since I haven't done that in a while.
Uploading the patch, but it still needs work.
Comment #39
mradcliffeI tried to change this to
$('.js_cookie_test_add_button').once('add').on('click', but that still didn't work in Nightwatch.However that did work when I made a custom module that did the same thing and installed locally for manual testing.
Should be
core/jquery.cookie.I fixed this in this patch.
My guess is that Nightwatch's async. is too fast here. Maybe click takes a function?
Also fixed this file name to match the real file name.
Comment #40
mradcliffeSwitching to once requires core/jquery.once dependency although that seems to be inherently required on standard/minimal installs (local dev) rather than testing installs (Nightwatch).
I think my original issue was with
$.findrather than$, but using once is the right thing to do here.This did expose a test failure with my current assertion.
Comment #41
mradcliffeThis addresses my comment in #40, and now the js-cookie test is working.
Also I found that I forgot to delete jquery.cookie.js in #38's patch, and that core.libraries.yml entry for jquery.cookie had that file name referenced, which is why the shim test is failing.
This should fix the patch and pass the tests now. :-)
Comment #42
mradcliffeHmm, I forgot to actually add js-cookie to the patch because I had a .gitignore active on this environment, which restricts "vendor" anywhere.
Comment #43
tedbowI don't think we need this test just the one that covers the shim. I don't we generally provide test for third-party libraries we are just including.
The shim has it's logic so the test there is good.
Comment #44
bnjmnmThis is looking good, all I've found so far are no-big-deals.
It may be good to get input from @lauriii, as he has a good sense of how BC-supporting the libraries layer needs to be. Removing these includes could be a potential BC problem in 8.9. Even though these libraries no longer use cookie functionality, there may be custom/contrib uses that are able to access jQuery cookie because these common libraries happen to be loaded.
These should use arrow syntax
This can be moved to the scope of
$.cookie()These functions should be annotated. In particular it would be good for the descriptions to be detailed enough that if someone thinks there's an issue with the shim, this will help them confirm or refute that suspicion. (It'll also help me do a more thorough review of the JS - everything looks solid so far but I'd like to be sure that assessment isn't based on incorrect assumptions).
This is another spot where a comment will help facilitate a quicker understanding of what is going on.
Somewhere I think it would be useful to make it clear that window.Cookies originates in js-cookie. Largely to benefit contributors that may not spend much time in JS.
Doesn't look like there's anything problem-causing about this approach, but there could be some future-proofing in matching the common convention and extending ControllerBase instead of using StringTranslationTrait directly.
Needs arrow syntax
This should use arrow syntax
All-caps the value for version:
Comment #45
mradcliffeI haven't re-rolled the patch in a while so this might fail to apply. I wanted to work on the changes in the review by @bnjmnm #44 and @tedbow #43, which are mainly all correct. Thank you!
The patch addresses:
Regarding 44-3:
I didn't do this because the deprecation message is used in both
$.cookieand$.removeCookie, and I don't think I need to define another const with the same text.Patch does not address 44-1.
Issue status remains at Needs work.
Comment #46
mradcliffeThis patch addresses the bug in the getter that was exposed when I looked at the JSON test. I had a good test for the setter, but not the getter since I was asserting based on the raw cookie value returned from Nightwatch. That is still important to assert that it is encoded.
In addition to changing that assertion, I added direct calls to jQuery.cookie in the Nightwatch to test the output of the getter in both tests. Nightwatch suggests not passing arrow syntax into its execute method so I needed to disable some eslint style rules for those lines.
I also removed passing in a js-cookie style converter because the shim should only support what was supported by jquery.cookie, which was a function.
44-1 is pending frontend framework manager review so I'm adding that issue tag back.
Comment #47
mradcliffeI created a "test only" patch and I think this should not fail on core since the test module is only testing the shim and not js-cookie. Hopefully. :-)
Comment #48
mradcliffeOh, wait, it's going to fail because the deprecated message won't be there.
However the first test does pass up until that point which asserts the encoded and decoded cookie value :partyparrot:
Comment #49
mradcliffeRemoving the deprecated assertions from a test-only patch works up until a change from how js-cookie 2 handles decode/encode from jquery.cookie. The raw cookie value is allowed to contain brackets unencoded in js-cookie, but not in jquery.cookie. This shouldn't matter if using JSON because the object is returned correctly, but if manually encoding/decoding than it is a BC break. It is a very minor one though, and it could have been considered a bug fix because the change was to only encode unallowed characters in the cookie value rather than any URI encoding.
However the safest thing to do is to use the decode/encodeURIComponent from jquery.cookie, and then document this behavior as deprecated in the change record. The JSON test needs to be updated to assert that the cookie value is fully encoded for BC.
Comment #50
mradcliffeI wasn't successful in fixing the bc break yet, but I'm uploading my work-in-progress patch as well as an update test-only patch that should pass on 8.9.x or 9.0.x or whatever as-is.
It took a bit more than expected in my previous comment. The encoding went through the parseCookieValue function in jquery.cookie rather than just decodeUriComponent.
In this patch there is an issue in the converter.write as it is either not using it for JSON or something else.
Comment #51
mradcliffeFrom Slack, @laurii mentioned "Is there any downsides in keeping those dependencies? Maybe we could move removing the dependencies to a follow-up?"
I don't think there are any downsides to doing #44-1 (other than confusion N years from now), and definitely we could remove them in 10.0.0. I'm going to remove the review tag.
Comment #52
mradcliffeI needed to use withConverter in the setter as well as the getter. :-)
Since that's the case, I reverted some changes in #50 in the withConverter.
This now passes jsCookieTest locally for me.
Comment #53
mradcliffeI create a change record.
Comment #54
bnjmnmThis is looking very good!
This isn't a complete review yet as I've mainly only looked through the shim JS. I'm providing what I have so far as I may have spotty internet access for a bit.
I notice that that 'add' is used as an arg for two instances of
once(), which doesn't seem to be a problem in this case, but it can't hurt to reduce overall the number of calls toonce()by wrapping the click handler additions in a singleI noticed this is called only once and just calls to
encodeURIComponent(), perhaps I'm overlooking a specific reason for using this vs not callingencodeURIComponent()directly?Since this is largely a refactored-for-Drupal-coding-standards copy of
parseCookieValue()from jQuery cookie, could this get renamed toparseCookieValue(), accompanied by an explanation of what is different about this function vs the one in jQuery cookie (including why the try block was removed). This makes it easier to confidently review since it'll be apparent it's logic that has been in use in Drupal for many years.The return values for a setter call are not entirely accurate since it appends the attribute values ex:
"myCookie=myCookieValue; path=/". This is mentioned in the description for @return but could be clearer in these examples.Remove the leading dot from '.myCookie'
Since there are only two potential elements and they're immediately assigned to variables in the function, perhaps it's more readable to use
A few of the lines in this comment can have one additional word. I noticed the line breaks happen at the 79th character in a few spots so I'm guessing this is based on the cursor position
afterthe final character, which can be 81 since the character appears in column 80.'return' can go on the previous line
Comment #55
bnjmnmA few additional things to finish up the review:
In my previous comment I mentioned that this may not need a dedicated function -- now I'm wondering if the intent was to add a condition based on
$.cookie.raw, since jQuery cookie checks the value of raw when encoding?.
'URI' can get moved to the prior line.
This is technically removing a browser cookie.
Perhaps change "undefined" to "successfully removed"
This is the console output of what I did to check this.
$.cookie('key', 'value')returns a differently formatted string than when calling jQuery Cookie. I did a search of http://grep.xnddx.ru/ to see if the return value of cookie setting is used in any contrib and there are no instances of it. At the very least the change record should mention this, but it would still be ideal for the shim to return the same format of string so it's truly backwards compatible.Comment #56
mradcliffeThis patch addresses the minor standard review fixes and attempts to make the shim more backwards-compatible with jquery.cookie quirks.
The raw parameter had two functions in jquery.cookie:
1. When setting a cookie, it would not uri encode the cookie name, but had nothing to do with the cookie value.
2. When getting a cookie, it would not uri decode the cookie name and value.
The reason why the encoding is different in 55-4 and 55-5 is that the json option wasn't used and an object is passed in. In jquery.cookie, values are typecast to string unless the json option is true. js-cookie has a different behavior to always parse JSON, and if the output made sense, use it. To solve this, the shim will now only pass in an object if the json option is true, otherwise, the cookie value will be the string representation of the value, which matches jquery.cookie.
The patch does not address the path of the cookie. The default option should probably be to not use js-cookie's default '/' for the path. This is needs review, but is still pending because of that. The draft change record still needs to be updated to manually set path.
Comment #57
mradcliffeThis patch reverts path to always be an empty string to be backwards-compatible. The behavior of path when empty differs between Firefox and Chrome, and I was unable to write a Nightwatch test to assert the path using ChromeHeadless. The path always came back as / even though I manually tested in the browser and checked storage that the path was set to the current page correctly.
Comment #58
xjm#57 has some JS test failures:
Comment #59
mradcliffeI'm not sure why this is now failing on drupalci. It still passes locally for me. I went and applied the patch manually again too. :(
I don't think concurrency is an issue here because it's the same test.
Comment #60
lauriiiI'm wondering if this is an async problem. I tried to convert this to use a BDD type expect which would retry until certain time has passed. Unfortunately, it seems like Nightwatch doesn't have API for checking if cookie exists or not so we would have to add that ourself. I opened upstream issue so that this could be added there too eventually https://github.com/nightwatchjs/nightwatch/issues/2304.
Comment #61
lauriiiIt seems like we're not using the same default values in the
$.removewith$.cookie. This seems to work fine for some reason when Drupal is being run on root path, but DrupalCI runs Drupal in a subdirectory. I could reproduce the problem manually while running Drupal in a subdirectory. This should fix the tests too.Comment #62
damienmckennaTagging as a requirement for Drupal 9.0-beta1.
Comment #63
lauriiiCould we expand the test coverage to be closer to what is covered by https://github.com/carhartl/jquery-cookie/blob/master/test/tests.js? I'm still concerned that this change would end up changing behavior in some edge cases. Writing test coverage seems like the only way to catch changes in those.
Comment #64
finnsky commentedUpdated patch with multiple tests based on jQuery.cookie tests coverage.
Comment #65
finnsky commentedsmall quickfix
Comment #66
finnsky commentedlocally that test was passed. not sure why.
Comment #67
lauriii@finnsky did you try running tests when Drupal is installed in a subpath of the host? That was the reason for the DrupalCI failures in #57.
Comment #68
finnsky commented@lauriii i just runned it on local php server as described here
https://git.drupalcode.org/project/drupal/blob/8.8.x/core/.env.example#L11
Comment #69
mradcliffeThank you for moving this issue forward @lauriii and @finnsky. I found a couple of minor issues.
Patch doesn't apply anymore because of #3086369: Deprecate matchMedia. Needs re-roll. I was able to apply the patch with
git apply -v -3 <PATCH>and accept both changes.I ran core:lint-js and got the following
I think this needs the eslint override and the comment that explains why we need Nightwatch to deviate from the code standards.
I didn't get the same error in test bot, but I did get a test error when I tried running locally.
TypeError: Error while running "getCookie" command: Cannot read property 'value' of nullI think this is because that this is not expected to be a boolean or string. It needs an explicit undefined and/or null check in JavaScript.
This would be fine in TypeScript.
Maybe also
This seems odd to me. Shouldn't this be another
.execute(getJqueryCookiecall?I don't think the Nightwatch getCookie method will work here. This might be the reason the test is failing.
Comment #70
mradcliffeThese cookies are leaking in the test because this is all in the same browser session.
The "js_cookie_test_remove" cookie in the failing test is removed, but these other cookies still exist.
The SIMPLETEST_USER_AGENT cookie probably should remain.
So I think instead this should be
Comment #71
finnsky commentedseems we have to skip this test case https://github.com/carhartl/jquery-cookie/blob/master/test/tests.js#L151
Comment #72
lauriii+1 to #71.
Comment #73
mradcliffeI moved the remove cookie test to the first test as a clean up, and made the code standards fixes.
Also I found that the non-es6 shim file needed an update.
Should this issue be critical to match its parent?
Comment #74
mradcliffeWhoops.
Comment #75
mradcliffeYay, we're green again! :-)
I updated the issue summary's remaining tasks
Comment #76
mradcliffeBumping priority based on parent issue being critical.
Comment #77
lauriiiThe test coverage looks much better now which should give us some confidence as we move forward. 👍
Should we also deprecate this library?
We can probably remove this documentation?
Object.assignisn't properly supported by IE 11 and Opera Mini. We should provide a polyfill for that. We already have a polyfill for this in #3108402: Update to Popper.js to 2.0.0 so we could copy it here too.Should we use
strictEqualhere?Comment #78
bnjmnmThis addresses 1,2,3,5 of #77
Item 4 is unnecessary as #3108402: Update to Popper.js to 2.0.0 landed which provides the polyfill.
Comment #79
lauriiiWe still have to add the polyfill as a dependency to ensure that it's loaded 😇
Comment #80
bnjmnm🤦♂️oh of course!
Comment #81
mradcliffeAwesome! Thank you.
I reviewed the patch in #80 and it resolves the code review items in #77. js-cookie now has the polyfill as a dependency, the deprecated messages are more specific, and the extraneous comment block has been removed.
I added a release notes snippet for review.
Comment #82
mradcliffeAdjusted the snippet to be clear that there is a new core js library.
Comment #84
bnjmnmAdding the deprecation to core/jquery.cookie resulted in the many test failures in #80.
Added this to the skipped deprecations list as core/drupal.form and core/drupal.tabledrag still declare core/jquery.cookie as a dependency. They don't actually use the library, but removing it would impact BC for contrib/custom projects that expect cookie to be available when form/tabledrag are used.
Added a comment to each library declaring core/jquery.cookie as a dependency to make it clear this dependency can be safely removed.
Comment #85
catchI don't think we need to do this - contrib projects should be setting their own dependencies, the presence or not of a particular library on a page isn't an API we need to provide bc for.
Comment #86
mradcliffeI think our concern is that a contrib or custom/private module/theme may not even know that they need to explicitly depend on jquery.cookie because their use of this library gives them access to it.
Comment #87
catch@mradcliffe yeah that's really not something we need to provide bc for.
See also #3113400: Deprecate more jQuery UI library definitions
Comment #88
mradcliffeOkay, so if we don't need to keep jquery.cookie as a dependency where it's not in-use, then we can remove those entries like in #50, but added back in #52, @catch?
So we can remove not only the comments but the libraries here because these are not used in core.
And the Nigtwatch tests should still pass because jquery.cookie is added as a dependency for the test module's library.
Comment #89
catch@mradcliffe yes #50 looks more like I was expecting.
Comment #90
bnjmnmGlad to get some clarification that it's acceptable to remove these dependencies - I wasn't 100% sure and this makes for a cleaner core.libraries.yml.
The one part I wasn't sure about was the joyride dependency -- it doesn't need jQuery cookie, but it will use it when available. Removing jquery.cookie could result in an unexpected functionality change. Because I'm not certain how critical it is to continue including the jquery.cookie library with Joyride, I provided two patches, one that removes the library from Joyride (and as a result doesn't need to modify the deprecation skip list), and one that keeps it there.
Comment #91
catchI would consider making jquery.cookie or our bc layer for it a direct dependency of the jquery.joyride library definition - that would maintain the same behaviour without requiring us to add anything to the skip list.
Looks like there's an open and active issue for replacing jquery.joyride.
Comment #92
bnjmnm#91 sounds good
Diffing from the version that still included core/jquery.cookie with Joyride.
Comment #93
mradcliffeI updated the change record based on #91 and #92.
Comment #94
catchNo comment on the JavaScript but the library deprecation stuff all looks good to me now.
Comment #95
lauriiiIt doesn't seem like we have any test failures with the current code but we are throwing a deprecation warning in the JavaScript code as well. That would lead into deprecation warnings on a standard Drupal installation (assuming JavaScript deprecation warnings have been turned on). Are we okay with that?
Comment #96
xjmComment #97
catch@lauriii what if we made jquery.cookie the direct dependency instead of the bc layer? Then it wouldn't throw any deprecations.
Comment #98
lauriii@catch Unfortunately, that wouldn't work because they are both loaded globally to the same interface
$.cookie.Comment #99
catchSo I would lean towards js deprecation errors being an acceptable trade-off for not having stuff in the skip-list, but I am not really sure either way to be honest.
Comment #100
xjmLet's also add the dependency evaluation for the new dependency: https://www.drupal.org/core/dependencies#criteria
Comment #101
xjmI tried reviewing the patch WRT the JavaScript deprecation but couldn't tell what the JS deprecation error that would be thrown is.
Drupal.deprecationErroras per https://www.drupal.org/core/deprecation#javascript ?Comment #102
catchIt's not, that's a mis-speak from me. The current patch has no PHP deprecation because jquery.joyride is updated to use our jquery cookie bc layer directly (i.e. not via the deprecated jquery.cookie library definition). I was thinking if jquery.joyride used the actual jquery cookie js instead of a shim, but you can't use both, and it's being removed from core here.
So the choices for joyride are:
1. PHP and js deprecation errors
2. Only js deprecation errors (current patch)
3. Whatever joyride does for a fallback when jquery.cookie isn't available, no deprecation errors but would need manual testing or similar.
Yes it's a bc layer we're adding in 9.x, for removal in 10.x. But we can only remove it in 10.x if we do #3051766: Deprecate and replace jQuery Joyride (for tours), or option #3 above.
Comment #103
catchThis is the deprecation that gets triggered.
Comment #104
xjmIsn't it a premature deprecation then, that belongs in the other issue?
Comment #105
catchIt's a bit borderline.
No core JavaScript triggers the deprecation, and we'd want to inform contributed modules to use the new cookie API. In that sense the deprecation isn't premature.
But joyride we can't fix the deprecation in, we can only replace it entirely.
We definitely could remove the deprecation message from the bc layer, then add it back once joyride has been replaced, again for removal in 10.x
Comment #106
xjmThis seems like the right approac to me. For now we could add a
@todo.Comment #107
catchOK moving to needs work for that.
Comment #108
xjm@bnjmnm is actively working on this now.
Comment #109
bnjmnmReplaced the messages in the shim with a @todo, will be adding the dependency evaluation next.
Comment #110
mradcliffeThis is fine, right?
I think these tests will fail now because t hese should be removed too?
Comment #111
bnjmnmThis is a draft of the dependency evaluation. I'm particularly unsure about #2 so I'm not adding it directly to the issue summary. It may require a formal inquiry like this one, which may be best delegated to someone who speaks security.
Here's the draft:
Comment #112
xjmFor the security policies, I would indeed post a question in their queue. We can post an issue like https://github.com/popperjs/popper-core/issues/823 (and additionally, close with a suggestion that they add documentation of the security policy at https://github.com/js-cookie/js-cookie/security/policy).
That doesn't necessarily need to block this issue going in since it's sort of an upgraded version of a dependency we already have, but we can at least start a discussion.
Comment #113
mradcliffeChanging status to Needs work based on Nightwatch test failure. There seem to be some random failures in HEAD too?
Comment #114
xjmYes,
WidgetUploadTestandQuickEditIntegrationTestboth have known random fails, so we can ignore that part for now.Here's the Nightwatch output:
Comment #115
bnjmnmYep, forgot about Nightwatch. This removes the tests looking for the removed deprecation messages.
Comment #116
xjmAdding the draft dependency evaluation from #111 to the IS so we don't lose track of it, but leaving the tag on for posting the issue as mentioned in #112. We can ask them both about their security policies and about how long 2.x will continue to receive security coverage (if at all) after 3.0.
Comment #117
mradcliffeI read through the dependency evaluation, and changed "a year. are . " to be "a year. Releases are available at ".
Comment #118
catchI think this is probably the most honest version we can do:
1. We've added a new cookie library, which can be used without triggering any deprecations.
2. We've added a shim for bc with the old jquery.cookie API with its own library definition. Modules using the old library definition will get a deprecation notice in PHP.
3. Joyride relies directly on the bc layer shim (not via the library), because this is a third party library we can't update ourselves - only replace (which has an open issue). This doesn't trigger any deprecation errors in either PHP or js - to avoid giving site owners warnings they can't do anything about.
Once Joyride is replaced (or if we decided to let joyride run without cookie support and rely on its fallback behaviour), we can add the JavaScript deprecation in a 9.x minor release for removal in 10.x (or really it would be OK to just remove the library in 10.x with the existing PHP deprecation).
Note that js.cookie just released 3.0.0-beta4, so we should probably open an issue to evaluate whether it's possible to update to 3.0.0 prior to release if a stable version comes out - will depend on the impact of any changes.
Still need the issue asking about security support before we can commit here.
Comment #119
bnjmnmInquired about documentation of the security policy here https://github.com/js-cookie/js-cookie/issues/614
CC attribution to @wim-leers as it was based on his PopperJs inquiry 🙂
Comment #120
catchThanks for opening that. Removing release management tags.
Comment #121
xjmTagging for followup for js.cookie 3.
Comment #122
xjmAdding to the IS.
Comment #123
xjmComment #124
catchPosted #3118726: Upgrade to js.cookie 3.
Comment #125
xjmUpdating the release note based on the maintainer's responses so far about their support policies and version 3; see #3118726: Upgrade to js.cookie 3 for the rest.
Comment #126
tedbowreviewing
Comment #127
tedbowyarn prettierandtests/Drupal/Nightwatch/Tests/jsCookieTest.jschanged. Should we still run this for all js changes?I see
// prettier-ignoreis used in this file. Should use before this code which changes when it is run?It seems like we could just let prettier change it but not sure
< /li>
Otherwise it looks good
Comment #128
tedbowwhoops didn't mean to assign it berdir
Comment #129
bnjmnm#127 is right, this needed a prettier and build:js
Comment #130
tedbowThese files weren't in the last patch.
I did notice that they were updated when prettier was run but I guess that is an existing issue. I think we just have to run prettier and then restore these.
Comment #131
bnjmnmPolyfills unprettified.
Comment #132
tedbow@bnjmnm thanks for the updates. It looks good to me! RTBC, assuming tests pass
Comment #133
mradcliffeI went through js-cookie v3 changes, and I don't think we would need to do much to keep compatible.
Major upcoming changes:
- Path is now settable via the API
- URL-encoded cookie values may change existing cookies.
- getJSON removed
- IE10 support removed
- defaults removed in favor of withAttributes factory method.
I think this would make the jquery.cookie shim more light-weight in v3.
The biggest concerns are getJSON removed, solved by not recommending its usage in the change record for people using Cookies directly. And any encoding changes as that could affect existing cookie data. The latter may necessitate a separate shim.
Comment #134
mradcliffeI was testing the patch locally and I noticed
TypeError: cookies is undefined
I updated to latest 9.0.x, applied the patch, ran database updates, cleared cache again, and when I went to /admin, the library was loaded, but window.Cookies was undefined.
I didn't try a fresh 9.0.x install, and I'll do that next to make sure it wasn't something in my local environment.
Comment #135
mradcliffeYes, i can confirm this on a fresh 9.0.x install.
- As a privileged user, go to /admin.
This is throwing the error because js-cookie isn't a dependency.
This library is working and not throwing errors. And so Nightwatch is passing.
I guess a further test could go to /admin, and assert no console errors.
Comment #136
mradcliffeSo basically in order to do the suggestion in #91, we also need to add js-cookie and the polyfill. Order might matter there.
Comment #137
catchIf we add js-cookie as a library dependency, that should be loaded before the polyfill, so it ought to work - needs manual testing though and agreed a sniff-test of browsing to admin and no console errors would be a test coverage addition.
Comment #138
catchI would also be fine with moving #91 to a follow-up if it's actually making things complicated here. We are still working on a similar strategy in #3113400: Deprecate more jQuery UI library definitions but nothing doing that trick has been committed yet.
Comment #139
bnjmnmAssigning to myself to work on fixing #135, this will get tested manually, but adding a test to look for console errors seems out of scope (as helpful as it would be to have it). There's an existing issue to check for console errors in tests: #3091870: Fail Functional Javascript tests that throw Javascript errors. That issue probably deserves a priority boost.
Comment #140
catchI didn't realise we don't fail on JavaScript errors, bumped #3091870: Fail Functional Javascript tests that throw Javascript errors to major for now. If we don't have test infrastructure for this then agreed it doesn't make sense to block this issue on it and manual testing should be OK (otherwise we'd never make any js changes at all).
Comment #141
lauriii+1 for not adding test coverage for JavaScript errors here and bumping the follow-up to major. Replacing the needs tests tag with needs manual testing tag to make sure that the patch gets manually tested with browsers that are targeted by the polyfills.
Comment #142
mradcliffeI manually tested both #136 and #137, but adding a Nightwatch test has proven tricky due the nightwatch testing profile. Tour on its own does not provide any tours so the error comes from another core module with a tour.
Before @laurii's comment, I did add a test so I've attached the patch.
The patch here adds a test that installs tour and tour_test in the Nightwatch test. Logging in as an admin slows the test down so I opted for a non-admin tour page.
The test-only patch here is the entire patch with only the additional test.
Edit: +1 on manually testing on IE.
Comment #143
mradcliffeUpdated issue summary with details about manual testing and linking to #3118726: Upgrade to js.cookie 3.
Comment #144
bnjmnmManually tested #142 on Chrome and IE (so I could test with the polyfille), and there were no problems.
I spotted one small thing:
It would be better to add js-cookie as a library dependency since it's not a deprecated library, and this ensures it is always loaded first.
Comment #145
lauriiiI almost crossposted with #3 saying that we should depend on the library instead so +1 on that 🤠
I don't have anything against having extra test coverage. Just would prefer the lack of additional test coverage to not delay this issue from getting done on time 😇
Comment #146
mradcliffeAddressed review by @bnjmnm, and added js-cookie as a dependency instead of a file for jquery.joyride.
Comment #147
bnjmnmLooks like the test-only patch is passing though it shouldn't be. If it's fairly clear what is happening it can be fixed in this issue. I think it's also acceptable to remove it since #3091870: Fail Functional Javascript tests that throw Javascript errors would also provide coverage for that error (and has been priority-bumped) and #145 also pointed out that it's not necessary to delay this issue with test coverage.
Comment #148
mradcliffeAgreed. It was failing for me locally, but I guess something else was going on.
Comment #149
mradcliffeI have an hour or so. I'll try to get a patch here in the next few minutes.
Comment #150
mradcliffeRemoves the test in this patch.
I added an interdiff between @bnjmnn's work in 131 and 150 as well as an interdiff between 145 and 150.
Comment #152
catchUnrelated test failure.
Comment #153
tedbowinterdiff-2550717-131-150.txtlooks good to me. I agree that we can wait on #3091870: Fail Functional Javascript tests that throw Javascript errors to add the test coverage.I just wrote the patch on #3091870. It doesn't currently apply but if we could get it rerolled writing a FunctionalJavascript test method should ver simple. I don't think we would even need a test module. Just enable tour and goto /admin. Not sure about a Nightwatch test but we should be able to prove it with FunctionalJavascript for now.
Waiting on test to RTBC
I chatted with @bnjmnm and he tested #150 on IE11. I think he will confirm here.
I tested 131 and confirm the error and 150 to confirm it doesn't happen, on chrome for Mac
Comment #154
bnjmnmCan confirm #153, I tested #150 in IE11 and there is no error and the polyfill does its job.
Comment #155
mradcliffeRemoving Needs manual testing tag.
Comment #156
tedbowok back to RTBC!
Comment #158
catchSorry wrong issue, although one tag was nearly applicable so fixing back to that.
Comment #159
lauriiiNit: should we use the
%library_id%placeholder here?Shouldn't the jQuery Cookie shim be loaded before joyride?
Let's also add a comment that clarifies why we're loading the file manually, pointing to both, this issue and the follow-up.
Nit: let's remove the word JavaScript from here since the cookies are not specific to JavaScript even though they are set in JavaScript. If we feel that it would be unclear to reference to just Cookies, we could call them as browser cookies which we're already doing in
$.removeCookiedocumentation.Nit: Could we add a @see link to the docblock that references the README.md?
Should we link to the change record here?
Nit: I guess another option would be to make Joyride not depend on jQuery cookie. Could we rephrase this to be a bit more open ended to not close that door?
Comment #160
bnjmnmWorking on #159
Comment #161
bnjmnmWhile working on the feedback in #159, I discovered that removing joyride's dependency on jquery.cookie (or the shim) has minimal consequences.
jQuery cookie is used by Joyride for one reason: if a tour configured with
{autoStart: true}has been viewed, this value is stored in a cookie so the tour does not autostart on future visits to the page. If jQuery cookie is not available, Joyride falls back to using localStorage - which works but has less longevity than a cookie.Tours created with the Tour module would not be impacted at all, as the Tour module API does not expose auto-start. The only potential impact is to contrib/custom modules/themes that use the core/jquery.joyride library directly and have configured
{autoStart: true}. Those impacted that don't wish to use the localStorage fallback can make jQuery cookie available via a custom library, and functionality will be unchanged.This means we can reintroduce deprecation messages in the shim.
Comment #162
mradcliffeI thought based on #118 we would continue to have jquery.joyride depend on jquery.cookie.
Comment #163
bnjmnmThis addresses all but one feedback item* in #159 other than that superceded by the realization in #161 that joyride did not need to depend on jquery.cookie.
*the one unaddressed item is #159.7 -- expanding tests to cover cookie expiration. My local nightwatch setup is not cooperating and I didn't want my other changes to wait on those as there's a chance another contributor (with working local nightwatch) may be able to take care of that. If someone is available to do that perhaps they can let me know via Drupal Slack so I know I can attend to things other than my local nightwatch.
Comment #164
bnjmnmRe #162
Discussed this with @catch upon discovering just how minimal the impact would be, and he agreed it would be fine to remove the dependency.
Comment #165
catch@mradcliffe that was before we investigated exactly what joyride's fallback behaviour was and whether it would change behaviour for core or contrib, and it turns out nothing will be affected (and there's a 1-5 line fix for contrib or custom code we can put in the change record in case it turns out someone is relying on the cookie behaviour somewhere).
Comment #166
mradcliffePing @bnjmnm on Slack. I have Nightwatch working and an hour so I'll try writing the cookie expiration test.
Thank you for the clarification, @catch.
Added Needs tests tag again.
Comment #167
mradcliffeI added three tests from jquery.cookie:
1. expires option as days from now -> Test jquery.cookie Shim expires option as days from now
2. expires option as fraction of a day -> Test jquery.cookie Shim expires option as fraction of a day
3. expires option as Date instance -> Test jquery.cookie Shim expires option as Date instance
The sevenDaysFromNow variable in the "expires option as days from now" original test is inaccurately labeled because it's incrementing 21 days rather than 7, so I changed it to "daysFromNow".
The third test is failing. I think that the Object.assign polyfill or native function is doing something to the native Date object so that it no longer has the toUTCString method. I don't have any more time to work on it at the moment so uploading a new patch.
I also found the following lint error, but I had trouble addressing it:
Assigning { path.: '' } to a separate variable and doing Object.assign(var, cookies.defaults) failed the entire test run.
Comment #168
mradcliffeBack to Needs work for the failing. test.
Comment #169
bnjmnmThis is great @mradcliffe! Turns out the test was failing due an issue with the shim, so it did its job quite well. This patch fixes the shim and ran
yarn prettieron jsCookieTestComment #170
lauriiiThere's an ESLint warning with the patch:
Comment #171
longwaveFixed #170.
Comment #172
tedbowI looked at all the comments since my RTBC in #156 everything looks like it has been addressed
The test coverage @mradcliffe added in #167 look great. Thanks!
It exposed a problem in the shim which @bnjmnm addressed in #169
I ran
everything looked good(prettier did affect 2 files but not that were changed in this patch)
I went back through the patch again and everything still looks good. RTBC 🎉
Comment #173
lauriiiThis test is failing locally:
This could be related to timezones / daylight savings since the hour is off by one hour. If I use a date that is before 29th of March (date when daylight savings starts in my timezone) the test passes.
Comment #174
longwaveUnable to reproduce the fail above at the moment, but I did manage to trigger a different issue with the dates, where the seconds are out by 1 - I think this happens when the second ticks over between the two date operations:
Comment #175
bnjmnmSomething that may narrow down the issue: The test where a date object is passed as the value of
expire:passes. The failure happens in the test that passes expire as a number.Comment #176
bnjmnmIn an approach endorsed by @lauriii and @catch, this patch removes the two tests that pass in numbers as expiration values.
The test runner and browser are two different environments, and those environments can potentially calculate a different
Date()value from the number provided. Both #173 and #174 are evidence of this. There is too much risk of those causing intermittent failures that have nothing to do with the shim itself for them to remain in the patch. I'll create a followup to find a reliable way to test those use cases. (The test that adds a Date object as the expires value can remain, as it is not calculated in two separate environments.
Comment #177
bnjmnmFollowup: #3119727: Expand jsCookieTest to include numeric expiration values
Comment #178
tedbowThe idea in #176 look good. I looked at the follow up and it looks good.
Back to RTBC. 🤞that tests will pass since it is only removing 2 test cases!
Comment #179
lauriiiThis test has the same risk as the tests removed in #176. Let's move this to a follow-up as well. Feel free to mark this issue as RTBC once this has been done.
Comment #180
lauriiiRemoving the needs subsystem maintainer review tag since we haven't received a review from the subsystem maintainers in 4 months. I didn't have any particular aspect in mind when I asked a review from them on #31, but I wanted to make sure we remember to reach out to them to give them a chance to look at this if their time permits. We have a sign-off from a subsystem maintainer on the overall plan in #16 which is great. We can address any further feedback from the subsystem maintainers in follow-ups. ✌️
Comment #181
bnjmnmThe test in #179 does not need to be removed, it's not subject to the same concerns as the other tests that were removed. That test send a specific datetime that is calculated in only one place, as opposed to the removed tests that provide a numeric value, which means the date is calculated by the test runner AND the browser. Specifying a time should be fine, it's specifying a distance from the current time that is a concern. (plus its nice to have expiration somewhat covered by tests 🙂)
Comment #183
lauriiiAwesome, #181 addresses my concerns in #180.
Committed b389f67 and pushed to 9.0.x. Thanks!
We probably want to backport this to 8.9.x, but we need a separate patch for that since the patch doesn't apply as it is.
Comment #184
bnjmnmHere's a patch for 8.9.x. It's pretty similar but required adding the object.assign polyfill, which is already in core in 9.x. Was a little uncertain about the "Deprecated in 9.0.0" deprecation message but left as-is as it's an easy enough change for anyone if it should be something different.
Comment #185
lauriiiDiscussed with @catch and @xjm how we should handle the deprecation. We agreed that we should remove the deprecation errors from the 8.9.x patch because this is supposed to be deprecated in Drupal 9.0.0, not in 8.9.0. On top of that we should open a follow-up to discuss if we should add the deprecation error back in 8.9.x.
Comment #186
bnjmnmThis removes the deprecation messages and refactors a bit of JS due to the Babel version being different in 8.9 than it is in 9.0. Created a followup to discuss the message: #3120309: Discuss jquery.cookie deprecation message in 8.9
Comment #188
lauriiiAdded
core/misc/polyfills/object.assign.es6.jsto the eslint ignore file to make ESLint pass.Committed 690b076 and pushed to 8.9.x. Thanks!
Comment #189
droplet commented`Object.assign`, the first arg is target, 2nd is source.. `options` will be merged into `$.cookie.defaults` and changed the defaults for the whole env.
https://github.com/carhartl/jquery-cookie/blob/7f88a4e631aba8a8c688fd899...
It should be
a few places above may have similar problems.
And as a shim.. I think to keep the same functions (code logic) better than upgrade it to ES6 (no jQuery).
Comment #190
bnjmnmThese address #189
Comment #191
tedbow#189 @droplet thanks for catching this.
This seems right to me. confirmed with this behavior.https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global...
the fixes #190 look good. Thanks @bnjmnm
Not sure what we want to do here. But I consider the fix RTBC.
Comment #192
lauriiiThe patch leads to following eslint warnings:
After we fix this, the
Object.assignisn't needed anymore for the shim so it can be removed from the dependency list and 8.9.x since it's no longer being used there.Comment #193
bnjmnmThe 8.9 patch from #190 shouldn't change due to differences in JS dependencies. The transpiler in 8.9 does not support object spread, and prefer-object-spread is not an eslint rule.
9.0 does need this change (and so does my PHPStorm lint config, incidentally).
Comment #194
bnjmnmCorrecting a stray dot that snuck into a docblock.
Comment #195
tedbowI ran
Everything as expected.
Re #193
So against 2550717-190-89x.patch on 8.9.
I also ran
Everything as expected.
So since @bnjmnm is saying in #190 that we can't actually change the 8.9.x patch I won't expect to see changes related to this but for the 9.0.x patch shouldn't there be changes in core.libraries.yml
Comment #196
bnjmnmRe #195.4
The object.assign polyfill can stay in 9.0.x, it is still a dependency of popperjs and was not introduced in this issue.
Comment #197
bnjmnmIt was clarified that this means the object.assign polyfill no longer needs to be a dependency of core/jquery-cookie.
Comment #198
bnjmnmAnd for 8.9, the object.assign polyfill can be a dependency of core/jquery.cookie instead of core/js-cookie. Object.assign is only used in the shim, so js-cookie itself does not require it.
Comment #199
tedbow#197 looks good!
#198 looks good too!
I ran all the yarn checks also that I did in #195 for both patches.
Comment #204
lauriiiCommitted 97b964b to 9.0.x and cff472e to 8.9.x. Thanks!
Comment #205
xjmI published the CR.
Comment #206
xjmComment #208
avpaderno