Follow-up to #1825952: Turn on twig autoescape by default
Problem/Motivation
In system_requirements(), we mark as safe a long $description variable containing concatenated t() calls, whitespace, <br /> tags, drupal_render()calls, etc. We also mark the php version text as safe. As a best practice, , we should not use SafeMarkup::set() around a variable.
Beta phase evaluation
Proposed resolution
Many resolutions were discussed. The path decided on is to use a render array, leave the uses of t(), and remove the SafeMarkup::set call.
Follow-up for other approach: #2505499: Explicitly support render arrays in hook_requirements()
Remaining tasks
Update summary explaining various proposed resolutions.
User interface changes
None.
admin/reports/status
Manual testing steps
- Confirm no change to the markup at /admin/reports/status "Cron maintenance tasks" section
- Confirm no change to the CLI output by running "drush rq" and looking at the "Cron maintenance tasks" section
- Repeat the above two tasks after changing
core/modules/system/system.install to think there has been an error by adding this line: 433 $severity = REQUIREMENT_ERROR;
head


API changes
None.
Comments
Comment #1
xjmIssue clone fail.
Comment #2
xjmUpon reflection, that patch is probably totally the wrong way to do this. We should look into why this got added here in the first place; maybe double-escaping some
<br />tags that we maybe shouldn't be adding anyway?Comment #3
xjmComment #4
jibranComment #5
arpitr commentedComment #6
arpitr commentedI wonder if this still needs a change as SafeMarkup::implode() does not seem to exist any more.
Comment #7
aneek commented@arpitr, no the function name you mentioned is not available now. So we have to re-create a new patch.
Comment #9
aneek commentedUploading a new patch. Review!!
Comment #10
arpitr commentedI think we should update the issue as well as the solution does not go inline to what was proposed.
Comment #11
star-szrJust a suggestion: Would it be possible if we're going with inline template to actually construct the description in the inline template, in other words do all the concatenating and such inside the #template?
Coding standards: spaces needed after
//ref. https://www.drupal.org/node/1354#inlineTwig coding standards: Missing space inside closing curly braces. Ref. http://twig.sensiolabs.org/doc/coding_standards.html (Drupal's Twig coding standards are at https://www.drupal.org/node/1823416)
Comment #12
lauriiiFixed the standards
Comment #13
aneek commentedMy bad @Cottser. Thanks for the proper review. And thanks @lauriii for saving me some time ;-) ha ha... Making this RTBC to get alex's review.
Comment #14
aneek commentedComment #15
alexpott@aneek - it's not customary to rtbc patches you've worked on - unless the patch has completely changed or was rtbc for very very minor nits.
Comment #16
aneek commented@alexpott - noted. Well then someone else needs to set this as RTBC if review is fine.
Comment #17
herved commentedHello,
I noticed that the output in
drush rqis wrong:Drush expects a plain string and gets the inline template for the description and therefore prints it as "Array".
Attaching a patch that fixes this + fixes some spaces issues after/before sentences in
drush rqas it strips tags.Comment #20
herved commentedNot sure what happened here with the testbot O_o
Comment #21
aneek commented@herved awesome!! But can you post an interdiff?
Thanks!
Comment #22
herved commentedSure here it is.
Comment #23
pfrenssen$description actually doesn't contain any Renderer::render() output at this point.
I've tested it and it works fine when passing the $description render array directly.
If this doesn't work with
drush rqthen this is actually a bug in drush.I would put $description here.
Comment #24
pfrenssenReported the issue in Drush: Issue #1314: drush core-requirements does not output descriptions in render arrays.
Comment #25
mile23I'm not sure about the applicability of the patch, but this issue needs a summary update, as well as a beta evaluation.
Here's some things to consider:
Not a fan of re-using the variable.
This might be a little out of scope, but this line needs array reformatting for coding standards. Pull it out to multiple lines. https://www.drupal.org/coding-standards#array
Wrap on 80. The comments also need some punctuation.
Comment #26
codemonkie commentedPulled long line into multiple lines and actually found a bug.
The 3rd argument of the call to Drupal::url() ('absolute' => TRUE) never reached the method call because of a parenthesis mixup.
Removed comment regarding using the Renderer::render() method since it is not applicable.
Comment #27
xjmComment #28
xjmComment #30
codemonkie commentedOld patches failed applying, created a new patch with same changes.
(There was a period added in 8.0.x in one of the lines that is removed by this patch causing it to fail applying)
Comment #31
star-szrSetting to needs review so it gets tested. Thanks!
Comment #32
codemonkie commentedUpdated the issue summary to reflect the @todo comment I found in the system.install file line 366 (see below)
Did some more reading on the issue (this thread and parent issue etc) and made some additional changes:
- constructed the description 'in' the inline template as suggested by @Cottser comment 11 moving the use of the
<br />into the template.- removed the reuse $description variable as pointed out by @Mile23 comment 25 sub 1
- removed the call to \Drupal::service('renderer')->render($description) as it is unnecessary as pointed out by @pfrenssen in comment 23 sub 2
So far I think this patch solves the issue present in the file system.install line 366
Btw, can someone please point out where/how I can see this code in action? (I am new to the drupal codebase)
Comment #33
alexpottNow we're using the inline template we can get rid of all of the creating t'd strings in PHP variables - which has the massive advantage of actually putting all the description test in one place.
@codemonkie you can see this text on the url
admin/reports/statusComment #34
alexpottOne of the spaces was not in the logically correct place - moved. This actually made no difference to how it looks on screen but it makes more sense.
Comment #35
xjmI like the approach in #34 more and more, but it's definitely different from how we've composed HTML for messages previously. People are for sure used to seeing
t(). It could be good to make it so people get used to seeing Twig, and it's certainly an advantage to have the markup, conditions, and translations all in one place. It definitely reduces the risk of composing HTML out of strings containing variables, and also reduces the chance that we'll end up with nested string format/translation calls.We need to keep in mind a few things if we adopt this pattern. One is translations:
This is changing the strings for translators. Previously, there were two meaningful strings: "Cron has not run recently" and "For more information,..." were separate strings for translators. Now we are merging them into one new string which will need a different translation. The second string is also used elsewhere in core, which means translators will have to re-translate almost exactly the same text. So it might be better to retain this as two separate translated strings, even though they are right next to each other.
Another consideration is readability.
The Twig template is all on one long line that is difficult to scan/parse. Would it be appropariate or feasible to break it up onto multiple lines, e.g. so the if and endif were on their own lines? Does that cause problems with whitespace?
Another consideration is non-HTML output of this string. What happens with Drush now? See #17.
A final, broader consideration is documentation. Is the documentation for inline templates available when you're reading the documentation for
t()andSafeMarkup::format()? This is out of scope for this issue, but if we're going to use inline templates as more than a temporary hack before one moves actual markup into a "proper" template, we should double-check that we have documentation in the codebase that explains the difference between these two methods of composing strings.I'd suggest testing this manually on
/admin/reports/statusand with drush. I also would double-check how the strings are available to translators. (The Twig trans tag is hopefully scanned exactly like t() is? But would be good to double-check if we're doing the translation inside an inline template.)If this works, and @Cottser is okay with it (maybe double-check with @Gábor Hojtsy or another multilingual contributor on the translation bit too), maybe this is a pattern we could use elsewhere for #2280965: [meta] Remove every SafeMarkup::set() call as well.
Comment #36
star-szrI agree that we probably should keep this particular translatable as 2 separate strings, at least for the scope of this issue.
As for how the trans tag is scanned, there is #2040089: Support for Drupal 8 twig %trans% template translatables but I'm not sure if that handles inline templates or not.
Things like
ifs,transblocks etc. shouldn't have a problem being on their own line. If need be whitespace modifiers can be used to trim any additional whitespace.By splitting up the template onto multiple lines we not only have easier to read code but we also have better odds at complying with Twig coding standards, upstream and our own.
Edit: And I'm not sure yet how I feel about widespread use of this pattern :)
Comment #37
xjm@Gábor Hojtsy linked: #2315329: Support for Drupal 8 inline twig translatable
Comment #38
effulgentsia commentedSee #2280965-20: [meta] Remove every SafeMarkup::set() call for my thoughts on
SafeMarkup::format()vs.inline_templatein general.Regardless of that though, I suggest especially not using inline_template, or any other render array, or invoking drupal_render(), within hook_requirements() implementations. Seems like an unnecessary and potentially problematic dependency for use cases that want to check requirements outside of the rendering pipeline.
Comment #39
David_Rothstein commentedAgreed. Also a lot of the problem with the existing code is that it goes against the translatability guidelines at https://www.drupal.org/node/322774 (at least the way I read them).
Fix that, and then the whole thing basically just becomes
SafeMarkup::format('@line_one<br />@line_two')which is relatively simple. See attached.Comment #40
alexpottI don't get it to be honest. If we shouldn't use all of drupal's toolset here and if we care about cli output why are we concatenating html in the first place?
Also the use of inline template should actually result in less happening when something external to the render pipeline invokes hook requirements.
Comment #41
star-szrYes the fact that there's HTML kind of messes up some of these assumptions unless we are also assuming that a CLI is going to strip tags and maybe "br2nl". If we are actually caring about CLIs or others then (out of scope) maybe we need to provide plain text versions of requirements?
The fact that there is logic needed to display this string (HTML or not) to me steers towards an (inline) template-based implementation. What if third or fourth lines are added to this string later? The inline template can scale much better to accommodate that versus a big concatenation.
In other words…
This just seems unfortunate.
Comment #42
effulgentsia commentedI agree that in general,
ifstatements that are purely front-end logic are better in Twig than in PHP. That might be a good guideline to explicitly add to https://www.drupal.org/node/2311123.What I wonder though is if that is better enough to require callers of hook_requirements() outside of Drupal's own Web UI to have to know that 'description' can be a render array and not just a string. Maybe yes, though we should recognize that that is an API change for the @return of that hook.
The difference is that if a tool like Drush outputs an HTML string to a non-HTML agent, the meaning of the message is still clear to the person reading it.
Arrayis not.I suppose for completeness, we should also consider if this is the kind of use case that option 3 (calling the Twig service directly) is best for?
Comment #43
David_Rothstein commentedI don't think there's any completely pretty solution here except the one that's already in place - SafeMarkup::set() :)
Which, with the other suggested refactoring, would actually look something this:
To be honest, I'm not sure there's anything wrong with that pattern? It's pretty clear from looking at that code why the concatenated string is known to be safe. I get that we don't want to encourage developers to decide on their own if something is safe... but maybe it's OK for really simple situations?
Comment #44
David_Rothstein commentedOr another option, just put the
<br />inside a single t() call (along with the sentences before and after it) and be done.The sentences are all related, so it's basically just a single paragraph anyway. It would be a little bit of duplicate work for translators to separately translate the two versions of that paragraph, but really not much.
Comment #45
effulgentsia commentedAs an example, drupal_check_module() calls
$message = SafeMarkup::escape($requirement['description']);, which would fail if $requirement['description'] is an array.But, looks like HEAD already has system_requirements() setting some other 'description's to 'inline_template' arrays. So this issue wouldn't introduce the problem, only add to it.
However, I checked all of HEAD's other (not system_requirements()) usages of 'inline_template' and they're ok in terms of only being used within a place where render arrays are expected. So my preference would be to explicitly decide here what we want as the @return of hook_requirements() and either fix the existing deviations or the existing callers as needed.
Comment #46
xjmThanks @David_Rothstein and @effulgentsia.
Radical notion, but why is the
<br />tag there at all? A before-and-after screenshot of whether it impacts the usability might be informative.Yeach. I really don't think we should do that. It sets an example that is, like, worst practice.
That aside, I think it's definitely legit to have HTML in
system_requirements()messages (and other messages). Not only whitespace but also things like lists, links, etc. And it seems to me that we already have an assumption Drupal 8 at this point that returning a render array should be equivalent to/interchangeable with returning a string of HTML. So my vote would be to update the documentation for the hook as @effulgentsia suggests (in a separate issue).I read @effulgentsia's comment on #2280965-20: [meta] Remove every SafeMarkup::set() call and @dawehner's response in #21 and will follow up over there, but what I think we should avoid is nested calls that add separate strings to the safe list and then their composite on top of that.
Comment #47
effulgentsia commentedI don't think we have that assumption at all. For example, controllers, BlockInterface::build(), FormatterInterface::viewElements(), and many other similar methods must return render arrays; returning an HTML string would fail. Similarly, children of render arrays must also be render arrays, not strings. I think the only place they're interchangeable is for render array properties (such as
#description) and variables prepared in preprocess functions.I also don't think we have any services that return render arrays, though I might be wrong on that. Though that's correlated with services also generally not returning HTML.
Currently, hook_help() can have all those things, but the @return is a string, not a render array. What other message-like return values should we consider if we decide to make the API change to
hook_requirements()[]['description']?Comment #48
codemonkie commentedMoved on in the direction @alexpott was going in #34 as there seemed to be a little more support for that and did the following work
To break the inline template into multiple lines you end up with somewhat of a whitespace war. This is because whitespace at the end of a line is not accepted and this is where you would typically put the whitespace. I ended up with a method where I explicitly defined a whitespace as a variable
space. Further I stripped the whitespace due to indentation with the '-' modifier. This results into nicely formatted output, so no extra whitespace whatsoever.Not sure if the whitespace variable is the way to go though. It does end up in a more readable template with nice output. I would like to hear more opinions on this.
Comment #49
David_Rothstein commentedYou removed the
<br />entirely. Not sure if that was intentional (it was actually a suggestion in #46) but if so there is definitely no need for a Twig template at all... It's just a single translatable string with two variations.Any thoughts on that (or also #44, just put the
<br />inside t())?It seems really weird to me to have a whole inline Twig template for something this simple. There is way more code logic inside that template than anything HTML-structure-related. I get that it might be different in the general case, although hopefully not too many people are putting long paragraphs or bulleted lists in what's intended to be a short description.
Comment #50
codemonkie commented@David_Rothstein, indeed, I forgot to mention the removal of the
<br />tag (as suggested in #46 by @xjm).I added some before-after screenshots for a comparison of the usability impact.
I agree with you that the use of a an inline template is like killing a fly with a hammer in this case, but maybe this is more of an example case.
I do see some advantages:
<br />there is also a conditional string, of which the render logic is now clearly visible.and some disadvantages:
@David_Rothstein, about combining the strings (#44), this results in new strings to be translated, so more work for translators. I'm not sure what the impact is of that.
I do think it is actually a better solution, because it gives the translators more control where and how to use the variables. The opinion of an experienced Drupal translator would help in this.
Comment #51
yesct commentedThanks for the screenshots. I think we would also want to see how it looks for the other condition. And someone asked also for testing with drush.
The inline template ... looks like too much.
I'm going to try using format again, keeping the logic and the translatable strings as is.
Comment #52
yesct commentedNo interdiff since it is a different approach.
With this approach:
======
Tested with the conditional string by adding
$severity = REQUIREMENT_WARNING;before the ifI added more context to the screenshots (mostly so I would remember that this is the status report page, and not the cron form, which confused me for a bit).
severity: warning
head
markup
patch
markup
severity: info
head
markup
patch
markup
=====
Note there is a difference in the markup for the case where severity is info, in head, there is a leading space due to, in head:
(No difference in markup for the severity warning case.)
@herved thanks for noting the drush command in #17
I also tested with
drush rq(drush requirements):in head
with the patch
[edit: php tag to show the html]
Comment #53
yesct commentedNow with patch.
Comment #54
kgoel commentedIt would be nice to comment that you are using these variables in the safeMarkup::format.
For better readability of the code, lets flip the order of variables.
Let's flip the order of $description_string and $description_replacements to be consistent.
Comment #55
kgoel commentedComment #56
yesct commentedaddress both of those points.
Comment #57
David_Rothstein commentedWhen I tried that above, the reactions included words like "unfortunate" and "yeach"...
I think these placeholder names are at least better than mine, though - mine were "@line_one" and "@line_two" :)
Comment #58
David_Rothstein commentedI agree. I was basing what I said on https://www.drupal.org/node/322774 (which basically implies that if a whole paragraph is translatable, it should all go inside the same t() call) but hearing from someone who actually does translations (which is not me) would be good.
Comment #59
xjmSo, I'm still uncomfortable with this pattern (of translated strings globbed together around whitespace inside a
SafeMarkup::set()) and I still think it would be worth considering:t()call<br />tag.hook_requirements()and the like.I'm leaning toward #2 (after checking with a translator, as suggested) and maybe #3, with more general followups for #1 and #4.
That aside, looking at the specific patch, I think we should always avoid this in particular:
$description_stringand$description_replacementsshould not be defined as separate variables. Even if we do use this pattern ofSafeMarkup::format(@thing_1 @thing_2)(which I don't think we should), the top-level arguments ofSafeMarkup::format()should not be variables. It makes the code more difficult to read, more difficult to security audit, and more likely to promote bad practices with string sanitization. We should continue to follow the same rule we use fort()andformat_string()in D7 that the first argument should always be a string literal and the replacement tokens should be clear as well.The less we compose variables across many lines, the better. It'd better to have multiple
SafeMarkup::format()ort()calls in different code paths if necessary, rather than composing strings across many lines of code.Thanks everyone for your patience here! It's definitely not a straightforward thing.
Comment #60
yesct commentedok. I'll try doing 2. now.
Comment #61
yesct commentedok, this does 2.
with warning severity, markup:
with info severity, markup:
Comment #62
yesct commentedopened #2505499: Explicitly support render arrays in hook_requirements() for #59 4.
Comment #63
yesct commentedupdated issue summary
added beta eval
and manual testing was done (including drush rq)
Comment #64
kgoel commentedI like it how universal @cron placeholder has been separated into meaningful name placeholder names which makes it so much easier to read and understand.
Text is displayed correctly since cron ran pretty recently on my local.
I have read xjm's comment in #59 and all her concern has been addressed into the patch.
Comment #65
alexpottThe reason this string was broken up into separate calls to t() was so that translators only had to translate each sentence once.
If we're going to go down the t() route can we prepare the array of variables only once.
That said the reason for ditching the inline twig approach was that:
I'm still not convinced by that argument. I think the inline template approach kept the translation contexts the same and put the entire message and it's logic in one place. I think each of the cons of the inline template approach listed in the summary are refutable. Which I'd do in later comment because atm I'm time poor.
Comment #66
pwolanin commentedI think the patch in #56 was was fine and should be committed. We can't spend endless time bikeshedding the formatting code for 1 message.
It makes the minimal change needed to remove the SafeMarkup::set() call and minimizes the number of strings that get marked safe.
Comment #67
yesct commentedWe have been trying to be thoughtful about translators, ... but for different reasons. Tagging to get someone to chime in.
Comment #68
yesct commented@alexpott #65 2.
The two arrays are different. I'm not sure how to prepare them just once.
--------------
I updated the issue summary to be more clear about which pros and cons surrounded translation.
Here is an excerpt
$descriptioninto an array, and use an inline template for rendering.Comment #69
pwolanin commentedAt least 2 of the t() strings are re-used in core, so I think we should keep them separate as in #56, though in the second case make both @cron instead of !cron
used once:
'Cron has not run recently.'used 2x:
'You can <a href="@cron">run cron manually</a>.'used 2x:
'To run cron from outside the site, go to <a href="!cron">!cron</a>'Comment #70
xjmSo what @alexpott is asking for in #65 point 2 is to store these to variables once:
Comment #71
xjmI also like the idea of using an inline template, as I've said many times on this issue. However, it looks like using that approach doesn't have consensus from other stakeholders (@David_Rothstein, @effulgentsia, the Twig maintainers), which is why I suggested going the simpler route. I tihnk we should discuss inline templates or templates for "simple" string logic in a separate issue for that reason, and not block this patch on it.
@alexpott, regarding #65 point 1, that is exactly what we have been discussing on this issue, and why one of the remaining tasks was to get input from a translator. I think it should be at NR until that happens.
Regarding #69, here is what I'd do:
Do this once before the if statement and store it to $description, because it is one logical translatable string.
Change this to t('Cron has not run recently. ....@run_cron'), where @run_cron is the $description above.
Comment #72
xjmJust making the summary list an
<ol>so we can refer to specific points.Comment #73
alexpottSo I've no wish to hold up the removal of the SafeMarkup set for inline template in hook_requirements discussion. But also, I don't think was should be changing translation strings in this issue if we can help it - it is out of scope too.
I do think that we should try to simplify the logic so it's obvious what strings appear where and we only produce the variables for translatable strings once.
Comment #74
xjmAssigning this to myself so I can propose some patches. @alexpott and I discussed this for awhile.
Comment #75
yesct commentedOK. But I could try to do a new patch with those clarifications.
Comment #76
xjmI've pinged @Gábor Hojtsy in #2501639: Remove SafeMarkup::set in drupal_check_module() for feedback on the use there of
That will inform the most correct fix here as well (i.e. whether #71 is part of a correct solution).
Comment #77
davidhernandezComment #78
David_Rothstein commentedThat issue is different, though, because it's a case where one of the translated strings is returned from another function call (its value is not known). So that's really the only way to do it.
In this issue, it is known, so the options are:
vs.
Where the first has the advantage that it avoids repetition, but the second has the advantage that it gives more context to the translation.
Comment #79
gábor hojtsy@all: Re string concatenation, I don't think either of the short strings are ambiguous. In #2501639: Remove SafeMarkup::set in drupal_check_module() the
"Currently using !item !version"string was VERY confusing without context. This is not true for the strings here AFAIS. So I think either way works.@YesCT: Re the array, there is only one item difference in the array (in 61 at least) and t() does not care for extra elements in the array, so you can build one array and reuse for t()s or build the common part of the two arrays and add the additional item only for the t() which needs it.
Comment #80
lauriiiShould this be still postponed?
Comment #81
gábor hojtsyThe issue seems to have been postponed on #1825952: Turn on twig autoescape by default and then my feedback. Both of which are now in.
Comment #82
yesct commentedthat assign to @xjm was from June 13 ... and I'm guessing they dont still have patches to propose.
Comment #83
xjmPicking this back up; sorry for ignoring it for a month.
Comment #84
xjmAlso the title for this keeps goofing me up.
Comment #85
xjmSo now that #2505499: Explicitly support render arrays in hook_requirements() is going forward, we can update this to be a render array too. :)
Comment #86
josephdpurcell commentedReviewing this now.
Comment #87
josephdpurcell commentedNo longer reviewing this; stefan_r will review.
Comment #88
stefan.r commentedComment #89
stefan.r commentedIn manual testing this seems to display the same as before.
Comment #90
stefan.r commentedComment #92
josephdpurcell commentedReviewing this now.
Comment #93
josephdpurcell commentedUpdating proposed resolution part 2 in the summary to match what was discussed.
Comment #94
josephdpurcell commentedFix link to comment in description.
Comment #95
josephdpurcell commentedAdd manual testing steps to the summary.
Comment #96
josephdpurcell commented1. Markup at /admin/reports/status w/no error
BEFORE:
AFTER:
2. drush rq w/no error
BEFORE:
AFTER:
3. Markup at /admin/reports/status with error
BEFORE:
AFTER:
4. drush rq with error
BEFORE:
AFTER:
Comment #97
josephdpurcell commentedFrom my manual testing results in #96, I see these issues:
drush rqdisplays an array instead of the text@cron-handbooklink is not getting replacedSetting to needs work.
Comment #98
stefan.r commentedIs the drush problem in scope here?
Comment #99
stefan.r commentedComment #100
stefan.r commentedI just checked with @alexpott on IRC and the drush issue looks to be out of scope here, let's file a PR at https://github.com/drush-ops/drush
Comment #101
xjmThanks @josephdpurcell and @stefan.r!
For Drush, let's file a pull request against Drush and just reference it here.
I updated the remaining tasks in the summary.
Comment #102
stefan.r commentedComment #103
josephdpurcell commentedReviewing and manually testing now.
Comment #104
josephdpurcell commentedHere are my manual testing results for patch #98. Note that the "BEFORE" is not mentioned here since it should be the same as the previous results, see #96.
1. Markup at /admin/reports/status w/no error
AFTER:
2. drush rq w/no error
AFTER:
3. Markup at /admin/reports/status with error
AFTER:
4. drush rq with error
AFTER:
Comment #105
josephdpurcell commentedUpdating issue summary to reflex the resolution we are going with.
Comment #106
josephdpurcell commentedMarking remaining task as done.
Comment #107
josephdpurcell commentedI manually tested #98 and saw that it fixed issue #2 mentioned in #97.
Because issue #1 from #97 is being addressed by drush, see the pull request and issue, I believe all issues are covered.
Setting to RTBC.
Comment #108
josephdpurcell commentedI was confused by the summary. It only mentions the use of SafeMarkup used when setting the $description, however, there is a second use when printing phpinfo. See these lines:
This needs removed as well since there is no other issue that addresses it.
Setting back to needs work.
Comment #109
josephdpurcell commentedUpdating summary to clarify there are more than one calls to SafeMarkup.
Comment #110
josephdpurcell commentedComment #111
josephdpurcell commentedComment #112
stefan.r commentedThanks @josephdpurcell!
$phpversion is used only once? If so, not necessary to assign to a variable.
The wording here has changed and has been moved from the value to the description. Can we post a screenshot with the visual change?
Also misses a full stop.
Just so I can understand, as they don't have any links, why are these necessary?
You hardcoded the link here, this is likely making tests fail. This will also break on sites that run on a prefixed URL.
Comment #113
stefan.r commentedComment #115
josephdpurcell commentedScreenshots
Comment #116
josephdpurcell commentedWhoops, I left in the manual testing changes to make the screenshots.
This patch puts back the real code.
Comment #118
stefan.r commentedYes, better to stick to the scope of the issue so as to not drag this out too much, we can then improve the message in a followup :)
I think putting a markup array in the value key is perfectly fine, we put markup in there now as well (as a string). The docs are a bit confusing -- I would read them as giving examples of what you /can/ put in there, rather than what you /must only/ put in there. So if anything the docs could use updating...
Also, I think we forgot a "before" screenshot?
Comment #119
josephdpurcell commentedOk, here is what it would look like if we just did a render array.
Comment #120
josephdpurcell commentedAnd the screenshots:
BEFORE:
AFTER:
Note: I didn't do all 6 cases since the code only changed for this one case.
Comment #122
stefan.r commentedGreat :) For me this can go back to RTBC if green...
Comment #123
joelpittetThis looks good, least amount of changes to get it done. Nice working removing SafeMarkup as a dependency on that that file!
Comment #124
alexpottThis is a really nice fix. The translated strings have not changed and the neither has the logic. Since this only effects runtime checks I think we don't have to wait for #2505499: Explicitly support render arrays in hook_requirements(). Two less SafeMarkup::set() calls. Nice. Committed 6f106ac and pushed to 8.0.x. Thanks!
Comment #127
alexpottSo @xjm pointed out
This is a value not a description. In fact let's file a separate issue to deal with this an not include it in the patch.
Comment #128
josephdpurcell commentedThis patch removes the link entirely. I have created a follow up task to put the link back in #2551725: Remove system_requirements() SafeMarkup::set() use with 'value' key.
Comment #129
josephdpurcell commentedIgnore that last patch, I misunderstood what we were doing.
Okay, so this patch takes us all the way back to #98, i.e. this patch is identical to #98.
The second SafeMarkup::set that #119 meant to address will be worked on in #2551725: Remove system_requirements() SafeMarkup::set() use with 'value' key.
Comment #131
joelpittetBack to RTBC, and back to #98
Comment #132
stefan.r commentedjust wondering what's wrong with it being a value? The value field should still be able to have markup (it did for PHP before the patch), so if anything don't the docs need updating to reflect render arrays are fine there? We were working on test coverage for this in the child issue.Ah nevermind, just noticed we have #2549803: hook_requirements() 'value' key is confusing and used inconsistently, and 'title' is only displayed during installation if the value is also set
Comment #133
xjmThis looks great! The one thing we may still need is to explicitly add a
#weightkey to each to ensure they are in order, similarly to @Wim Leers' feedback in #2505499-48: Explicitly support render arrays in hook_requirements().Comment #134
akalata commentedComment #135
joelpittetNit: Need some ending commas on these array items.
Comment #136
akalata commentedI knew it!
Comment #137
David_Rothstein commentedI do not understand how we've come to a point where the above is required when you simply want to write a paragraph with 3 dynamic sentences in it???!!!
If it's going to be a render array, couldn't the strings at least be concatenated in PHP (like everyone else does everywhere) and then simply this:
Yes, it means that somewhere down the line, the render API would do a xssFilterAdmin on this when technically it isn't needed, but is that really a big deal compared to the much simpler developer experience? Concatenating strings like this isn't too common anyway, so the extra xssFilterAdmin call wouldn't happen too often.
Nevertheless, I see this pattern is used in core at least once already, so if this needs to be a followup issue instead, then so be it...
Comment #138
joelpittet@David_Rothstein because the
t()returnsSafeMarkupand it has<a>tag that once concatenated will lose its "Safeness" and withXss::filterAdmin()via#markupwould be filtered out.Comment #139
joelpittetAlso, you aren't crazy it's horrible DX.
Comment #140
David_Rothstein commented@joelpittet, Xss::filterAdmin() won't filter out
<a>tags; it allows pretty much everything to pass through except some dangerous/invalid/uncommon tags. It would only not work in the rare situation that you need to use one of those disallowed tags.In code terms my suggestion is that this whole patch could just become:
I tested it now and it seems to work fine.
Either way, glad I'm not crazy :)
Comment #141
joelpittet@David_Rothstein oh my bad I'm wrong about links filtered out, just their possibly nasty href="javascript:" and onclick type attributes are stripped., glad I looked at it again.
Comment #142
joelpittetAlso, I like you're idea @David_Rothstein in #140, @akalata and @xjm want to give that try and see if we can get that potentially better DX working?
Comment #143
joelpittetHoping everybody agrees this is a bit more sane... I know @akalata said she was busy for a couple of days on client work so I'm rolling a patch so others can have a review.
I converted the ! to @ too because they ensure the unsafe values in those variables are escaped.
Comment #144
stefan.r commentedI don't really understand what would be wrong with a single markup array here, it looks fine to me! We did have this same discussion last thursday while discussing this issue:
Comment #145
alexpottThis where we should be doing url filtering. Because this does not really do anything apart from an call htmlspecialchars() that does not do much. Not sure what's the right thing to do here.
Also we still have !cron in the string.
Comment #146
alexpottDone a bit more thinking about
!cronissue. It's not in scope for this issue - it should be part of #2506427: [meta] !placeholder causes strings to be escaped and makes the sanitization API harder to understand. The output of\Drupal::url('system.cron', array('key' => \Drupal::state()->get('system.cron_key'), array('absolute' => TRUE))can not be unsafe.Comment #147
xjmOne of the !cron didn't get converted to @cron here also. But is @alexpott saying in #146 that we shouldn't change that line anyway?
I'm still kinda concerned about the unnecessary escaping-followed-by-filtering, but it is less verbose. So if people think that's the way to go, OK... but we're trading one problematic pattern for another. =/
Comment #148
stefan.r commentedvoila
Comment #149
alexpottThe latest patch looks nice and simple and accomplishes the goals. By making the description a render array it is going to admin filtered - which seems fine since this is definitely admin content.
Comment #150
alexpottOkay I prefer #136. I missed the previous discussion and did not think hard enough. My bad. #136 does not need the #weight though.
Comment #151
stefan.r commentedComment #152
stefan.r commentedComment #153
stefan.r commentedComment #154
stefan.r commentedComment #155
alexpottI've been thinking about @David_Rothstein's objections to the complexity of multiple render arrays with
#markup. How about we allow #markup to be a array of strings that we join safely. Here's an issue that shows what I mean - #2554073: Allow #markup to be an array of strings and join them safelyComment #156
xjmSo #136 is not actually really more verbose than HEAD. All the same lines are there in HEAD, and actually in places that are less scannable because they are spread out.
I do share the reservations in general about the DX of joining multiple
t(), and @effulgentsia also shared concerns about the use of#prefixand#suffixas patterns that get abused, but I feel like arrays of the strings are the best solution so far because the render API is an API we already have, that developers from D7 already know how to use and will need to use elsewhere in D8, and it has minimal side effects.#2554073: Allow #markup to be an array of strings and join them safely is interesting but we already have multiple examples of using render arrays in HEAD, so I don't think it makes sense to block this particular patch on it. If we want we can improve the DX later.
Comment #157
alexpottI agree with #156. If #2554073: Allow #markup to be an array of strings and join them safely lands then we can simplify all of this post 8.0.x for all it is worth. And yes it ugly but it is the way we've solved the problem before. Sorry again for my earlier rtbc in #150.
We are working on #2505499: Explicitly support render arrays in hook_requirements(). Render arrays are our current solution for this. The patch keeps all the translatable strings the same. Let's do it.
Comment #158
joelpittetexcept for projectile vomiting and a boggling DX WTF, yes I agree.
#143 still looks like the best choice. I honestly don't see what we are gaining from moving these strings to render arrays.
Comment #159
joelpittet@xjm can you explain #147
Especially the part where the first pattern is problematic.
Comment #160
joelpittetI think this needs a bit more discussion, because I really don't see the problem with the approach in #143 and others have thought that was a good solution, though rightfully questioning it as a solution, it seems to be an elegant solution. I'm sure @David_Rothstein is on board with that version and both @xjm and @alexpott had thought that was ok as well... so I'm really curious what was the clincher.
Comment #162
xjm@joelpittet, my comment was about the fact that using
#markup => t() . ' ' . t() . ...unnecessarily filters text that has already been escaped, so that the patch changes what happens in HEAD.I think we can explore fixing the DX in #2554073: Allow #markup to be an array of strings and join them safely. Since the pattern does already exist in HEAD, and this issue is the child of a critical, I am going to commit this patch and we can evaluate those instances in a non-release-blocking followup.
Thanks everyone for your patience on this issue. It's definitely something of the canary in the coal mine, as @effulgentsia put it, in terms of highlighting deficiencies in our best practices. I think though we need to move on from this particular instance and revisit this and other usages in non-release-blocking issues.
Comment #163
xjmOkay I actually did not commit this from Needs review... I had been typing the comment for the past 90 minutes and discussing it with @joelpittet at the same time in IRC.
However, I reiterate that the pattern exists elsewhere in HEAD and the difference in this function is not worth blocking release on. This issue was also committed once before because of a hunk that was added to the scope that has since been removed and also fixed elsewhere.
Comment #164
joelpittetFTR: Still hate this render array approach, but between render arrays and
SafeMarkup::set()removal, the latter wins, and at least we know our hands are not tied with that approach in contrib/IRL.Comment #166
hass commentedFollow up issue #2563757: External cron URL on status page is not absolute.