Closed (fixed)
Project:
Drupal core
Version:
8.0.x-dev
Component:
base system
Priority:
Critical
Category:
Plan
Assigned:
Unassigned
Issue tags:
Reporter:
Created:
10 Sep 2015 at 17:09 UTC
Updated:
11 Oct 2015 at 12:44 UTC
Jump to comment: Most recent, Most recent file
Comments
Comment #2
justachris commentedSwitch this scope to non test, since test only is covered in another issues. Marking this postponed on that issue #2506445: Replace !placeholder with @placeholder in t() and format_string() for non-URLs in tests
Comment #3
xjmComment #4
sutharsan commentedAs discussed in #2506445-176: Replace !placeholder with @placeholder in t() and format_string() for non-URLs in tests and #177. Include these changes in this issue:
Comment #5
sutharsan commentedAlthough this issue is still on hold and taking the risk that my work is in vain, I had the time to work on it.
This time a patch from scratch. Only low hanging !placeholder fruit was picked. The more difficult !placeholders are listed below and may or may not deserve a follow-up issue. In all cases the !placeholder contains HTML in some form.
contextual.module:
t('Hovering over the area ... The icon typically looks like this: !picture', array('!picture' => $sample_picture))system.admin.inc:
BreakLockForm.php:
ViewEditForm.php:
AccountForm.php:
ConfigSync.php:
$this->t('The following items in your active configuration ... !changes', array('!changes' => $change_list_html))Comment #6
sutharsan commentedSome !placeholders are suggested by the regex but not replaced by the patch:
Placeholder contains a URL in:
Placeholder used in non-html context:
! not used as placeholder
Comment #7
sutharsan commentedRemove. Is handled in #2567159: Fix improper usage of t() in ViewsBlock
Comment #8
xjmThe blocker is in now.
Thanks @Sutharsan for the starter patch and the information on the unmatched lines.
I'm assigning this one to myself to read through and make recommendations on how we can break up the scope a bit.
Comment #9
xjmTaking some notes while I look through the starter patch on how to split it up. I'll try to summarize afterward.
This one is fine.
This one is probably okay, since the message is probably safe on its own, but we should double-check and maybe look for other similar uses in the patch with error logging.
This one will cause double escaping as it is. Let's split this one out 'cause it's sort of a weird used of t() and we may want to discuss how to change the translation.
",
+ '@query' => "
" . $query . "",...
- $this->fail(t("The %setting setting is currently set to '%current_value', but needs to be '%needed_value'. Change this by running the following query: !query", $replacements));
+ $this->fail(t("The %setting setting is currently set to '%current_value', but needs to be '%needed_value'. Change this by running the following query: @query", $replacements));
@@ -225,9 +225,9 @@ public function checkStandardConformingStrings() {
- '!query' => "
" . $query . "",+ '@query' => "
" . $query . "",...
- $this->fail(t("The %setting setting is currently set to '%current_value', but needs to be '%needed_value'. Change this by running the following query: !query", $replacements));
+ $this->fail(t("The %setting setting is currently set to '%current_value', but needs to be '%needed_value'. Change this by running the following query: @query", $replacements));
These will cause double-escaping. We can fix this by putting the code tag in the string itself; no idea why it's not. Let's put these and any others like them in another issue.
I think in all three of these the things passed to the
@placeholderare translated, and therefore returning safe strings or string objects, so these should be okay, but someone should probably confirm that.Why would this size ever have had
!placeholderin the first place? May need a little quick research.I think all of these form validation errors with the
#titleelement should get their own issue. (It was a big swath of the tests too.)This one is fine; obviously no reason for it to use
!placeholder.This one is a handful; it's sometimes a
#titleand sometimes a#markupand eek. Own issue maybe.Exception messages should not be translated. Lets' split these and any other translated exception messages in the patch out into their own issue.
This looks like it was broken in the first place. Single escaping would be the correct thing to do here. So I think this is okay if someone confirms that.
Entity labels and
#title. Tricksy. These probably also want their own issue (separate from the form one).Let's remove all
format_string()from this patch and dot()only please.Absolutely no reason for this to have ever been
!placeholder, so this one is fine to include in this patch.Preprocess function. This one should maybe have its own issue. We had some fun with
rdf_preprocess_comment()when removingSafeMarkup::set().It took a bit of digging, but I found:
(That's from StorageInterface::createCollection().)
So this one is also fine to convert here since no special characters are possible.
This one is more entity labels, so maybe with that other issue? Also a line of docs. We should evaluate the whole hook example.
More
#titleand entity labels. Also this one looks like it's mixing templating with translator context. Maybe its own separate issue for that reason, though it'd need whatever info from the main entity label issue.More entity labels. You get the idea.
This is in the MailHandler so a non-HTML use I think, so should not be included.
More
#title, used in another string this time.format_size() returns translatable strings which will in the safe list or safe string objects later on,
so these are fine. Edit: @alexpott said something this morning about this being not as simple as it looks, so these should probably get their own issue.These are more putting of error messages into other strings; can probably go with that one from the first hunk.
Search excerpts are a whole complicated thing. Let's give this one its own issue for proper manual testing and maybe some fixer-upper.
More opportunity for extension authors to inject XSS by putting XSS in their extension names. ;) Yahuh. All these SystemController theme name ones should be fine I think but it could be its own patch.
This one is fine.
For these, I think maybe the version requirements can contain angle brackets, e.g.
> 5.5.2or whatever? So single escaping is probably correct, but we should test that there is no double escaping. Could be its own little issue.Yowza at all these. Let's give them their own issue.
This also looks like an integer.
Integers; no reason to use
!placeholder. So these are fine to include in the patch.This'd normally be a serial ID unless someone is implementing a weird storage, and this is just hook docs, so this change is fine.
This hunk can go since it got fixed elsewhere, as mentioned above.
Oh Views. Why so many
!placeholder. This is why I started with Views first back a few months ago. ;) I'm tempted to say leave Views out of the intial patch entirely, actually, so we can take a more general look at everything that's going on there. We could just resurrect the original views issue thta I closed as a dupe for that.Phew!
Comment #10
xjmOkay, so let's start by getting some split off issues for:
core/lib/Drupal/Core/Database/Driver/pgsql/Install/Tasks.php(those two seem okay to combine in one patch).$e->getMessage()inside another string.template_preprocess_comment()SystemControllertheme_system_modules_details()search_excerpt()Comment #11
xjmSomeone else can file those 9 issues in #10. :) Postponing this one until some of the targeted fixes get in to reduce reroll hell.
Comment #12
xjmUpdating summary with the scoping recommendations.
Comment #14
xjm(Fixing the grep command a bit more.)
Comment #15
effulgentsia commented#2506427-63: [meta] !placeholder causes strings to be escaped and makes the sanitization API harder to understand lists the 3 behaviors of
!that 8.0 won't be released with. While there's still some debate on that issue on how to proceed for URL usages and non-HTML usages, there is no debate that we must fix all of core non-url HTML outputs to work correctly with@. Therefore, raising this to Critical.Also, unpostponing to make clear that there's an actionable next step, which is to file the child issues and start rolling patches for them, per the issue summary.
Comment #16
dawehnerComment #17
dawehnerComment #18
dawehnerComment #19
dawehnerComment #20
dawehner.
Comment #21
dawehnerComment #22
xjmWhen working through the child issues, where it's non-onerous, we should:
Comment #23
xjmSeparating out a section for issues still to file.
Comment #24
xjmComment #25
xjmLooking at #5, we could also file issues for ConfigSync (non-best-practice of
t()), the systemsystem.admin.incmachine name/markup thing (that's an interesting use of directional markup), AccountForm (another non-best-practice use oft()) andcontextual.module(that one looks like it needs some specific focus, because weird).BreakLockForm should be equivalent for
!or@since it's adrupal_render()for the username right there inline (...we seem to early render usernames a lot).Comment #26
xjmComment #27
xjmComment #28
dawehnerThere we go. Some of them certainly have not the best title / wrong component but well, feel free to improve as you go.
Comment #29
alexpottComment #30
alexpottComment #31
dawehnerComment #32
alexpottComment #33
alexpottComment #34
dawehnerComment #35
xjmComment #36
xjmComment #37
justachris commentedUpdates from child issues that may break translations thus far (Patches that have been committed.)
May be useful for the translation teams to help them figure out what needs to be updated. Excerpts are pulled manually thus far.
Comment #38
dawehner@justAChris
This is super helpful, especially once we provide an update path for them, which be should better do, I guess?
Comment #39
stefan.r commentedComment #40
dawehnerComment #41
dawehnerComment #42
dawehnerComment #43
justachris commentedUpdated the list of possible broken translation strings to include all children of this issue committed thus far. Will probably move this list to a separate issue and combine with the list of strings changed in the hook_help() issue and others (still to be created.)
Comment #44
dawehnerWe probably need a postponed critical to create some kind of update functions for all of them, I think.
Comment #45
justachris commentedRelated to comments #43 & #44, I've opened an issue to contain discussions on how to proceed with already translated strings and hopefully list out all of the strings that were updated: #2572771: List translation string changes following replacement of !placeholder and @placeholder and discuss ways to assist translation teams. Not marked as critical since from my understanding, fixing the translations that we broke does not block RC.
Comment #46
dawehnerI'm actually convinced that we need an update path for all the strings, which should be critical. In general it is a good idea to list the strings in a non critical meta whatever issue.
Comment #47
xjmOutside of Views and the test coverage for
!placeholderitself, we still have this usage that should get an issue filed when grepping:Comment #48
sutharsan commentedThese are the strings that potx module found in current HEAD (general.pot.txt). Also adding an error file of strings that potx excluded mostly because t() contains a variable and not a string.
Comment #49
dawehnerAdded a new issue, but I think its not critical #2575599: Remove !placeholder in ContactPersonalTest and Drupal\migrate\Plugin\migrate\id_map\Sql
Comment #50
berdirThanks, looking through that list and found one more, trivial case:
This might be a valid test:
The only other is syslog (non HTML output) and ViewsEditForm which is being fixed in one of the remaining issues.
Looks like we're almost there!
Comment #51
plachThis is a grepped list. I'm trying to address it in #2571695: Remove !placeholder and unsafe string return from SafeMarkup::format().
Comment #52
plachComment #53
plachPlease, stick this time.
Comment #54
plachWe should be done here.
Comment #55
xjmDiscussed with @plach, @berdir, and @dawehner. We aren't quite entirely done. There is:
However #2506427: [meta] !placeholder causes strings to be escaped and makes the sanitization API harder to understand is still open, so I will just move those outstanding tasks over there.