Problem/Motivation
Drupal has no easy way to dynamically create SVGs, currently you either have to write the markup yourself and use string concatenation or use the "inline_template" render element.
Original summary:
I am a contributed module maintainer and am working on a port of a module that dynamically generates inline SVG in page content. It does so to produce charts, graphs, etc based on SQL data. Currently the rendering system removes all inline SVG from markup. Most of the suggestions I've seen suggest writing custom theme layers to embed the SVG, but in my case, I don't know how many SVG graphs will be in a document in advance.
Can anyone suggest how to proceed? Would it be appropriate to try and include every possible SVG tag in the #allowable_tags attribute? Are there issues with namespaced attributes such as xlink if I attempt this?
Proposed resolution
Add support for nested elements to the html_tag render element. By doing this, users can freely form dynamic SVGs without Drupal having to be explicitly aware of the SVG specification, which is quite complicated.
An alternative resolution proposed in this issue is to add new render elements for every SVG tag (see https://www.w3.org/TR/SVG/eltindex.html), which is possible but the resulting render arrays would end up looking a lot like they would when using html_tag.
Remaining tasks
Write a change record.
User interface changes
None.
API changes
The html_tag element now supports render array nesting, much like the container element. Ex:
$element = [
'#tag' => 'svg',
'#attributes' => [
'width' => 150,
'height' => 150,
],
[
'#tag' => 'rect',
'#attributes' => [
'width' => 50,
'height' => 50,
'style' => 'fill:rgb(0,0,255)',
],
],
];
Data model changes
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #71 | interdiff-2694535-65-71.txt | 2.2 KB | samuel.mortenson |
| #71 | 2694535-71.patch | 7.11 KB | samuel.mortenson |
| #55 | dynamic-svg-2694535-55.patch | 4.5 KB | samuel.mortenson |
| #40 | dynamic-svg-2694535-40.patch | 9.59 KB | WidgetsBurritos |
Comments
Comment #2
metzlerd commentedI found a hack that sort of works, where I was able to get SVG to go through using a rendering element of #type=inline_template. I'm not sure if this is the right solution but it seems to render the SVG. I am of course interested in more info regarding the "proper" way to do this if there is one.
Comment #5
tstoecklerWorked on this a bit for #2660124: Dynamically build layout icons based on well formed config. I added a render element for the outer < svg > and for a < g > and a theme function for a < rect >
This is still alpha-level but would love to get some input on this.
Comment #7
tim.plunkettThis is a great start! Not really my area of expertise, but it works great.
Comment #8
samuel.mortensonThis is awesome, and enables work I started and @timplunkett continued in #2660124: Dynamically build layout icons based on well formed config.
Some review of the alpha patch:
If we have attributes as variables like width and height, setting width and height as a part of #attributes would override those variables.
I would remove every template variable that could be represented as a part of the attributes array and have defaults for width and height instead.
Width and height are not used in the template for svg_g.
s/svg-g/svg-rect
Missing x, y, stroke_width, stroke, and fill.
I also have the same note about having attribute variables regarding x/y as above.
Dynamic inline styles aren't super common in Drupal - are there any security concerns with inlining Javascript here?
Comment #10
WidgetsBurritos commentedCurrently working through this at DrupalCon
Comment #11
WidgetsBurritos commentedCurrently working through this at DrupalCon
Comment #12
WidgetsBurritos commentedHere's an updated patch. I'm not quite sure it's "complete" but at least is a step towards the right direction.
Comment #13
WidgetsBurritos commentedFor some reason the patch didn't get attached on that previous message. Attempting again.
Comment #14
WidgetsBurritos commentedRemoving mistakenly added file in previous commit.
Comment #17
tim.plunkettFixing tags
Comment #18
idebr commentedFunny how I was just about to start a sandbox for rendering SVG elements and then I bump into this issue. Some great progress here!
A few points:
<title><rect fill="blue" />instead of<rect style="fill: blue;" />@WidgetsBurritos Please let me know if you are interested in finishing this patch. I would be happy to help with reviews; or I could assist with coding the patch.
Comment #19
WidgetsBurritos commented@idebr I'm cool doing either. I probably won't be on this for the next week or so as I'm gonna be catching up on work from when I was at DrupalCon, so if you wanna continue from where I stopped that's fine by me. Otherwise I'm happy to jump back on it as soon as I get some downtime.
Comment #20
tstoecklerAwesome to see progress on this. My patch really was just a proof of concept for something I was playing with, please feel free to build on it, rip it apart, etc. #8 and #18 are all valid to me. In general it seems that at least @idebr and presumably @samuel.mortenson and @WidgetsBurritos as well know a lot more about SVG than me and I would love to see this being pushed forward.
Thank you all!
Comment #21
idebr commentedAttached patch implements the following Render elements:
<svg><g><rect><use>The interdiff addresses the following feedback:
through the SvgUse Render element.
Comment #22
WidgetsBurritos commentedI still need to pull this down and test it, but at first glance:
Was this supposed to be here? I'm assuming that's some test code that never got removed?
Same here.
Comment #23
WidgetsBurritos commentedThis seems to be working well and I definitely think it makes sense to just let the form api handle all the attributes. Based on my previous feedback in 22.1 and 22.2, here is an updated patch.
Comment #24
idebr commented@WidgetsBurritos Ah yes, that was the code I used for the render element example code. Thanks for the updated patch!
Comment #25
WidgetsBurritos commented@idebr Not a problem.
Assuming others are okay with this implementation strategy, I guess from here we have a couple questions to answer:
1. Per @metzlerd's original question, which SVG tags do we need to provide support for?
Here's a nice long list: https://developer.mozilla.org/en-US/docs/Web/SVG/Element
Is it something we want to start with a small group of tags and then just let people add them as they need to (maybe someone could create a separate "SVG extended" contrib module as needed), or do we try to provide support for as many out of the box as possible?
We're just talking about a lot of tags, so I'd imagine before contributing that many tags to core it should at least be discussed.
2. Also, should there be some sort of automated testing around the form rendering as well?
It may very well be worth confirming that something like this actually produces the desired output:
Comment #26
metzlerd commentedIn my original issue the concern was not to "build svg with render elements" which seems like madness to me, but rather to be able the support the creation of inline SVG as content... both programmatic and otherwise, without needing to use the twig template system. There are other tools or libraries that I could conceive of for creating SVG, none of them would produce SVG as render elements.
Remember that putting svg content in #markup strips out all inline SVG. The idea that I'd have to convert a drawing into complex render array in order to get it through the render system is a non-starter for my use cases. I don't know if #26 was trying to illustrate how this is being used, but dealing with an element this way does not really solve the problem I was trying to achieve.
In my case I'm using a php graphing library to create the SVG. I just need a better way to not have the markup stripped out when encorperating into a render array for display in a drupal page. Imagine making a block that displays an SVG dynamically generated by a standalone PHP library and you'll get a better feel for my use case.
If someone wanted to build dynamic SVG, twig SVG templates would be a much more sane methodology than building up a drawing as a render array. The question is how would you attach the rendered SVG to a render array, and not how can you build dynamic SVG out of render arrays, which does seem more like custom module work than core.
I still have yet to here why people consider inline SVG in markup as a security risk considering that it is static content, so I've never understood why it gets filtered or #markup data.
Comment #27
samuel.mortensonFollowing up on #25.1, I wonder if the html_tag element, which is already in core, would provide all the needed functionality that the patches so far provide. Looking into \Drupal\Core\Render\Element\HtmlTag, I think the only thing we would need to add is a "#children" option, or edit '#value' to support nested render arrays.
Here's an example of how this could look:
Comment #28
WidgetsBurritos commentedIn regards to #26, it appears the original use case of this issue had been modified to address other use cases along the way. I didn't realize that initially, as I didn't read that close enough. I'm not entirely sure the best approach to solve your specific use case, but I do think it's a valid concern. In regards to your comment about this approach being more of a module, than as part of core, I believe part of the emphasis is that it's a prerequisite for #2660124: Dynamically build layout icons based on well formed config, which is attributed to the Layout Initiative.
In regards to #27, I think that is a very valid point. My concern with trying to implement every possible tag up front, is that it's only valid until the spec changes. And while it's not extremely frequent that happens, it's still something the community would be responsible for maintaining. By taking the generic #html_tag approach, it's as scaleable as we need it to be.
Comment #29
WidgetsBurritos commented@samuel.mortenson I've taken an initial stab at #27.
May need a little bit of work, but it does seem to work from a POC stand point anyway. Thoughts?
FYI: Since this is a brand new approach, I didn't include an interdiff.
Comment #30
samuel.mortensonThis assumes that each child contains
#markup, which may not be the case for various render array elements.I think we would need to handle children in the Twig template by calling
{{ children }}if children was set.Comment #31
samuel.mortensonComment #32
idebr commentedRe #27
There is a plan to remove #type => html_tag in #2544318: Remove #type => html_tag usages, so this implementation might be counterproductive for that issue.
Comment #33
samuel.mortenson@idebr From reading that issue, I think the plan is to remove the core usages of html_tag, not the render element itself. On my Drupal 8 site ten of the contrib modules I use rely on html_tag, so I doubt it will go away completely at any point before Drupal 9.
Comment #34
idebr commentedRe #33
From the proposed resolution:
and subsequently in #12:
Comment #35
samuel.mortensonThe issue summary you're quoting is two years old: https://www.drupal.org/node/2544318/revisions/view/8741274/8745098, and #12 is still talking about removing usages in 8.x and removing the render element in 9.x. Removing the render element in Drupal 8 would constitute a BC-breaking API change.
Comment #36
samuel.mortensonI updated #2544318: Remove #type => html_tag usages with links to relevant contributed module code, and a reference to this issue to try and make a case not to remove the html_tag element in Drupal 9.
Comment #37
WidgetsBurritos commentedJust now getting caught up on this issue. When I get a chance, I will work through the recommendation from #30 under the assumption that #2544318: Remove #type => html_tag usages is not an actively pursued priority. If that issues changes, obviously this strategy may change as well, but I'll try to jump on this sometime this weekend, unless someone else wants to beat me to the punch.
Comment #38
WidgetsBurritos commentedOkay, so here is an attempt to accomplish @samuel.mortenson's recommendation from #30. It all seems to work locally, but I admittedly haven't thrown the gauntlet at it yet.
Here is a quick explanation of what I've done.
- There was currently no template for the generic 'html_tag' type, so I created one.
- Almost all of the markup that was generated in
HtmlTag::preRenderHtmlTag()got moved intohtml-tag.html.twig.- Only thing remaining in
HtmlTag::preRenderHtmlTag()is the check whether or not we have a void element.- Because
HtmlTag::preRenderHtmlTag()is no longer rendering markup, this means tests related to markup generation had to be removed. Instead, we're only testing whether or not void elements get marked correctly.- Because of the previous item, I'm not entirely sure what the best way to test for rendered twig markup is but if someone can give me some direction there, I'll be happy to implement.
- This change isn't without it's own issues. A bunch of other tests fail. I'm still looking into those, but really the point of this patch was to make sure this is the right direction, not to get everything perfect quite yet.
Once again, treating this as a brand new approach, so no interdiff supplied for this patch.
Comment #40
WidgetsBurritos commentedCleaned up the undefined index exceptions. Not sure if this is all of the errors but should significantly shorten the list anyway.
Comment #42
manuel garcia commentedI'm seeing this upon applying the patch, which essentially breaks the site:
@import url("/core/themes/bartik/css/components/comments.css?oqeccj");This would be fixed by rendering the {{ value|raw }}, but this would be opening the door to attacks, so that's not a good way to fix it.
I'm thinking we should treat the @import statements differently, perhaps a new render element that would print this?
@import url("{{ value }}")Though this change does not fix the above issue, the attached patch removes the escaping from the tag on
template_preprocess_html_tag, since twig does this for us.Comment #44
WidgetsBurritos commentedManuel,
Thanks for that. I'm definitely seeing this issue as well so thanks for the confirmation. I had tested using the raw filter locally as a proof of concept and when I did that it caused other stuff to fail, and given the security implications I didn't dive too deeply into that.
That's a good approach to solve the imports there. I'll look into it, or if you wanna take a shot at it, as it may be a few days until I can jump back on this.
Comment #45
manuel garcia commented@WidgetsBurritos no problem, about my suggestion: I had a go at it for a while, and the main hurdle would be modifying
core/lib/Drupal/Core/Asset/CssCollectionRenderer.phpwhich is a complex beast. All to avoid escaping those quotes... there has to be a better approach.Comment #46
manuel garcia commentedHere is what i did to follow the route of rendering the import statements on their own. Seems to work for CSS files, but JS is broken on the site for some reason.
Again, this may not be the best way to resolve the double escaping explained on #42...
Comment #47
WidgetsBurritos commentedGreat. Thank you. I'll check it out when I get a chance.
Comment #48
WidgetsBurritos commented@Manuel,
Had a brief moment to checkout your updates. In addition to the imported style issues, it seems JSON rendering is a bit messed up here too , and is one of the main reasons why so many tests are currently failing (i.e.
Exception: No #ajax path specified.failures)For example:
This issue is starting to feel a little bit like a rabbit hole...
Comment #49
manuel garcia commentedComment #50
samuel.mortensonHere's a simplified take on allowing nested elements for the html_tag element. I basically moved the tag to the prefix/suffix keys, instead of doing markup juggling in the preRenderHtmlTag method. This allows for nested render arrays without any explicit code for SVG (besides the related $voidElements I added), which I think is fairly clean. With this patch, you can render an SVG element using a render array like:
which outputs
<svg width="150" height="150"><rect width="50" height="50" style="fill:rgb(0,0,255)" /></svg>(notice how rect auto closes)
I think this is preferable to maintaining Twig templates and theme hooks for every possible SVG element, of which in 1.1 are many.
Comment #52
bighappyface commentedI realize it isn't a concern of this issue, but why are void elements being given a closing slash when Drupal 8 is using HTML5? XHTML backward compatibility for those who chose to implement it in a custom theme?
Comment #53
bighappyface commentedI see that the closing slash has been taken up on other issues, namely #1388926: Remove all references to "self-closing" void elements in core
Comment #54
WidgetsBurritos commented@samuel.mortenson #50 is definitely working for me. You can even abuse the hell out of it with nested and different tag types:
For example:
^ Now obviously some of that isn't a good idea in practice, but from a conceptual standpoint, I like it.
And only 2 tests to resolve instead of 65, seems a bit easier to swallow.
Comment #55
samuel.mortensonThis should resolve the remaining errors.
@bighappyface No idea on #52, but I think it's for backwards compatibility as you mentioned.
Comment #56
WidgetsBurritos commentedI have tested this locally and it seems to work great.
+1 RTBC
Comment #57
catchThe patch itself looks OK (didn't do a full review) but this could really use a change record (to explain how to use the new feature) and an issue summary update (since the title is completely different from the solution, changing the title now though).
Comment #58
samuel.mortensonUpdated the issue summary, @catch should a core contributor write a change record or can anyone do it? If I should write one at https://www.drupal.org/node/add/changenotice?field_project=3060 I will.
Comment #59
naveenvalechaSam,
Anyone could write the CR
//Naveen
Comment #60
samuel.mortensonFiled https://www.drupal.org/node/2887146, moving back to needs review.
Comment #61
WidgetsBurritos commented+1 CR looks good to me
Comment #62
manuel garcia commentedAbsolutely love this new approach @samuel.mortenson, brilliant!!
+1 to RTBC
Comment #63
larowlanLooking good, this will be very useful.
There is no test coverage for these new elements.
If we want to ensure they're not accidentally lost, I think we need some.
Especially given the work that's gone into it so far.
Comment #64
manuel garcia commentedGood call @larowlan, having a go at this.
Comment #65
manuel garcia commentedAdded coverage for all the added void elements.
I also thought about adding coverage for rendering a linearGradient element, which has nested elements, so as to make sure we can actually do this, but it's failing on me...
Comment #67
manuel garcia commentedHad another look at my failing test, looks like the current patch actually does not support nesting html_tags within html_tags, just #markup elements?
My test oputs:
I've even tried the example given on #50, with the same results. Looks like the failing test is actually finding a bug in the current patch.
Comment #68
WidgetsBurritos commented@Manuel Garcia,
Sorry I'm writing this from my phone on the road so I can't test it myself, but does it have anything to do with the
'#value' => NULL,in your test? Or maybe the lack of a type in that same section?Comment #69
manuel garcia commentedThanks @WidgetsBurritos for replying, that's not it though.. we have another test that passes just fine for:
If I omit the
#value => NULL,on the parent element I get error:Undefined index: #value. If I add it to the nested elements it has no effect.Having the
#typeseems to have no effect either, so we could perhaps omit it from the nested elements as well in a future patch... once we figure out why this fails ;)Comment #70
samuel.mortenson@manuel-garcia I didn't realize you could omit
#typefor this kind of element, cool! I'll update the change notice as well. From manually testing this on a standard profile install, nested html_tag elements render just fine. It's only in tests that I'm seeing failures. Not sure what that means yet...Comment #71
samuel.mortensonAh - since we're nesting html_tag elements we'll need to run the pre-render on everything, I've updated the test and things are passing locally. The newlines appended to void elements are a bit awkward, but we shouldn't change those as that may negatively effect existing html_tag users.
Comment #72
manuel garcia commentedAh perfect @samuel.mortenson that makes sense, thanks!
#63 has been addressed and we've expanded the coverage a bit more. So back to RTBC (assuming test comes back green).
Comment #73
manuel garcia commentedRemoving tag added in #49, new approach doesn't need it =)
Comment #74
larowlanCan we get a follow-up to name these test cases.
i.e. each test case has a key in the array, instead of being numeric.
Comment #76
manuel garcia commentedYou mean like making this
$tags['circle'] = [$element, '<circle cx="100" cy="100" r="100" />' . "\n"];instead? Should be very doable in this patch I think.Comment #77
jibranLet's do it then.
Comment #78
manuel garcia commentedActually, after going for it, I realized why a followup makes more sense... the existing test cases need to have their array keys as well, so it makes sense to do these all in one go after this patch gets in.
Created #2889378: Name test cases on HtmlTagTest as a follow up for this purpose.
Comment #79
WidgetsBurritos commented+1 on resolving in a separate issue.
Comment #81
catchCommitted/pushed to 8.4.x, thanks!
Agreed with adding the key names to the test cases in a follow-up.
Comment #82
manuel garcia commentedAwesome, thanks all!
Just published the CR
Comment #84
manuel garcia commentedimho :)
Comment #85
duaelfrThis issue caused some regression #2563069-12: Unclosed conditional comments in html tag result in broken html head.
Comment #86
polSo sad we cannot do the same with Drupal 7.
I opened an issue about this: #2981726: Backport D8 html_tag element changes
Comment #87
polComment #88
pol