Problem/Motivation

Theme functions are insecure as they don't auto-escape. In most cases Twig can be used instead.

* hook_theme() implementations can also specify that a theme hook
* implementation is a theme function, but that is uncommon. It is only used for
* special cases, for performance reasons, because rendering using theme
* functions is somewhat faster than theme templates. Note that while Twig
* templates will auto-escape variables, theme functions must explicitly escape
* any variables by using theme_render_and_autoescape(). Failure to do so is
* likely to result in security vulnerabilities.

Proposed resolution

Ideally removal, but Drupal core still uses some legacy theme functions and removing them before RC doesn't seem likely. It would also be a bit of a surprise to drop support this far into the release cycle.

If we don't get rid of the legacy theme functions in core on time, maybe just a @deprecated notice in the code for now, with possibly harder deprecation in a later minor release / RC.

Remaining tasks

Decide how to go about deprecation
Write up groups.drupal.org post/beta release notes announcing support for theme functions will be removed (such as in https://groups.drupal.org/node/482888)
Patch with @deprecated notice

User interface changes

N/A

API changes

Deprecation of theme functions

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

stefan.r created an issue. See original summary.

stefan.r’s picture

Issue summary: View changes
dawehner’s picture

I'm not sure whether we can do anything harder than a deprecation error. If we do something harder later, we should ideally break it now already, given how the promise of SEMV works.

catch’s picture

So this is a case where the BC layer is not just a wrapper, but a whole layer of code and concepts that are actively harmful.

In general I'm not a fan of marking things @deprecated while core still uses them, but we should clarify what our default position is on that.

I do think we should trigger_error() or assert() in a minor release after theme functions aren't used in core - i.e. that's something we could do in the 8.1.x branch once it's open.

At a later date we could even disable discovery via a container parameter and require intervention to re-enable it again. That would be a one-line (or ten line perhaps) change for modules that still need it which could be OK in a minor version for something already deprecated for months or years.

stefan.r’s picture

Seems we already had an issue at #2575081: [policy, no patch] Use E_USER_DEPRECATED in Drupal 8 minor releases.

+1 to #4... As to the semv problem, one can argue whether removing theme function support during a minor is an exception to the semv rules (considering how severe of a security problem they pose) but as long as we announce as early as possible, as widely as possible and with enough notice (6 months+?) that we'll likely remove theme function support during a minor that may still be fine to do?

RainbowArray’s picture

The biggest impact this would have is on contrib module developers Outright removal of the ability to use theme functions would be a pretty hard break at this point, but ultimately that's the direction Drupal needs to go. As a themer/front-end developer, I'm very much in favor of easier access to markup, ideally in templates wherever possible. So if we're able to put up some big warning sign to contrib to head in that direction, I think that's good. As for a hard deprecation in a minor release? I know joelpittet has at times downloaded all contrib modules to check how many times a certain thing appears in code, and others have probably done so as well. Maybe we could do a search to see how often contrib is using theme functions in d8. If there are only a few instances, reach out to those module's maintainers. If theme functions are rampant, that's a harder problem. So the sooner we nip this in the bud, the better.

davidhernandez’s picture

I don't quite understand what we are deprecating. The individual theme functions core is still using (that doesn't sound like something that needs deprecating,) or the ability to implement a theme function?

catch’s picture

The latter.

stefan.r’s picture

So I quite like @catch's idea, which if I understand correctly would be:

  1. Mark the ability to define theme functions as @deprecated before RC1, even if core still uses a handful of them.
  2. Announce as widely as possible and as early as possible what steps 4 and 5 will be and clarify how that ties in with Drupal's semantic versioning policy.
  3. Get rid of all remaining (admin-only at this point) theme functions in core.
  4. As soon as that is done, and the notice period is over (6+ months?), implement a trigger_error(E_USER_DEPRECATED) or an assert() in a Drupal 8 minor release that checks if any theme functions are still used.
  5. In a later Drupal 8 minor release (likely at least a year from now), disable discovery via a container parameter and require intervention to re-enable it again.

We could also add in a hook_requirements() warning as soon as 3 is done. And given we only use theme functions on admin pages anymore I wonder if we could already do 4 for non-admin pages in the RC

catch’s picture

We can also do 5a/b which would allow people to prevent theme functions being discovered on specific installs as security hardening.

Could still then flip the default later.

theme_functions module :P

davidhernandez’s picture

The latter.

Woah, Nelly.

I guess #10 sounds like plan. We'd definitely need some broad communication about this. I'm not about 5. though. Is that necessary, or do we consider this important enough to break sites?

catch’s picture

I don't think we should break existing sites, not really what I meant, so here's a more fleshed out version of #5.

In a minor release, we add a toggle for theme function discovery, or better provide an alternate service with a subclass that doesn't do it.

Otherwise everyone else just gets the E_DEPRECATED in theme registry rebuilding.

Option #1:

- default to no discovery
- add a module that enables discovery again
- for every existing install, we add a hook_update_N() to enable that module.

new installs wouldn't get it though.

So a contrib module or theme could break, a distribution could break, but a site couldn't - unless they disabled the module manually.

Option #2:

- default discovery to on
- have a module that switches it off
- enable that module in minimal/standard profile

Contrib modules and themes could break, sites and distributions would be fine.

Option #3:

- default discovery to on
- have a module that switches it off
- don't enable it by default anywhere

Sites can opt-in to the security hardening and lose the E_DEPRECATED notice.

None of these break sites, they involve different levels of breakage for contrib modules and themes. Allow people to stop executing that code is both a nice security hardening and small (once the theme registry 90% speed up patch lands) performance improvement so even if we only do it as an addition rather than a change, it's worth looking at ways to do it.

davidhernandez’s picture

#1

new installs wouldn't get it though.

New sites aren't always built with new modules though.

#2

distributions would be fine.

Assuming they update accordingly to turn it off if they need to, particularly in cases where they don't provide their own copy of core now.

#3 sounds reasonable and the least risky to me. It is a nice security enhancement, and leaving it optional means limited chance of breaks while giving anyone who cares the ability to do it. Does it need to be a module, or can we turn off the discovery with a setting?

lauriii’s picture

My huge +1 for #1. Contribs can add the module as dependency. In order to make that work properly with #2 & #3 modules would need to write some special logic. #1 makes it easy to see which modules are requiring deprecated code. It would be also very clear message for everyone: "DO NOT USE THEME FUNCTIONS!".

stefan.r’s picture

I think the main worry is contrib becoming too dependent on theme functions. If some of the most popular modules use them it may not even be possible for distributions to ship without that legacy_theme_functions module enabled, or for individual sites to be anything close to functional with theme functions disabled as they won't be able to use those modules.

So the goal is to put pressure on contrib not to use theme functions by making enough installs opt out of them, if only a minority of installs disable theme functions that won't have the same effect as a majority doing so.

In any case the trigger_error/assert would hopefully mostly do the trick as it will make their tests go red :P

dawehner’s picture

In any case the trigger_error/assert would hopefully mostly do the trick as it will make their tests go red :P

Well, trigger_error() would allow people to still use it, you can workaround that. asserts() would be too harsh and well semantically wrong. trigger_error() talks about deprecation, which matches perfectly here.

stefan.r’s picture

@dawehner well we're solving a security issue so I think it's allowed to be harsh, if it's not harsh it's useless.

The problem with E_USER_DEPRECATED is you have to explicitly lower the error reporting level to stop getting it which is not really desirable.

An assert() is perfect for this case as it will always be turned off on production setups but will still make people's tests fail and give an explicit exception on dev. Semantics be damned, isnt this exactly what we want? :)

So my vote is for option #1 as well (in comment #13) with an assert() that will still trigger even if the legacy_theme_functions module is turned on - otherwise people can just enable the module in their tests, now at least they'll have to set up their own error handler to work around that, which is more obviously wrong.

By the way option #1 would not break distributions - distributions ship with specific versions of core and can enable the theme_functions module in an update hook.

So this all is harsh, but this will likely only happen a year from now so as long as we communicate clearly about what will happen, and how it relates to our semantic versioning policy, I don't see any big problem here?

catch’s picture

The problem with E_USER_DEPRECATED is you have to explicitly lower the error reporting level to stop getting it which is not really desirable.

Or stop using theme functions.

stefan.r’s picture

@catch heh :)

Let's be realistic here, most contrib developers are pressed for time and will go for the easy option, which is why I'm worried about theme functions to begin with - in D7->D8 ports won't some just reuse their old theme functions rather than bothering with twig templates?

dawehner’s picture

The problem with E_USER_DEPRECATED is you have to explicitly lower the error reporting level to stop getting it which is not really desirable.

If you are stupid you are stupid, there is no way around that. assert() is IMHO a tool for a total different usecase.

davidhernandez’s picture

distributions ship with specific versions of core

Not necessarily. It depends on the use case. There are many different ways of building out a distribution. The things you can download from drupal.org are not the only things we have to account for. Many orgs a have a lot of different internal products built in different ways, and deploy sites in their own ways.

I agree with catch that throwing an error is a good way to keep things working but be annoying enough to incentivize people change.

catch’s picture

There's only ever one version of core that's supported - the latest patch release. It's possible we do 8.0.6 and 8.1.0 the same day, then there'd be two for a month.

So distributions with old core versions are in a pretty bad state.

stefan.r’s picture

In any case we're not breaking the distributions themselves.

Distributions come with a specific core version and the most used ones put out regular updates - so if they ship with a core version that disables theme functions by default, the distributions can enable them themselves in their install/update hooks (or list it as a dependency).

If people use a different core version than what comes with the distribution, they're doing something unsupported and at their own risk - they should be smart enough to turn on the "theme functions module" if needed in such case, and as soon as modules put out new releases and haven't replaced their theme functions with Twig templates yet, they'll add it as a dependency anway.

stefan.r’s picture

Status: Active » Needs review
Issue tags: +Needs change record
FileSize
3.67 KB

Since no-one objected to at least a deprecation notice in the docs, let's start with that?

dawehner’s picture

It would be great IMHO to have a follow up which adds the E_USER_DEPRECATED thing.

stefan.r’s picture

Now that we don't use theme functions, maybe we could even add a trigger_error with E_USER_DEPRECATED for 8.0 or 8.1 and encourage people to turn that off on production?

See also https://github.com/symfony/symfony/issues/12973

star-szr’s picture

I think we should focus on docs here, doing that seems pretty non-controversial.

Then we can reuse #2575081: [policy, no patch] Use E_USER_DEPRECATED in Drupal 8 minor releases for the assert()/trigger_error()/whatever. Cool?

+++ b/core/lib/Drupal/Core/Render/theme.api.php
@@ -68,12 +68,12 @@
+ * functions are deprecated in Drupal 8.x and will be removed before Drupal 9.0.

@@ -98,7 +98,8 @@
+ * vulnerabilities. Theme functions are deprecated in Drupal 8.x and will be
+ * removed before Drupal 9.0. Use Twig templates instead.

@@ -1072,9 +1072,10 @@ function hook_page_bottom(array &$page_bottom) {
+ *   - function: (deprecated in Drupal 8.x, will be removed in Drupal 9.x) If

Shouldn't these be 8.0.x (or 8.x.x) and 9.0.0?

catch’s picture

Yes docs here and actual E_DEPRECATED/assert in a follow-up is good.

star-szr’s picture

Title: Deprecate theme functions for removal before Drupal 9 » Deprecate theme functions for removal before Drupal 9 (docs only)
Issue tags: +rc eligible

Thanks, doing some retitling and tagging.

snehi’s picture

Adding 8.0.x and 9.0.x

tim.plunkett’s picture

+++ b/core/lib/Drupal/Core/Render/theme.api.php
@@ -68,12 +68,12 @@
+ * functions are deprecated in Drupal 8.0.x and will be removed before Drupal 9.0.x.

@@ -1072,9 +1072,10 @@ function hook_page_bottom(array &$page_bottom) {
+ *   - function: (deprecated in Drupal 8.0.x, will be removed in Drupal 9.0.x) If

Now these two lines need to be rewrapped to 80 chars :)

star-szr’s picture

Status: Needs review » Needs work

Yes please.

snehi’s picture

Status: Needs work » Needs review
FileSize
3.68 KB
1.63 KB

Done.

joelpittet’s picture

Status: Needs review » Reviewed & tested by the community

This looks great. Still needs change record. Would you mind trying your hand at one @snehi?

snehi’s picture

@joelpittet i did not get you.

joelpittet’s picture

Oh so this issue needs a change record to indicate that we are deprecating theme functions. Just needs a draft started. Here are the instructions: https://www.drupal.org/contributor-tasks/draft-change-record

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 34: 2576881-34.patch, failed testing.

joelpittet’s picture

Well it probably needs to be NW for the chang record anyway. You win this one testbot!

snehi’s picture

Status: Needs work » Reviewed & tested by the community

  • catch committed fdd7dcc on 8.1.x
    Issue #2576881 by snehi, stefan.r: Deprecate theme functions for removal...

  • catch committed be070f9 on
    Issue #2576881 by snehi, stefan.r: Deprecate theme functions for removal...
catch’s picture

Status: Reviewed & tested by the community » Fixed

I've linked some existing change records - the default to template one, and theme_render_and_autoescape() (which I retitled), since both of these were deprecation in-practice, and we just waited to document that in core until core was done.

Committed/pushed to 8.1.x and cherry-picked to 8.0.x, thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.