[meta] New theme system

I really love the ideas of

1) Moving to a real templating system that is not just PHP and
2) Having Twig be that templating system. I've been using it on my new project and it has been a joy. I especially love its template inheritance system, which is a lot more elegant than our current override system.

One thing about Twig which will have to be resolved is that it works by compiling the templates into php classes. Currently we have no way to do that securely - the files directory is the only place we can write to, and we can't have executable code there. I'm unsure how/if Twig works without that functionality, or how we would implement it if we have to.

Still, +1000 to the idea.

There's already a lot of feedback in #1382350: [discussion] Theme/render system problems

Just going to quickly chime in to day that I had a great 1.5 hour discussion with chx about this today (Thank you @chx), and we went back and forth for a while trying to get to a place where everyone in this mix would be happy. I think using Twig might just be it.

I don't like the idea of tokens for a few reasons, but mostly because it is a homegrown solution that would need to be mixed with PHP. We would still need loops, conditionals and other basic PHP functions in the end. I believe this mix would leave us at a disadvantage by default. I've never been against using a different templating engine, I've only been cautious as the problems we face run much deeper than syntax IMO. Twig looks very similar to Liquid (which I'm familiar with), and seems to have a good balance of basic functionality more and advanced features that we often need in templates, so it could be a good fit.

I still think we need to concentrate on the work I have started and explained here. However, I think that both of these goals are very much related and that we can hopefully do both at the same time. I'm curious to see some examples of what our templates will look like. We (themers) really need to see how this will affect our templates and workflow on real Drupal stuff to be able to give proper feedback, but I think this is very much a step in the right direction.

Rewritten the whole thing.

As I mentioned in Denver (I miss you guys already!), I support the use of twig - it gives us a component that people outside the Drupal world already know, making it easier to bring new people in as well as leveraging an existing support base. Furthermore, from what I've seen in my initial research it is very approachable and is inherently more secure than phptemplate. It also feels like the toke used former that was talked about while still allowing for easy looping over arrays. The only concern I'd have is over maintenance of the core component, but I think the already ongoing integration of the Drupal and Symfony2 communities mitigates that.

A really nice initiative! Definitely for it! @chx, I like your no-nonsense style in this.

Twig looks like an excellent choice for Drupal. I like especially the extendability. If Twig, or something similar would be included in the core, it should be leveraged by its full potential. Some core features turned to Twig extensions perhaps? Tokenizing?
I can already see the two-way benefits this would bring to both Twig&Drupal communities. Oh and look, there's our bestest-symfony-friend fabpot leading the group again. :)

Finally a front-end developed friendly initiative. Definitely for me and many others who are not for arrays and spaghetti php in themes.

Pros:
+ IDE support
+ Clean theme templates

Cons:
Can't find any.

Also see
#1441320: Evaluate Twig template engine for Drupal core (as an alternative for PHPTemplate and possibly even theme functions)

http://drupal.org/node/1441320

If we managed this- it would be a massive boon to bringing in design folks.

I've lost arguments with designers who prefer systems like expression engine... "you just put these tags in your templates" as one young designer explained to me.

And I think that is why we see many more wordpress themes. It's much easier to see how to pull information into your own fancy template. Of course there's big limitations to WP's templating system. Because they are limited to what they can hack in the template only. These are not code-phobic people. They are Mark-up obsessed, they can read logic in templates... But Drupal's system is currently not intuitive or even legible.

Drupal is peculiar because of what I describe as "a big middle man" of configuration. I think that has developed often as a work around to the complexities of the templating system actually. Developers finding ways to make these elements configurable for so-called "non-coders".

So now we have an opportunity to bring together the powerful middle man, while still giving a more learnable, easy to hack template system. Seems like a great opportunity to solve a unique problem, and have a crazy cool tool suitable for the new breed of coding designers and true non-coders.

chx pushing for Twig, now that's an unexpected twist :-)

I think that using Twig (or any other non-PHP templating system if its sufficiently flexible/extensible, and Twig does seem like a great candidate, plus it's consistent with the overall symfony movement) has great potential. Copy/pasting myself from #1441320-10: Evaluate Twig template engine for Drupal core (as an alternative for PHPTemplate and possibly even theme functions) :

I guess there is a potential big win regarding #1158090: Separate usage of hide(), render(), and other template features from print statements to enable non-developers to create themes and #953034: [meta] Themes improperly check renderable arrays when determining visibility here.

Twig being extensible, we could provide our own logic to unify the "is $foo a string that should be printed, or a render array that should be render()ed" subtleties, hiding the php logic.

For instance (just throwing ideas and syntax without much thought, obviously debatable) :
{{foo}} : "print $foo if $foo is a scalar, or render($foo) if $foo is a render array"
{% if !foo|empty %}<div>{{foo}}</div>{% endif %} : does the correct empty() check depending on whether $foo is a string or a render array (possibly means rendering $foo to see if the result is a non empty string)
{{foo.bar}} : "render($foo['bar'])"
{{foo-bar}} : "hide($foo['bar']); render($foo);"
{{foo|keys}} : "prints the subkeys of the $foo render array in a dsm() or something, for discoverability"

Bottomline is : all the discussions around "granular rendering" and "render arrays in templates" since the last 3 years or so have gone in circles around the intricacies of the PHP syntax for strings and arrays and how we have tried (and failed ?) to ease the differences. An expressive, extensible, non-PHP syntax would make that a non issue.

I don't know the internals of Twig well. I am not sure if it even works without compiling down into PHP files, though if not it could likely be made to just do in memory PHP classes (did someone discuss this with Fabien in Denver?). That being said, the performance will not be a lot of fun. I am not sure if the security team is willing to relax restrictions here for the benefit of security, since Twig should be safe from evil PHP code. But I am not a security expert.

Like yched mentioned, Extending Twig to support what we currently have ismost definitely possible. One existing use case is in the Twig Troll post I made, were we'd use {{ item|render }} to have a custom render() call extension. Again, this is just a thought-dump and would need proper testing/implementation.

Overall though, I am in 200% in support of this. It's well-designed, well-documented and well-tested... PHPTemplate is so 2004.

When I first heard the concept of using Twig as the new default theme system while in Denver I was strongly against the idea. This was largely driven by the fact I didn't want Drupals templating engine to have another syntax outside of PHP. Many themers (a term that has yet to really be defined) already know at least some PHP so it makes some sense to use it as the core for how we theme. I also felt that using Twig could possibly limit the flexible of that we have in Drupal's theming layer and cripple those who are front-end devs in Drupal. But after reading through some of the points made here and researching Twig further - I have had a change of mind.

Twig's clean syntax I feel could really clean up our templates making them easier to understand and learn. Not having different stuff like print $var, print render($var), or show($var) and just having a statement to the affect of {{ $var}} (or whatever) I think is a huge win in terms of learnability. Re #10: I also have heard designers gloat about how 'simple!' it is to theme expression engine and believe that having a templating system that is cleaner could be much more attractive to designers. Coupled with that I am reminded of a statement Crell said some time ago via twitter:

The fewer #Drupal specific concepts we have, the more Drupal developers there are in the world.

. Being able to utilize a system that is already established could cut down on the fairly steep learning curve for Drupal theming currently. And also attract more themer types to use Drupal.

In light of that - I don't like the idea moving to a Token based system for templates. If the point of improving the theme layer is in some form to simplify structure and improve security. Why not implement a system that already provides that, rather than rolling our own? I also don't find the syntax proposed of node:field_link:0:url any less complex to understand than $content['file_link'][0]['url'].

This is good news, more love for twing in Drupal land, and Guess what... The drupal 7 version is already done :) See my github : https://github.com/renebakx/twig-for-drupal/tree/Drupal7

And depending on the implementation details for drupal 8, i'am willing to have a serious look into that.

The current implementation works as a d7 theme engine, and uses the twig node traverser to detect render() arrays, cause.. I didn't see the need for an extra render method in my templates. And yes, twig will change the way you look at templates, will bring a new way of thinking and add a extra layer of 'complexity' for not so skilled developers. But.. After you made a node template or 3 all based on the same layout with just small details, you are going to love the {% extends.... %} way of doing things. Same goes for base themes.

And did i mention the cleaner feel in your template code, automatic escaping of output, and a little forced abstraction of application logic and php logic. Sure every frontend developer cold learn php, but I yet have to meet the first frontender who volunteers to learn php.

This is the actual page.tpl.twig version of MortenDK's mothership

{{mothership_poorthemers_helper}}
{% block head %}
<header class="cf" role="banner">
    {% if logo %}
    <figure>
        <a href="{{front_page}}" title="{{'Home'|t}}" rel="home">
            <img src="{{logo}}" alt="{{'Home'|t}}"/>
        </a>
    </figure>
    {% endif %}
    <hgroup>
        <h1>{{site_name}}</h1>
        {% if site_slogan %}
        <h2>{{site_slogan}}</h2>
        {% endif %}
    </hgroup>
    {% if page.header %}
    <div class="header">
        {{page.header}}
    </div>
    {% endif %}
</header>
{% endblock %}
{% block page %}
<div class="page cf">
    {% if page.sidebar_first %}
    <div class="sidebar-one">
        {{page.sidebar_first}}
    </div>
    {% endif %}
    <div role="main" class="main">
        {{title_prefix}}
        {% if title %}
        <h1>{{title}}</h1>
        {% endif %}
        {{title_suffix}}
        {{breadcrumb}}
        {% if action_links %}
        <ul class="action-links">{{action_links}}</ul>
        {% endif %}
        {% if tabs %}
        <nav class="tabs">{{tabs}}</nav>
        {% endif %}
        {% if page.highlighted || messages %}
        <div class="drupal-messages">
            {{ page.highlighted}}
            {{messages}}
        </div>
        {% endif %}
        {# perhaps this should be block to?#}
        {{page.content_pre}}
        {{page.content}}
        {{page.content_post}}
    </div>

    <?php if ($page['sidebar_second']): ?>
    <div class="sidebar-two">
        {{page.sidebar_second}}
    </div>
    <?php endif}}
</div>
    {% endblock %}
    {% block footer %}
    <footer role="contentinfo">
        {{page.footer}}
    </footer>
    {% endblock %}

All the named block instances can be over-ridden using the extends method.

So if for some obscure reason i want to use a different head in my subtheme, i could do something like this :

{% extends page.tpl.twig %}
{% block head %}
<table>
<tr>
  <td>        <a href="{{front_page}}" title="{{'Home'|t}}" rel="home">
            <img src="{{logo}}" alt="{{'Home'|t}}"/>
  </td>
 </tr>
</table>
{% end block %}

This is enough the render the table instead of the figure and leave the rest of the page as it is in the original template.

#12, don't worry about performance. Just make sure you have APC on your production server. Twig is clever enough to check if a template is changed. And if it's not changed, it reads the compiled version from disk, of APC cache if it exist. The overhead does not really out-weights the gains imho.

As for uploading pre-compiled templates.. DON'T.. just don't.. You miss out on all the extra goodies like traversing and such, and you will not gain any speed by it. Compiling it into a twig_node object using twig itself is the only sensible way of implementing twig ;)

This issue is almost surreal. :-)

I am going to make only one small pushback: This is a decision that should not be made by PHP developers. This is a decision that should be made primarily by Front-End Developers, aka themers. Quite simply, in this discussion their voice counts more than that of PHP gurus like chx, heyrocker, or myself.

So, we need to get a decent number of Drupal themers to look at Twig, kick the tires, and see if they like that as a new Drupal theming/templating layer. If they do, then it becomes the job of the PHP gurus to make it work, one way or another, including the code-generation-security question (which I also agree is a solvable problem).

(I'm not sure how many of the people in this thread so far would classify themselves as themers, but we should reach out as much as we can to get a broad-base of themer input here.)

How about a post to g.d.o/core, specifically asking themers to take a look? Maybe at ReneB's sandbox?

How about i try to setup a distribution based on my twig implementation and drupal 7 somewhere the next days? My schedule is quite busy the next weeks. But perhaps I can squeeze it in somewhere.

As for gaining frontenders... Well I do live near Amsterdam, and already working on a evil plan to take some hostage at frontendunited next month *Insert evil laughter here*

Just a quick note to add that Twig is actually greatly inspired by JInja2 (a very popular Python template system http://jinja.pocoo.org/), which is itself inspired by the Django Template Language (from the eponymous Python framework https://docs.djangoproject.com/en/1.4/topics/templates/), which has been specifically created with web designers in mind. Actually, you can think of Django as being a CMS framework as it was created by a company with many such needs. So, the Twig/Jinja/DTL/Liquid syntax is proven to work for web designers (and for so many years now).

As mentioned before, being a syntax used by Python template libraries (Jinja and Django), some Ruby ones (Liquid), and Twig (PHP), the syntax itself is becoming a de-facto standard.

Of course, and as always, if you have any question or concern, don't hesitate to ask me ;)

I almost forgot to mention that there is also a JavaScript port of Twig (https://github.com/schmittjoh/twig.js), which brings some more awesomeness to the mix. Think of templates being the same in the backend and in the frontend!

@Crell Thank you for the "backend support". Hopefully the surrealism is a good thing in this one. :)

It really should be a themer-run initiative. I myself am somewhere between the two worlds, so I understand the pain in both sides. The themers need to step up in this one and state the requirements that the theme layer needs to run with. We should start to think theming layer from the #DTX perspective and not #DX.

@tlattimore In http://groups.drupal.org/node/157999 @Todd Nienkerk defines themers as

A themer as a distinct role is somewhat unique to the Drupal ecosystem. They work with HTML, CSS, JS, and PHP to build markup, style it, make it do interesting or dynamic stuff, and override Drupal's behavior using theme and preprocess hooks. A themer works with the entirety of Drupal's presentation (i.e., theme) layer. In the broader world, a Drupal themer is largely a "front-end developer," though not at the exclusion of being a "designer."

I can stand by that. In any case let's not turn this thread also into a "what's a themer" argument. Case closed.

@ReneB That would be awesome if you'd have the time! If you get it done this week, I pledge to take a look at it in 24 hours of you posting it. ;)

And one more thing...

We need to address the need for client side templates also in this discussion. As Drupal wants to be a really modern RESTful server, we need to accompany it with at least good readyness for JS-templating. If Drupal doesn't design its frontend with JS-heavy web apps in mind, we'll be lagging behind a long time. Luckily as @fabpot mentioned, Twig has stuff done on that end too.

Not knowing anything about Twig prior to reading this issue, this sounds like a great move all the way around. Agreed that it shouldn't be up to people like chx, heyrocker, or Crell to decide, but this already has a lot of support from other spheres of our community. But yes, let's get some more feedback from the people who are most intimately going to be affected by this change before making a final decision.

@chx: great proposal. Just don't get out of balance when Crell says this issue is surreal -- it is, in a good way. As yched expressed, it's surprising to see you taking this position after some of your previous ones. But the proposal is totally sound, well-reasoned, and well-written. Crell was just expressing some friendly and happy surprise. It's not something you need to defend against.

Not sure why this is in the DB update system component. That only contributed to the sense of surreal. ;)

I think i need to be in this thread ;)

To be honest the idea of doing {{foo}} instead of pring $foo have never really appelead to me, based on the fact that i at some point have to be able to drill down and grap some kinda value out of a field to do "nasty things with it". But to be fair to the twig (and not complain over the kerning of the logo ... the G is way off)
Is there anyway i can try this out for realz? - we have some kinda of a stable release of anything - then i will test it out on a real life project this week (my own site, is gonna be upgraded anyways) that is the only way to really figure out if a thing will work out

@mortendk there's the aforementioned https://github.com/renebakx/twig-for-drupal/tree/Drupal7 Or you can wait for @ReneB's distro.

If we're doing this for security, then I'm all in favor of this, plus I like this more than the token approach. However, if we still let themes use template.php and thus the ability to override theme functions, we're back to square one, I think, or is that simply a non issue regarding security ?

Morten, to try for real.. Give me a little time this week and I will cook up a distro with the latest version of twig in it, working and all. I must admit I did not cover to much ground in converting the rest of mothership into twig, but that has everything todo with lack of time on my side. But I can provide enough information to get everybody started with D7 and Twig :)

Swentel, I fail to see how using template.php is a security issue? Sure you can override theme functions, but closing out template.php will not fix that problem. Because one could easily do the same in a .module!
But there is more to security then just executing PHP code. In twig you could automaticly enable the escaper, so everything that is send to the browser and is NOT exclusively marked in the template as a non-escaped block is html-safe escaped.
Templates can be sandboxed, so if needed a themer could only use a given set of tags,functions etc.

The only real issue, but that is like chx and crell mentioned solvable is the fact that my current implementation uses sites/../files/twig_cache to write the compiled templates. And that is a public folder. But that is a whole different discussion, because in reality I would personally prefer to have different webroot excluding all the PHP code from the webservers browsable path.

@mortendk, you would still be able to reach down into template values and do nasty things with them, you might just have to do them in a preprocess function instead of the template itself. That still leaves the actual template layer as a 1000% friendlier (and more secure) place for the 80-90% (just a guess) of themers who would never do that in a million years.

Having used Django more than Drupal in the last year or so, I was super-excited when I saw Twig last week to discover a PHP templating language with the same syntax as Django's. It is really so pleasant to use.

I can also confirm that Expression Engine advocates often tout their templating system as a distinct advantage over Drupal's. I'm not familiar with all of the suggested problems, but overall, I think this would be 100% win.

ReneB: I guess that answers my question. The security argument just isn't valid, because as you say, it can go wrong at any level. But maybe we shouldn't focus on that.

Looking at the example in #16, the syntax looks nice. Getting rid of 'echo' or 'print' or the php opening and closing tags is going make themers less scary fiddling with template files, I guess. Only '{{'Home'|t}}' might seem strange here, because the "t" still points to a Drupalism, but I guess we can get rid of that.

I'll ping my themers at work to play with this and I'll do some testing with Display suite (and most likely panels as well).

Swentel, the | filters are needed to call drupal core functions. In the case of the |t it's a call to t() to translate. Same goes for |url or |l

I did give much thought and failed to come up with a other system todo that, that is workable for both experienced field-level theme implementers like myself *, and people trusting everything to the render() method. Ofcourse we could pre-process every variable before handing it over to the template engine. But that felt like a extra layer of fuddle.

* = When i normally implement a node.tpl I like todo stuff like

<h1>{{node.field_my_title[0].value|t}}</h1>
<img src="{{node.fid|image_url('preset')}}">

Instead of letting it being handled by your display_suite for example, but as drupal grows more and more into a UI driven system I can imagine that would, or better should need to change for D8. However I strongly believe that both should be possible, and given the current way that the render array from doom (tm) works, in combination with the traverser I implemented into twig it actually can :)

However I strongly believe that both should be possible

Absolutely, I'm not questioning that. Also, did a very quick test already and it just works fine with ds as well :)

As someone who spent a year or so with Django's templating system, I can attest that the syntax isn't all that difficult to get into. During that time, I only had access to the templates themselves, which was sometimes frustrating since there were occasions where I couldn't get to underlying data that I needed, but most of the time one could work around that.

I've only used Twig superficially while playing around with Symfony2, but it seems pretty similar. Assuming that we still have a mechanism like template.php or preprocess functions to adjust data before we get to the theme layer (which is addressed in #28/#29), then I'm definitely interested.

Just my 2 cents - I used Twig with Symfony about a year ago on a project that was decided to be too much of an overkill for Drupal. The themer had never handled Twig before but was proficient with Drupal 6 and WP. He was up and going at it in no time. Normally on a Drupal project I would of had to write all the preprocess functions in the theme myself, instead I got to focus on the backend issues.
It seems hard, but it really isn't that much effort to "get" Twig.

Twig specific discussions belong at #1441320: Evaluate Twig template engine for Drupal core (as an alternative for PHPTemplate and possibly even theme functions). I think this issue is a dupe, since the only concerete proposal in the Summary is about Twig. Splitting the energy among various issues will not help.

Twig reminds me of Smarty, which I used to love a number of years ago until I realized that compiling a different syntax down to PHP made little sense when I could just use PHP itself. Security problems, as mentioned in #30, are going to happen in the theme layer, no matter what engine we use.

Almost every reply is about using Twig, but they seem to be missing the actual point. The actual issue description, and Jacine's comment in #6 go way beyond that.

The main issue as a themer is the lack of consistency and the huge amount of options in the Theming Layer. Even if we could just make the current system consistent en simpler, we would already be halfway. A new template engine will make templates easier to read, but does not solve the mess that templates are.

I just read Jacine's proposal at http://jacine.github.com/drupal/, and for the first time, someone captured the real issue and found (a start of) a solution.

So as a Drupal Themer, I vote for these solutions, in order of priority:
- Jacine's proposal at http://jacine.github.com/drupal/
- Integration between Jacine's containers and items and the Layout Management Initiative
- A new template engine such as Twig

P.S.: I'm a Drupal Themer, not a developer :)

please.. don't let this discussion go down in yet another PHP is already a template language, so why do we need another one on top of that??. Because, like chx says, there is more to the table then just that.

Oh and for those who like reading, even Fabien had to come back on his ideas ;)

But seriously, besides the fact that the current render system uses an array for everything, and it would be a lot easier if it was a traversable object or arrayobject. I do think that having all the raw data (without markup indeed) available at the render/response time in a request is a good thing. It makes the transition to other formats then HTML a lot easier I think.

Because afteral, Twig is just an output generator, just like json, xml or whatever format is needed for a project should become a first class citizen in drupal land. Once that road is done, we could start to look at direct access trough a URI for other objects available to output generators. So we can use SSI for example, or use a more fine grained caching system. Twig, or any other template system is not going to help us achieve that in anyway.

As Jyve pointed out in#38 if the proposal is not just to introduce Twig but even to streamline the theme system and integrate it with the layout initiative all I can say is: let's do it!

From a developers point of view, Jacine's container is nothing more then an object containing data, much like the current renderable array is ;) Only the render array contains way to much unneeded cruft ;)

However it is indeed way more interessting to see what is going to happen in the wscci iniative. Not because they are leading, but I do believe that part of the key to a new render system is indeed the above mentioned ways to get a specific block or container from an URI without all the cruft.

Quick question: so would there be a twig equivalent of, say:

print t('Hello, @name', array('@name' => $name));

Or would this just have to be 'hidden' inside a preprocessing function and just have {{ message }} on the template? (Perhaps we'll still need some form of the dreadad Arrays Of Death inside templates..? )

The question should be, where does $name come from in your .tpl.php file? And why would you like to translate a string with a variable in it?

But besides that

{{"Hello, "~name|t}}

should get that job done..

(see http://twig.sensiolabs.org/doc/templates.html#other-operators )

I just want to add that I fully support and agree with @jacine. I have not looked at the benefits of twig but there is a bigger underlying problem and she addresses them perfectly. I will do what I can to support her work.

ReneB: to clarify: I'm not translating the name, I'm translating the "Hello, " part, with the placeholder @name (which is untranslated). I'd like, for instance, for the template to support showing:

   Hola, ReneB

when the interface is in spanish, and

   Hello, ReneB

when it's in english. That can be acheived right now with the aformentioned print t('Hello, @name', array('@name' => $name)); which can now be put in any Drupal template.

So my question still stands, how would that be done?

janusman: http://symfony.com/doc/current/book/translation.html#twig-templates

{% trans %}Hello %name%{% endtrans %}

please.. don't let this discussion go down in yet another PHP is already a template language, so why do we need another one on top of that??. Because, like chx says, there is more to the table then just that.

From a developers point of view, Jacine's container is nothing more then an object containing data, much like the current renderable array is ;) Only the render array contains way to much unneeded cruft ;)

@ReneB Please don't bikeshed this discussion. Not one person has been against using Twig so far, so I really don't get your replies. Also, you completely missed the point of my proposal. Maybe forget that Render API is mentioned and read it again. Twig with 300 templates will be just as much of a fail as PHP template with 300 template is. Let's not ignore underlying problems.

Also, you will need to realize that 99.999% of theme developers in this community uses PHP template today. They will have questions, and concerns. That's natural. The best thing you and anyone else can do to help move the discussion forward is be sensitive to those questions and help answer them. Again, no one has said anything against Twig so far.

Jacine, my comment was made because the previous twig discussion here on drupal.org also ended in the same discussion, and I just want to prevent that from happening.

And what is bikeshedding a discussion, because what I remember about bikeshedding when I was like 14, it was more about smooching then coding ;)

But I do think we have a communication error here, I don't think i missed the point of your proposal, I just looked at it from a coders point of view. And the way I see your proposal, is that you want to add the markup using some sort of theme_ derivative functions in PHP over containers with context on the data.
And that was exactly that's the point I was trying to make by adding the render-able array (or object) into the game.

As far as theming goes, I'am with you for 200%, themers should decided the markup and not code. And when I see you mentioning render() as a function my first reaction is.. Wait a minute.. I don't want to hand it over to a PHP function. I want to provide the HTML myself.

So in my twig world, your example.tpl.php :

<article<?php print $attributes; ?>>
  <?php print render($prepend); ?>
  <?php print render($content['title']); ?>
  <footer>
    <?php print render($content['user_picture']); ?>
    <?php print render($content['submitted']); ?>
  </footer>
  <?php print render($content['image']); ?>
  <?php print render($content); ?>
  <?php print render($content['links']); ?>
  <?php print render($append); ?>
</article>

Becomes my example.tpl.twig

<article class="{{attributes}}">
  <h1>{{title}}</h1>
  <footer>
    <img class="user-photo" src="{{content.user_picture|image_url('userpicture'})}}" alt="{{user_picture.attributes}}">
	{{content.submitted}}
  </footer>
  <figure class="image">
    <img src="{{content.image|image_url('imagemedium')}}" alt="{{content.image.attributes}}">
    <figcaption>{{content.image.title}}</figcaption>
  </figure>
	{{content.body}}
	{{content.links}}
</article>

I admit, it looks like this is more work to start with. However it allows me to independently of whatever theme_ function a developer created to control the output in the way i like it.

I see the future render-able object more as a object that tells me about the data and the context (as in the dictionary not the module) of the item that needs to be rendered. Independently of the needed output format. So that the output generator or theme engine can determine how to render the data.

@#47, that is part of the symfony internationalization bundle if I remember correctly.

@ReneB I'm just saying, cut us a little slack. I guess bikeshedding is the wrong term. What I'm trying to say is don't pound on people asking questions here. People need to feel comfortable to ask questions, and when you snap back at people in fear of repeating the other discussion you end up scaring the people we want feedback from away. That doesn't help. As far as my proposal goes, it has zero to do with the syntax of templates. It has to do with providing sane defaults, categorizing those defaults properly and building from there. I don't need to be convinced to support the Twig. I already support it. Hope that makes sense and we can let this discussion continue. Thanks. :)

As long as we can choose between Twig and PHP. I like being able to trim or array_filter on the fly.

When I review the things in #1, and read some of the comments, it seems almost everything could be solved by using these two proposed changes (Jacine's proposal + Twig). Most (me including) seem to like these proposals. I for one would like to hear some wise comments AGAINST them to have a better perspective. By thinking beforehand what would suck in this, helps us overcome them now instead of D9.

Like Twig, *love* Mustache.

Btw PHP templating doesn't have to be this ugly:

  <?php print render($content['image']); ?>

You could just write

  <?=$content->image?>

if Drupal would only embrace objects and make $content (and $content->image) an (ArrayAccessible?) object that calls ->render in ->__tostring.

There's a reason short_open_tag doesn't exist anymore in PHP 5.4. (It's always ON, not OFF.)

This comment got long, so I'll summarize. I'm a themer. Twig sounds cool, but I'm somewhat indifferent. How can the new theme system address the confusing mess of template variables and inaccesible markup?

When I saw examples of Twig in some of these threads, I thought it was ugly and didn't want to learn another syntax. After briefly reading the Twig for Template Designers page, it is not as ugly as I thought and think it would be easy to pick up. I also like the idea of a template system that prevents me from making security holes by default. By nature, us designer/themer types don't think about the security risks of templates until someone points out the danger. So hooray for fixing problems most of us don't know we have.

Extending templates is intriguing. It seems like that may help allieviate the pain from managing out-of-hand template files. It also may compliment Jacine's container/component approach.

I still want to try reneB's Twig implementation out on a site before getting fully behind it. But I have no objections so far.

With all that said, the template layer is a small part of my frustrations as a themer. I am far more frustrated when it comes to the variables that go into templates. If I want to know what variables are available for me to print, I go to my node.tpl.php and add kpr($variables); When I refresh the page, I get an array of 59 elements (a.k.a Array of Doom) -- 50 of which, I probably don't care about.

In that array of variables, there is a mix of things that can be printed directly (safe and unsafe) and render arrays that must be printed through the render() function. There is also multiple ways to print the same thing which add to the confusion. For example, here's just 3 (of ?) ways you can print the body of a node in your template:

 print render($content['body']);
print render($elements['body']);
print $body[0]['safe_value']; 

We have 1 field that is accesible in at least 3 places. In contrast, there are certain variables containing markup that cannot be altered or accessed easily, but must be overridden in order to take control. $submitted is a good example. If you want to change what is output by $submitted, you have to copy and paste the code that creates it in template_preprocess_node(). This is a good example of structured markup that a themer is likely to want to edit, but needs to dig through core functions in order to do so.

What can the new theme system do to insure all markup is accesible to the themer in a consistent manner?

@Jacine True, I was jumping conclusions, and that indeed is not fair to anybody.

think #49 is not what we wanted, really. You wanted {{ content.image|image_style('large') }} or something much, much higher level than wriitng img tags. Yes you can, no we don't want you to.

True.. to certain extend i guess.. Personally I don't want a render system making up image tags for me, because for all I care I would like the print the full image cached URL. Or in a more real situation, I want to include the full URL into a JSON string for in a inline javascript.

In theory writing a image tag like you proposed would work, if content.image was context aware. Twig is not really aware of the context of the variables it prints. It just outputs them as told using either echo $var or render($var);

Twig.js suggests the possibility of defining a single set of templates that one can use server- or client-side. Imagine scenarios where, after initial page load, the subsequent page element rendering through AJAX leverages existing templates, interpreted client-side rather than bootstrapping a theme layer response from Drupal for a fragment.

https://github.com/schmittjoh/twig.js

The most compelling chat I'm reading here is support for a clean decoupling of presentation from data. This is how we'll get to the point of simplifying the theming layer, once its roots in core are dug up and removed.

@ReneB: So I have downloaded Your Twig for Drupal and put it into a Drupal 7 installation.
As feedback from themers is wanted: If you have some simple theme you created with Twig, it would be handy if you put this on github or post it somewhere.

Being lazy and knowing others almost as lazy, this makes it easier to get a head start. I know starting a theme from scratch learns me more, but...

Most of the debate here regards whether we like Twig or not, and has focused mostly on syntax. Here's a reality: we can't make everyone happy. Such is life.

One thing I'd like to see here is to make sure that moving to Twig will solve some of the problems brought up on #1382350: [discussion] Theme/render system problems

However, I am guessing some of those questions aren't a question of syntax but rather a question of implementation. Implementation questions would include:

* How should modules talk to Twig and vice versa?
* Moreover, what's the implementation of *any* theming engine in Drupal?

In my opinion, these kind of questions will take time, will need a lot of love (and may be appropriate for a different or multiple issue(s)).

I have no concrete examples or implementations yet, but I would like to definitely discuss the overall architecture of the theme layer rather than syntax. The fact that non-theme core functions assemble render() arrays (example: node_view_multiple()) is a big problem. I'd really like to see a better delegation between data and preprocessing. Maybe I've had too much to drink of the MVC kool-aid. :)

My ideal system would be a pluggable rendering/theming system that would ship with a default engine (like Twig) and hardcore themers could replace this with whatever other system they prefer. In this fashion, the syntax debate becomes relatively moot. Don't like Twig? Ok, plug-in something else and share your patches. :)

People, stop this now! Are you seriously? Why??? Why you hate PHPTemplate? He is perfect!
Simple example: if you want to change html-tag like "img" you need to change this tag in all template files. IN ALL FILES!!!
Just stop this now. PHPTemplate perfect.

@c4rl you are absolutely right on your comment and that is what was the actual issue atm. Sorry, I think I was too vague on my previous comment. I think commenting on Twig has taken too large of a focus in an issue that addresses a larger need. The start of the conversation should be now:

1) Read http://jacine.github.com/drupal/ thoroughly
2) comment on the proposal here
She has thought the simplifying of the theme layer quite well and should be considered as the basis of this discussion.

@zviryatko please, as @webchick pointed out, this is the place for ideas and discussion. Not thwarting them and bashing. (You also might have been sarcastic...:) )

We're not abandoning PHPTemplate. We're merely considering our options at the moment and trying to make D8 theming the easiest, yet most powerful one yet. The moment we consider Drupal, or any of its components as "perfect", we've fallen from Drace (that is "Drupal Grace" :) )

I do not think that multiple theme engines are really viable, for reasons already discussed in other issues. We pick one and make it rock. Like c4rl, I have no problem with Twig but there's more to cleaning up the theme/render system than just swapping in Twig for .tpl.php file. Render arrays still destroy the soul of too many module developers and need to go away, as well.

I haven't read Jacine's writeup yet, so I will refrain from further detailed commentary until I do.

I've tried to get a distribution working, but somehow I fail to get it working. Might have something todo with the fact that twig is not on the DO whitelist.

So here is a zip of my current working version as of this evening. It's the latest stable of drupal 7 combined with twig 1.6.3 and no other modules. It includes my working version of mothership, called mothertwig. But that implementation is far from complete at this moment, and my computer is bugging me with failing mysql servers. That ate up to much of my time this evening.

To get your own theme started, just do what you normally do when you start to write a theme just make sure that the engine points to twig

name = ...
description = ...
engine = twig
core = 7.x

Also make sure that files/twig-cache is writable. This version does not change the compiled template names into something unreadable, so for those who are curious. The entire theme/ structure is the same in the cached version except well the files are 'compiled' into PHP.

Hm. The mothertwig theme has a mixture of tpl.php and .tpl.twig files. Maybe we can continue these detailed discussions over here http://drupal.org/node/1503304

I really want to try this out, but not derail this discussion with a beginners tutorial of using twig in drupal.

offcourse it has, hence the not finished working version status ;)

For non-HTML output, render arrays are just barely this side of useless. I have no plans to use them for generating JSON, Atom, etc.

ReneB thanx for putting out the mothertwig gonna play with this asap. hope its ready for production, cause im gonna jump right in & see where this will bring me ;)

For twig.js, I figure we'd feed it a twig template string and a standardized serialized form of a node/user/entity/whatever, which we already know we need to figure out. (JSON-LD, perhaps?) I don't think we need to go through a render-array-like structure to do that. Almost any format we render data to is going to be more strict with less flexibility than HTML.

@Morten, mothertwig is not ready for production because so far I only translated 3 of your tpl.php into tpl.twig. Drupal + Twig however is very ready for production. My previous employer is using it for quite some time on their production sites.

@chx, your gist is similar to what Nodevisitor.php in TFD/Nodevisitor does for the D7 render arrays.

For everybody else trying my d7-twig distro, be aware that drupal does not call twig.engine if you do a cache clear, so the twig compiler cache is not removed!
If you want to do that, you need to call twig_clear_cache(); somewhere in a module or template.php

Nope sandboxing is a bit difficult at the current stage, and part of the render array of doom needs to be solved on a different level.
For me the only reason I build that traverser was convience, it allowed me to rappidly prototype templates using Swentel's DisplaySuite and such. But most of my real world templates are build using the {% with var1,var2,var3 %} derictive to force some form of sandboxing variables as I desire full control over my output and want to be 100% sure that no module can alter my HTML.

I think that converting the current render array into a stack/tree based object could help a lot. Just like twig does at the tokenparser stage. It builds a traversable tree of items, and for drupal that object can be dispatched to whatever render system we want.

It's nice that Crell has no plans of using them, but in the real world I still think that a single system will be very usefull instead of having seperate subsystems to gather the same data just for different output. It removes a certain amount of complexity out of a system.

Re: #65: "twig is not on the DO whitelist."

http://drupal.org/project/issues/drupalorg_whitelist?text=twig&status=All is empty. That's why it's not on the whitelist -- someone needs to request it as described at http://drupal.org/project/drupalorg_whitelist.

Cheers,
-Derek

created a request, hopefully I followed the dance correctly ;) http://drupal.org/node/1504008

Edit: sorry, I worked on an outdated page of this thread. Can't delete my comment.
@mortendk There's also @ReneB's sandbox on drupal.org. Don't know which repo is most uptodate...?
http://drupal.org/sandbox/ReneB/1075966

The sandbox is behind the github version, but I will start updating that in parallel with github again. I stopped doing that because besides me, there where not that much people using it and the former license issues with third party libs.

subscribing, but too embroiled in personal turmoil to read or contribute at the moment. I'll be back in a few weeks!
:-)

In #16 :

don't worry about performance. Just make sure you have APC on your production server.

And what about drupal on shared/small hosting ? We should consider every drupal usecase, (from The White House to a small barber shop.)

Hey, people!
Have you ever used Twig or Liquid on the real project?!
It is horrible and impossible to create template a little more difficult then standard one.
Present PHPtemplater in Drupal is very flexible because of using php.
Don't even think about these useless systems as Twig or Liquid.

@pazhyn can you be a little more precise on that? We do need constructive arguments against using Twig & the new theming proposal, but they need to be more precise than "it's horrible and impossible to create a template".

What made it so bad? What's the biggest minus in Twig from your perspective? How do you think templates and theme hooks should be done? That's why we are discussing about this. To figure out what is the best way to do things from the #DTX (Drupal Themer eXperience) perspective.

#84.. uh yes.. plenty actually.. Both large and small scale projects, and #82 some of these projects run shared hosts.

Twig does not compile on every hit in production mode, it compiles to php only when the tpl.twig is changed. So the overhead is minimal compared to tpl.php files.

Twig + D6 : http://vuconnected.nl/ & http://www.concertgebouw.nl just to drop two real examples.

No Twig + D7 in production as far as I know. But if somebody has any, please share :)

@pazhyn in #84 - There are several example implementations of Twig used with Drupal mentioned in this thread. And people are working on testing these to see how Twig compares to our current theming system. Your comment leans towards just "Twig sucks" and as a themer, I would like to here about the limitations of Twig in real use rather than just a blanket comment. Could you elaborate on your frustrations Twig and give some more constructive criticism?

I think a big part of this issue is consistency. We should stick HTML in templates. The fact that I might have to even look at theme_item_list to change markup is insane. This depends on Drupal's architecture AND template engine. How do I get away from "functions" to focus on, say, design (responsive design)?

Since we're mainly talking about a new theme system, I'm in favor of Twig because it looks like I have more, of the right kind, of flexibility. I'm still investigating.

I concur with pixelwhip in #56 when he asks:

What can the new theme system do to insure all markup is accesible to the themer in a consistent manner?

Does anyone have a good answer to that question? I think that is the crux of the Twig connection to this issue.

greggles hates the fact that 100% of the custom themes (aside from a rounding error) are insecure.

wtf does this mean ?

is 'zen' a 'custom theme' ?

#91 What is meant by this statement is that preparing sanitized data for use in templates in an effective way is a priority for the theming system.

greggles hates the fact that 100% of the custom themes (aside from a rounding error) are insecure.

The theme layer lives all over the place, and has created mass confusion about where and what functionality needs to live at a certain layer. There is an issue to vent about it here: #1382350: [discussion] Theme/render system problems

Oops, damn issue queue.

Disclaimer: I'm a themer who's very comfortable with PHP.

For me, the major issues are the ones @jacine brought up in her post. http://jacine.net/post/19652705220/theme-system That article should be required reading for anyone working on improving the theme layer.

• Reduce the number of templates and theme functions
• Use a consistent approach for sending variables and markup to templates
• Have a simple data structure, without duplicate and unneeded information, that's easy to manage during preprocess
• Reduce the number of processing layers
• Use a common library of components and formats instead of a myriad of custom theme hooks

As for the templating language, have other options besides Twig been investigated? Are there other alternatives? I like how WSCCI investigated several options before deciding on Symfony.

I took a look at Twig this morning and here are the Pros/Cons I see:

Pros

• Easier syntax
• Leverage a well-tested framework instead of duplicating similar functionality in phptemplate
• Easier transition to Drupal for designers and front-end developers who already know Twig
• Improved security
• Twig macros
• Template inheritence
• Twig can be rendered in PHP and JavaScript
• Prevents ugly PHP hacks in templates by developers who don't know Drupal

Cons

• Current themers and developers that provide templates need to learn a new language
• Extra work for contrib modules (with tempates) upgrading to D8
• phptemplate is a good stepping stone for learning PHP, Twig isn't

For me, the pros definitely outweigh the cons.

However... fixing the issues jacine brought up should be our primary objective. If implementing Twig (or another template language) hampers or slows down that objective enough to keep it out of D8, then I'm against it. I have a feeling the two will go hand-in-hand quite well, but I wanted to throw out that warning.

@chx Liquid is the ruby equivalent of twig/django if i'am not mistaken?

And yes... first things first.. Decide how the current _theme() chaos can be changed. And that's going to be a real bitch to tackle. On one hand we don't want to take away freedom in markup from contrib modules, and in the other hand we don't want the output that contrib generates.

I'am trying to get the grasp of theme.inc and how/where the render array is actually being build, but i'am not sure that I am looking at the right stuff. All I see is a lot of tossing &$variables around between template_preprocess, render and various theme functions. Honestly... without breaking backwards compatibility i see no way to refactor that besides refactoring the theme() method itself to assign data to a renderable object instead of an array. That way I think it could be possible to maintain currently functionality and have a more maintainable infrastructure.

However, I'am more curious in the scope of this initiative. Is it okay to break backwards, can we suggest a whole new way of theming? Cause that would implicit a lot more then introducing a output layer like twig and will not fall lightly in the community. Basically it would mean, forget everything you now about theme_ functions and rewrite all contrib into this new system. That will slow down acceptance of D8 more then D7 did. (The positive side effect is, that it would be a good way to restructure the whole contrib part on DO, and remove a lot of 'old' modules ;))

And if backwards compatibility is a must have, which i assume it is on .module level, we do need to figure out in what form we would like to keep it internally. Should the theme_ function be region aware or not. Or should it just gather renderable data, and start rendering output at the very last level, drupal_deliver_page() or drupal_deliver_html_page() even?

Disclaimer, I'am looking at the D7 code, but perhaps I should look at the D8 code if that is way more different then D7.

Like ebeyrent in #36, I like that twig reminds me of SMARTY!
I LOVED Smarty, which helps me imagine twig as a soothing cup of jasmine tea.

While I agree with dcmouyard in #95 that learning PHP is always a "pro", learning the kinder & gentler syntax of twig sounds like an even better pro to me.

@dcmouyard

I recently did investigate other template engines, as my current employer still uses the ancient and slow smarty2, and I'am not biased by twig. Looked at smarty3, engine wise it's a good system. Lot's of people know the syntax. My main issue was the lack of easy extendable traversable token stack/tree.
RainTPL was a good alternative, very fast, twig like syntax but not that easy to extend. Dwoo, not actively maintained, last release was 2010. And the last one was Haanga, also uses the twig/django style. Looked very promising, yet a very small development community behind them.
The guys at Sensio seemed to have done another job very well, in building what in my point of view at this moment is undoubtably the most versatile template system in php land. (Besides PHP itself, but i ruled that out for obvious reasons)

Speaking as primarily a site-builder this approach is very appealing, but to really make it amazing I need the layout initiative to allow me to assign a field from a content type to a particular "container" (using Jacine's terminology) within the template (think Display Suite).

Use case (using ReneB's example template from #13)

• My institution contracts a consulting company to design and build a theme (ala #13)
• I set up 10 basic content types (blog, basic page, etc) and for each of them I set up an image field, and assign the output to the "content_pre" container using the default image format'er (selecting the relevant image styles etc. The rest of the fields are arranged in the "content" container
• Next I setup my complicated content type of video. The video is assigned to the "content_pre" container, but the built-in video format'er doesn't meet my needs at all. First I check out doing some basic wrapping around my field by trying to setup a custom format via the UI (again very similar to Display Suite), but I discover that just won't do because I need some other other fields from my content type, so instead I setup a custom output field and point it at mycustomvideo.tpl.twig as the "component" for formatting it. The video abstract and speaker info are assigned to the "content" container. Additionally I add a "share" block to my output display for my video content type and and set it to the "content_pre" container under the video component field

So in this case I can dynamically display my 11 content types and I just have the basic page twig file and my twig file that does the formatting for my component (plus maybe an info file to include any support css and js that go along with my twig file to flesh out my custom component).

This sort of capability out of the box would be a huge win for Site Builders.

One note, having a (DS style) "Manage Display" on Entities in core, might solve the "what to actually pass to the twig template file" question. Rather then passing an render array, you would pass an "Entity display object" to the template. The one I defined above would be an object with members like $entitiydisplay->content_pre.video, but if you wanted to reuse that entity display definition in another twig template or override the template for a particular context, you could could call {{content.content_pre}} or {{content.content_pre.video}} from within the template for that context.

This is supposed to be a meta issue, I'd like to see a bit more of that.
It is fully understandably that Twig is the thing to talk about most easily, as it is already there and palpable.

Re-Reading Jacines Post, I discovered, that the main point she is making is quite a bit different.
If I get it right, it has a lot to do with the stuff we do no matter what templating engine we use:
- strip out _all_ html from system files, if possible
- make a clear separation between content and markup
- rethink markup from the ground up to not have it an afterthought
- create those awesome swappable and combinable Html elements and formats so Module builders and core devs who want to concentrate on the PHP part do not have to deal with them but can re-use them

Actually those points appear more at the root of the problem to me.

Having that in place PHPtemplate could be much more generic to use.
I find this gets somehow lost in all the template engine discussion, as the engine is no magic wand to solve deeper-lying issues.
I think there was a very productive sprint in Denver that tried to come up with some of that in reality.

Maybe it would help to strip out the Twig discussion to a subissue, or create a subissue "Evaluate other template engines" to keep it more open.

If there are already sub-issues, please someone add them to the issue summary.
It may be too early for sub-issues. But this talk now centers almost exclusively on template engines which should be reflected in the issue title, if so.

Repeating things like "Read comment #68" is problematic. Please use the issue summary to highlight important points. That's what they're there for. :)

There are some existing issues to fix some of the root issues already. Over the next few days, I'm going to gather them up and create some new issues, which I will link here to handle the separate parts.

For me, the question right now is where do we start and how will the override/inheritance system work? Is it okay to start refactoring existing theme functions and assume the same system we have now? I'm not sure if Twig changes any of this. Should we start discussing this here? Or should we just open another issue for that right now?

There already was a sub-issue at #1441320: Evaluate Twig template engine for Drupal core (as an alternative for PHPTemplate and possibly even theme functions). Since a lot of the discussion here has repeated things that were already said there, I'm not sure closing it was a good idea :( But be that as it may:

1. Regarding "magic", I'm not totally convinced there's anything in #68 that we couldn't do without Twig? Instead of {{item}}, you can already do render($item) and it's safe to call that on strings. It could become a coding standard to just use that everywhere.
2. Regarding security, I'm not sure I understand the security benefit of Twig but maybe I'm just missing something. If we wanted to run everything in the theme layer through check_plain(), then yes, Twig provides an easy way to do that automatically. But the problem is that we actually need to run some things through check_plain(), some through filter_xss(), some through check_markup(), etc. How does it help with that?

The benefit that seems by far the most interesting to me is the "same templates in JavaScript and PHP" thing... But that's down the road anyway, since that will never work as long as we keep calling PHP functions from inside template files (which we do a lot of currently). Nor does this really make sense if we continue to have lots of complex logic inside the template files; because if you're going to have complex logic, it's way better to express it using an actual programming language (like PHP) rather than inventing a new one. So I agree with others that it seems early to worry about a new template system, though if it's part of a bigger master plan it could definitely make sense.

And there's more reasoning in #68 for Twig as well: besides security we get the ability to do magic.

I believe what @chx is referring to that fact that by using Twig we can solve some of the really hard issues that we've been going around in circles trying to resolve with PHPTemplate. For example, with Twig we can render() in the background without having to expose that complexity to templates. There are also ways to specify the format (plain text/HTML/url/whatever) which could help us get at field values instead of having to do deeper into say... field.tpl.php.

I think these are the sorts of things that themers need more clarity around? I'm not entirely clear on all the specifics of Twig myself, but I think going into more detail about these features as they apply to real Drupal use cases would help.

Adding Nate and myself.

I think the most important thing about Twig is it moves the hard problems like security/escaping and most everyone seems to agree in rough strokes. Lets let chx hammer at some implementations see what he comes up with and discuss the pro's and cons of what we see.

added #68 up here

Tried to add some more headers to structure the issue summary better. Feel free to improve on that.

ReneB: As chx said, BC is a non-issue. We break APIs between major versions on a regular basis, provided that the new system is notably better than the old one. (Sometimes we get that wrong, but that's the goal, anyway.) The clear consensus at DrupalCon Denver was "the best patch for the current render/theme system is rm -rf. OK, so now what." So yes, proposed changes that blast away theme() entirely are definitely on the table if they address the underlying goals (ie, Jacine's post, same template server and browser, easier for module devs to use in a way that doesn't make themers want to knife them in a dark alley, etc.)

For me, the question right now is where do we start and how will the override/inheritance system work?

I'm on the same page with @jacine here. I think we should start by going through what we have at the moment and what we really need to salvage from there and feed the rest to the hungry kittens.

Currently we have:
- PREPROCESS functions which provide preprocessing for templates and theme functions. Are additive (instead of overriding).
- THEME functions which modules provide and users can override. Theme functions always override the original.
- Template.php for theme-level theme- and preprocess-functions.
- Template files (should) provide (only) markup with (minimal) PHP.
- Sub-themes (just to remind us that this needs to be covered also)

Now @jacine's proposal gives the basis for structural solutions in theme functions and template files. But the other two then. My first question is that could we throw away preprocess-functions as they are at the moment? It confuses a lot of people, when we have both preprocess and theme functions. We should be able to add similar kind but simpler functionality into the new theme functions, if needed at all.

Template.php has always been the themer's own playground, and we need a place to handle the theme functions in themes. But we do need to make it safer. I guess Twig would provide a more safe and sandboxed solution for this.

EDIT: Just to be clear, I do believe that

the best patch for the current render/theme system is rm -rf. OK, so now what.

is the awesomest but longest route, but we should also consider the things and patterns that are working (if any?).

#93 Thanks for the link.

PREPROCESS functions which provide preprocessing for templates and theme functions. Are additive (instead of overriding).

We also have process functions and the ability to implement some alter hooks. :)

My first question is that could we throw away preprocess-functions as they are at the moment?

Personally, I like preprocess functions. Process functions I can totally live without. I think we should try to use them way less, but that we would need to see how far we can get in templates, and how efficiently it all works before attempting remove preprocess functions, personally.

Template.php has always been the themer's own playground, and we need a place to handle the theme functions in themes. But we do need to make it safer. I guess Twig would provide a more safe and sandboxed solution for this.

I was guessing that theme functions would just turn into Twig templates. Maybe not?

the best patch for the current render/theme system is rm -rf. OK, so now what.

If we do this, then we're back to square one for all the problems Render API solved. Yeah, there's bad parts but there is good stuff too and the main complaint as it relates to theming has been the complexity and inconsistencies it introduced in the templates (which would be resolved with the move to Twig as I understand it).

So... if not, Render API, then what? Please elaborate. :)

I get that we're trying to remove function calls from the templates, which is a very good thing. I hate the fact that the templates are littered with render() calls. These could easily be moved to preprocess functions, but then we have a lot of preprocess functions, which I understand confuses some themers.

Twig doesn't really solve this though - you still have to process the variable somewhere in order to output {{foo}} in your template. And calling {{foo|filter_xss}} or {{foo|my_custom_function}} really doesn't change the fact that you're calling a function, it's just less clear, IMO.

@ebeyrent: chx in #69:

Here's https://gist.github.com/2210421 the integration of Twig and drupal_render. My template consisted of {{ build }} and it rendered. So we do not need to write {{ build | render }}, just {{ build }} works. Obviously #68 will need similar code. It's not a lot of code.

@chx Re: 100 Thanks for your reply.

Instead we will create data structure well-mapping to the relevant HTML structure and do just that

I agree with that approach, Sorry, I don't think that I was being clear. I wasn't proposing passing the entities to the template system my proposal was:

1. have a Display suite style mechanism (manage display) to map an entity/field bundle to the display data structure/layout in the ui
2. use a recursive structure of plug-able objects implementing layout/container, Item, format and component interfaces/implementations to act as the data structure rather than arrays

(1) and (2) are not in any way dependent, in fact with a combination of twig and a ui based way to set up the display data structure/layout the concerns i have would have with using separate arrays and theme functions is mostly moot.

@tim.plunkett Sure, that takes care of render(), but it illustrates my point that for anything else where we need to set the value of a variable, we still need to call a function/filter in the template or use a preprocess function to pass in the variable. I don't think the extra abstraction makes that simpler or clearer.

I don't know if you're familiar with Microsoft Razor syntax, but it seems Twig is a php version of it and Twig is even better, http://weblogs.asp.net/scottgu/archive/2010/07/02/introducing-razor.aspx

The point is that simplifiying the syntax is a way to go in the future of templating.

As a themer, using less logic in the tpls is desirable, I think using Twig will discipline most developers who are guilty of making tpls unreadable.

I always make websites with Drupal where I am both the developer (with basic PHP skills) and the themer. I said developer because you really need to delve into PHP if you are serious about themes. But Drupal is growing up and we should aim at separating concerns where the themer is the themer and the developer is the developer. If with D8 thanks to the layout system, a better theme system and a powerful templating system I can forget PHP, well, you're making me happy. I don't really want to mess with arrays and if statements to print a field and I always want to be sure I'm not compromising the security of the website. Was PHPTemplate a bridge to learn PHP? Yes it was. Am I now a skilled PHP developer because of that? No.
We're making a decision here about the people who will be involved with Drupal in the following 4 years. If we want to attract skilled front end developers we need a clear, easy to use theme system and a powerful and secure template system.

as much tempting as rm -rf sounds... I'am concerned about the time frame of getting things done ;)

I do believe that ruling out a lot of the possible hooks (http://api.drupal.org/api/drupal/core%21includes%21theme.inc/function/th...) out of the current system could be a more reasonable approach.

I have few hope there will be enthusiasm for my proposal, but to me, the ultimate templating system is... XSLT.

You have structured data on one side (XML), you transform it into structured data (could be HTML, JSON, anything) on the output. I have the feeling that this could fit well with the Web Services and Context initiative :

Request => Data => XML => XSLT => HTML.

I've been dreaming about that for years. I'll try to develop, even if I'm not very confident I can convince people here, XSLT is not a very loved technology, although it is very powerful (and not THAT hard to learn).

My answer in short : XSLT+HTML5 != best friends at this moment in time.

Besides that theoretically XSLT was a good idea back in the days, but in practice I do think the learning curve is even steeper then phpTemplate, Twig or Smarty for most themers.

-1 to removing PREPROCESS functions. I use them extensively and they were designed to be used NOT by themers but by developers. It's a great place for all the PHP code needed to create the custom variables that will be needed by the theme. There should be no reason for a themer to use PREPROCESS, so I'm not too worried about whether they are comfortable doing anything with them, but it's invaluable for developers.

The goal behind that part of the system was that all the PHP manipulation would happen in PREPROCESS, and it would result in nice clean .tpl files that just spit out the variables in the right places.

I think that idea was a big win and we don't want to lose it.

Also the PREPROCESS and THEME functions are separate. You won't have both. Either you have a theme that uses a tpl file (and all tpl files can have a related PREPROCESS function) or you have a theme function that does not use a tpl file. The theme functions were for small things that didn't require a tpl file -- little sub-elements like theme_list().

So you can't talk about PREPROCESS and THEME as two ways of manipulating the same thing, they are two different things entirely.

We could get rid of the two different ways of doing things -- get rid of either theme functions or tpl files. But I suspect that is not so easy to do either.

True, I actually agree on keeping preprocess functions, but wanted raise some thought into it as we haven't touched processing functions yet. But perhaps removing them out from themes, as to not being able to add theme-specific preprocess functions. That way it could be focused for the developers.

My main argument that we should reduce the points of contact for themers and make it simpler to find the correct function to override. Right now, if some custom component comes from a contrib module, we can't be sure whether we should edit theme functions or template+preprocess-functions without going into the module files and scanning them through. Of course there's devel themer, but that merely is a helper to mask the real problem of theming complexity.

There should be one set of functions, with logical inheritance as has been suggested here.

So... if not, Render API, then what? Please elaborate. :)

Sorry @jacine, I didn't realize that the quote had also "render" mentioned. I'm all for keeping the Render API and the goods it provides. Just the fact that the theming layer needs to be rethought ground up. My miscommunication there.

#122

There should be no reason for a themer to use PREPROCESS, so I'm not too worried about whether they are comfortable doing anything with them, but it's invaluable for developers.

Huh? What are you defining as 'themer'? I'm a themer and I have used preprocess functions more times than I can count.

I would absolutely love to have *everything* work in a single way. I don’t know if my thinking on this is outdated but there were performance problems with using templates for theme calls that recur multiple times in a single request. Calls like theme_menu_item.. But if we can do away with very granular theme calls and have them grouped as structured output it may not be a problem.

Also, how would the variables from preprocess map to what’s available in twig templates? There’s bit of a disconnect since they are different languages. Any thoughts on how to line them up so it’s as sensible as it can be?

I think #49 is not what we wanted, really. You wanted {{ content.image|image_style('large') }} or something much, much higher level than wriitng img tags. Yes you can, no we don't want you to.

Couldn't we think about leveraging whatever theme engine we decide to implement to "sub theme" those little elements that have html inside and are outputted afterwards inside another template ?

E.g. In node.tpl.php, we have {{submitted}}, which outputs some HTML. This piece of HTML should also have its own template.

What I mean is that if we want a unified way of theming things, it would be great if any piece of outputted HTML passed through a template file. A themer should never have to ask himself "How the heck do I change the HTML output of that variable inside node.tpl.php ?" If a variable outputs some HTML, it should be overridable via a template file.

Makes sense ?

@chx :

What we do not want is hook_page_alter, #pre_render, preprocess, process and heaven knows what else

Just a note that #pre_render is currently what makes "manage field" and its rich enhancers (fieldgroups, DS) possible at all.

Alos, #pre_render is *the* step that gets executed on a form that fails validation, and is therefore the canonical spot for some pre-display massaging (like putting the rows of a tabledrag in the correct display order)

Front-End Developers and themers would love that! +1

How do we handle fields in objects? How do we differentiate between the user's data and the user data?

Could we use a template language that speaks English? content.image or image_style instead of content.image|image_style.

Twig might have security because it accesses data indirect but that happens when you use object oriented programming. Twig might have a nice compiler but some of the cache systems, including authcache, give you something faster. What I am not seeing in this discussion, when I work as a themer, is how we identify data and that causes more problems that the other stuff, especially with language thrown in as an additional variable.

PHP templating was considered a great leap forward over Coldfusion and the Twig syntax shown on this page is essentially cryptic Coldfusion/Smarty/CICS templating. We still need page after page of comments in the template saying what data is available.

Here are some helpful points from our current system, including the token system, that I would like to throw into the user interface side of theme generation, with or without twig.

When we edit a node field and select Full HTML or another option, we get stuff at the bottom of the box saying what is available. If the same could be available in a theme component edit screen, life would be easier.

Some of the edit boxes in D7 tell you what tokens are available for that section. Would be nice to have a template editing system listing all the available variables at the bottom of the edit box.

Editing components then assembling components works extremely well. Twig supports components. If I am editing a list (page, box, or whatever), it would be great to edit the heading, the row, and the columns separately than drag and drop the columns into the right order. Each edit section could show me exactly what is available and not confuse me with more.

The Ajax version could have all the boxes/regions/divisions open then have the available data listed as an expansion of whatever I mouse over.

There were a bunch of really neat theme GUI ideas in a presentation at Drupal Downunder in Brisbane. The degree of presentation depends on the underlying ability to identify the available data and structure.

Getting back to the decisions on data. I will use an example from tokens. Tokens have user data available. They also have current user data available. How do you differentiate those in a theme using the Twig syntax?

My suggestion is that the difference is not a problem if you provide a way to edit themes in small reusable chunks, you provide a GUI for editing your chunks, and the GUI explains what is available in a similar fashion to the current token system but with the added advantage of knowing the environment available to the chunk.

Current I solve similar issues in D7,6,5,4 by placing components of the theme in the bodies of nodes, editing in the node, then using the theme to assemble the result. The node body can have any number of modules working on the content. Each edit can be aware of the available environment.

if you want a complicated use case, try the invoice sent out by Ubercart in an email. I want to theme the tax component because Australian GST is different to British GST and I want the changes to work everywhere GST is displayed. I also want my invoice theme to work in the email and in the page where people look at previous invoices, all with one template for the invoice. I want my email to the customer to use the same template with, or without, an invoice.

I think Drupal is more flexible than any of the examples of Twig show. The advantages of Twig syntax might be relevant to the previous millennium when people edited template files. Drupal 8 will be in the land of gestures. The important things will be the ability to theme components that can be reused and the ability of the GUI to help us identify data when editing a component.

A couple of questions on syntax. How would the Twig syntax access the city field from the addressfield module when there are multiple address fields for a user and we want the business address? Now throw in internationalisation. How do we access the city when our site is in English, the visitor sets their profile to German, but their business address is in Beijing?

(re #131) Just to respond to your last paragraph which poses a couple of questions: I'm not sure that any of the stuff you mentioned should be the job of the theme to do. In my mind, all that stuff should be performed/worked out before the data arrives in the theme section. For example, in the address case you mention, the theme should just be presented with $business_address (or whatever it happens to be called) rather than an array of addresses with which it must figure out what to do. All that figuring out what address is required should be performed beforehand.

Ideally, business logic should be kept out of the theming process as much as possible.

I think this issue is running away and delving into too many changes that aren't manageable. It is full of people saying things like "I see no need for XXX, let's get rid of it", and other people saying "XXX is invaluable to me, you can't get rid of it". The fact is there is a reason why all these things exist and they are all being used by various modules in various ways. So if we propose removing them we have to do a complete inventory of all the places they are used (both in core and in contrib) and evaluate what will be hurt by removing them.

That's a tall order. And it won't get done.

I propose we limit the scope of this issue to something more manageable. For instance there seems to be a lot of agreement that it would be nice to get render() out of template files, perhaps by using twig. That is a manageable project that might reasonably be do-able for Drupal 8. In addition to exploring how we like using twig we have to investigate what is lost by losing render() and be sure it is still possible to do what needs to be done, both in core and in contrib.

The other thing I see as a *big* topic is whether we could/should get rid of page_alter. That is an other manageable topic that needs further investigation to see what the impact would be.

My gut feeling is that those two items are as much as we can tackle for Drupal 8, especially since there are other initiatives that might be making major changes to the way that pages are constructed, which in turn might completely change the landscape of what might need to happen to the theme system.

Could we use a template language that speaks English? content.image or image_style instead of content.image|image_style.

Actually we have that.. even with twig.. I now realize my example was sending the wrong picture (get it... word-joke ;)) At my former employer we liked to control every aspect of the HTML, so we did everything by hand. But if you are used of doing things trough the UI of drupal, you don't need to if you are okay with preprocessors tinkering with your output.

Drupal 7 example :

If a image is coming from fully loaded node, and in the display mode (full, teaser, custom) of the image in that node you added the image cache preset, either trough the normal content type admin screen, of the advanced screens of display suite. The image field in $vars['node'] contain information in the render array. So you do not need to set or | the image_cache preset. Because the automatically called render function in my twig version automatically detects the availability of the render array and outputs the image according to the #theme hooks set in the field.

To make a long story short :

{{ node.field_my_cool_image }} 

would render the complete image tag including cache presets and all, given that the pre-processors haven't removed any of the render array settings.

I think I really like Twig, and support many of the ideas expressed above.

It also occurs to me that once we have our "ideal" input data structures fairly well defined and implemented, it might be possible to mostly port them to the Drupal 7 (by mangling the render arrays) engine version and allow themers to port(ish) their themes in advance of Drupal 8. Heck, if we got the input arrays close enough we could even consider adding this to Drupal 7 core as an additional non-default engine.

A couple of questions:

- I think the potential for security improvements is good - although I am a bit concerned about discussion of filtering in Twig itself {{foo|e}}. I think we need to have much clearer separation between filtered and unfiltered content (oh, if we could tag strings with a "filtered" attribute!) - hence either we should ensure all strings are filtered before they get into the template (as we do now, in a fairly broken way), or we should only ever pass unfiltered strings into the template and "|e" every. single. string. My sense is that the former is much stronger from a security point of view, in which case why would we ever escape anything in Twig?

- This is probably better discussed in a sub-issue, but using render arrays for template building (which I have never been a fan of) descended directly from using form arrays for declaring, validating and submitting form. They are the same thing. Which begs the question, how far up the chain do we go - replacing all of FAPI (with what?) sounds pretty overwhelming, so where would that leave us? Would we go back to using render arrays only in forms (or perhaps form-like situations, where we really really need alters) as with 4.7.x and 5.x, and use Twig templates (with template names replacing render callbacks?) for rendering each array element? Something else?

http://acko.net/blog/safe-string-theory-for-the-web/ is an interesting read. https://github.com/facebook/xhp/wiki is also an interesting comparison point, perhaps?

@ Owen Barton - There is some discussion about Form API and other things that could be replaced by Symphony components over at #1263478: Identify pieces of Drupal that could be replaced by Symfony code, but I don't think that is anywhere near the scope of this issue (Twig), at least it shouldn't be! I am strongly in support of using Twig for a new default theme engine, and agree with Owen that it'd be much less painful a transition if we could get some sort of backport so developers could start working in Drupal 7/existing sites to get themes and theme implementations ported.

There are already enough core components changing that it's hard for one person to keep track of them... I'm already looking at a few days' work just to get the contrib modules I maintain to compatibility with the current state of Drupal 8. That's not counting all the things that will change as a result of this and a few other far-reaching (but important!) issues.

Finally, there's no way we can take out preprocess functions, imo; they're one of the few things I love about Drupal's theming system currently, and they're used everywhere. Maybe process functions (Asks self: have I ever used one?), but not preprocess.

#133 I agree with divide and complete instead of trying for the Big Bang. Change rendering first. A separate project would be tackling all the ways we assemble a page. All the code and modules that group components into blocks, regions, whatever. That seems to be a good target for a layer under the render layer. If it was done the right way, we could theme an assembly without worrying too much about what will be in there. The overall them layer would then be a clean HMVC.

From the tag/token/id perspective, a component might be identified as footer. The tags/tokens could include the form :menu where :menu, within the footer, would become footer:menu at the system level. Preprocess functions could be supplied with 'footer:menu' and decide what needs to be delivered.

re #106 above

by using Twig we can solve some of the really hard issues that we've been going around in circles trying to resolve with PHPTemplate.

Very few of these problems are the fault of PHPtemplate. All PHPtempalte does is insert variables. That's the same thing twig does, though in a different way.

It's what Drupal's doing with PHPtemplate that's causing most of the confusion, and we shouldn't fool ourselves into thinking that adding a new abstraction layer on top of PHP is going to solve any of these problems. Yes, twig may make the template *files* more secure, but guess what, drupal is still made of PHP, and beginners will still find a way to do what they need to do - perhaps insecurely.

For example, with Twig we can render() in the background without having to expose that complexity to templates.

We can do exactly the same thing using tokens in PHPtemplate, and probably other ways too.

There are also ways to specify the format (plain text/HTML/url/whatever) which could help us get at field values instead of having to do deeper into say... field.tpl.php.

Also possible with PHPtemplate, tokens, and the drill-down affect.

Twig does give us the benefit of looking cleaner on the page than PHP, but it also has the *huge* drawback that it's something entirely new to learn. Now you can't be a drupal theme dev - even if you know PHP - until you learn Twig *too*. And once you master twig, guess what, you'll want to write a module. And if you didn't know PHP already, you'll need to learn it now.

This really increases the learning curve for getting into Drupal. How many people who are new to Drupal already know twig? How does that compare to new people who already know PHP?

I don't have much of a coherent opinion for my own use of the theming system, but I do have an integrated systems sort of perspective.

I am helping to build a platform which will have templating components in PHP, in JS, and in Java. If there is a library or syntax that would allow me to unify how markup is assembled, that would be a huge win on the Theming Experience for my team. Anyone else looking at multi-technology-sourced UI components would probably feel a similar flutter of stack simplification.

Not sure it's worth wandering off the map if the community doesn't go this route, but there are certainly advantages.

Cross-post.

Is still feel like we are missing the point here.

As a themer, I don't (should not) care that we have process/preprocess functions, hook_page_alters etc, because for me those are tools to change the data, not the theming.

The real problem is that I have to be in those functions to change the HTML output of my site. If only we could make a better/clear separation, then no themer would be complaining.

So, let's just focus on the main issue and stick to Jacine's proposal to put consistency in the templates/functions.

... but using render arrays for template building

Actually that made me think last night, that keeping a structure like the render array is not even such a bad thing after all to accomplish template compatibility between D7 and D8 if we follow some strict rules in the way we write theme code.

The render array contains meta information in the #theme array which methods or functions should be called when using the render function. If we can keep that structure the same, it would mean that we could keep the created templates in D7 for D8 just by looping everything to the same render() function. Only the underwater _theme functions would be different. And if we decided to actually use twig, we could even use the node traversing system to change the elements that need altered or render different on compilation time and never look back at it again :) That is the power of using a domain specific language for template's, the syntax is 100% predictable because of the limitation in available commands.

So, let's just focus on the main issue and stick to Jacine's proposal to put consistency in the templates/functions.

Agreed, and that is just what i personally am trying to accomplish, but it is not as black/white as it sounds. Jacine's proposal is on a abstracted level, which is a good thing! But it somehow needs to be translated in to code, and one of way doing that is weighing the pro's and con's of the current system against a full rebuild. And at this moment in time, I believe more in refactoring in what we have and moving it into a single responsibility object, what I think Jacine means with the container, opposed to the current system where stuff is added and duplicated on several levels to a humongous array where every theme_ method cherry picks on render level the content needed for it specific purpose. Every entity should by rule only occur once in the renderable object without any markup present and define what methods could be used to render the object. The renderable object is passed trough a preprocess function to allow for just in time modification and then passed on the render layer. And the render layer should be a theme engine that merges the markup with the object. Preferably a domain specific template language instead of php because of the above mentioned reasons.

#141

The real problem is that I have to be in those functions to change the HTML output of my site.

The real problem is that HTML is used for both data structure and theming. <table> is used used to indicate a data structure in the format of a table. <table class="example"> is used to add theming. If the source function does not add class="example" to the table, your theme cannot add CSS. You then dive into the preprocess function to add a class.

Or the preprocess function adds class abc and you refuse to add CSS to abc so you modify the preprocess function to add class xyz.

One solution is to change all HTML to XML so it is only structure then transform the XML to HTML in the Transformer layer in between the Assembly layer and the Render layer. As an example using the search box, the Assembly layer would produce header and find it includes assembly search. The search assembly would go through the transformer and get classes of header, search, header-search, etc. The render layer can then add CSS.

#143 : Totally agree, and who's better than XSLT to transform XML ? (Yes, still advocating for XSLT ;) ).

XSLT is great if you can assume that everyone already knows XSLT, which is not the case for Drupal. The learning curve is otherwise considerable compared to any of the other choices listed in this thread.

Also at Liip we used to mainly due XSLT, however all the die hard XSLT-forever people have switched to Twig with essentially no complaints (which was previously unthinkable as every other template engine led to months of whining and moaning).

#144, #145, My suggestion was not XSLT, although I did replace a major corporate Java site, many person years of code, with a few lines of PHP and a couple of thousand lines of XSLT, and love the super cool kudos of knowing a language nobody understands, not even me.

My suggestion was to produce XML with no formatting, which is compatible with HTML if you do not adopt the SGML variations. The render layer would apply the parts of HTML that are not structure, including the wiggly bits allowed under SGML.

If you want to use the almost meaningless word context, the XML could carry the context if the data is to be processed by Ajax because, in an Ajax system, the context has to flow with the data.

If you did choose to use XSLT, instead of an alternative such as Cell, the XSLT could be in the Assembly layer to construct larger assemblies and in the transformation layer to construct something that could be rendered. Neither layer need be visible to the theme constructor.

The point is to separate the structure of the data from the formatting. If we keep the structure in PHP arrays, we have to process it in PHP. If we switch to XML, we can process it in hundreds of different ways. Json is an alternative if everything in the system is written in Javascript but is a pain for anything else. XML is a format that can be sent anywhere for transformation by anyone/anything.

Transformation for the current Web is really about converting structure and identifiers to something understood by CSS. CSS currently only understands HTML tags, HTML ids, and HTML classes. The conversion of something like header:search:term to class="header search header-search header-search-term" happens in transformation and/or render, depending on how you construct your layers. If the transformation layer is pure and perfect, rendering is almost entirely CSS. XML is also a nice format to feed to tablets and pads.

From a front-end developer's point of view I would say my first concern would not be the template engine of choice.
I can deal with PHPtemplate AND/OR with twig, because from what I see in the examples and documentation, twig seems to be pretty straight forward.
And I can see the pros you have pointed out.

My biggest pain point is found in the inconsistency and obscurity we have now. When I first came to Drupal it took me a really long time to figure out at which point in the process I could override the markup that would be rendered. And now as a trainer for other front-end developers I'm having a hard time to explain the reasons and methods for "advanced" theming. There should not be such a thing as "advanced" theming, where you rely on tools like krumo(), dpm() and using a debugger just to find out which function or template is responsible for creating or overriding the markup. Plus there should not be the need to memorize all the variations and exceptions we have now in the theme layer.

As a German, I know how hard it is to learn a language like German, where there are practically no rules whether something is male, female or neuter, so you have to memorize all the words AND their gender. NO Drupal is not that hard ;) but you can see the point in having a system and rules, from which you can derive consistently how something is built. Staying in the language metaphore it would be nice to have something in place like in Spanish/Italian where you can say "the noun ends in -o, so we can be 90% sure that it is male". So in the end you would only learn a manageable set of exceptions, because there are and always will be exceptions.

So translating this (whew->word play) for Drupal, it would be nice for a themer to have to know only one system to access the data and variables available and only one method for creating or overriding the wrapping markup regardless if it's "the big page.tpl.php" or the small $submitted.
The above combined with a sane default markup and reusable patterns would be a big step forward

#138

Twig does give us the benefit of looking cleaner on the page than PHP, but it also has the *huge* drawback that it's something entirely new to learn. Now you can't be a drupal theme dev - even if you know PHP - until you learn Twig *too*. And once you master twig, guess what, you'll want to write a module. And if you didn't know PHP already, you'll need to learn it now.

This really increases the learning curve for getting into Drupal. --Jen Lampton

This is what makes this particular neophyte want to run and hide. Consistency would be great across theme files; lean HTML code would be even better, especially for people on dial-up or bandwidth restricted Internet access like me and the people around me who still have yet to access high speed.

But to try to learn a new way on top of what I've been learning from 6, then to 7...is overwhelming. I'm trying, guys.

Are we really sure that usign a different template langage will bring security? Imho what is insecure is using print($variable) everywhere, as is insecure to use {{variable}} without escaping / validating / wathever.

Maybe some effort could be made to provide some safety nets, like :

print_safe() -> escapes everything by default
print_raw() -> print raw variable content
print_int() -> convert to int before printing

etc...

And this only works if we educate themers to use the correct function/tag/wathever to output stuff. Escaping by default will eventually help people use the correct function. Or maybe they will start to print_raw() everything. Go figure...

This said, the template engine discussion is a neverending story. at the end, I'm for PHP as it is what most people will know at least a bit.

Maybe some effort could be made to provide some safety nets, like :

print_safe() -> escapes everything by default
print_raw() -> print raw variable content
print_int() -> convert to int before printing

Drupal already suffers from the 'much functions who roughly do the same' disease and adding more wrapper functions would imho only make things worse.

And straight from the manual http://twig.sensiolabs.org/doc/api.html

strict_variables: If set to false, Twig will silently ignore invalid variables (variables and or attributes/methods that do not exist) and replace them with a null value. When set to true, Twig throws an exception instead (default to false).


autoescape: If set to true, auto-escaping will be enabled by default for all templates (default to true).

So yeah.. output is properly escaped. Compiled templates could be moved to a place outside of the document root with the use of private:// instead of public:// given that the drupal structure would provide a safe place for private files.

oh... and check this : http://twig.sensiolabs.org/doc/recipes.html

The first one was particular useful in creating a mobile template with the same data as the normal one. (pre-respsonse era). I did some browser sniffing in PHP, added a variable in the page_preprocess to instantly switch between mobile and normal with the need of writing multiple themes as the inner content scaled pretty darn good inside a fixed mobile container.\

@philippejadin
Problem is, we already did all that. And that sort of thinking got us in this mess to begin with.

I support Twig, or any other option that would move us towards *less* flexibility, and more usability, where we have simple templates that output data, instead of calculating the answer to life, the universe, and everything.

42... there you go, write it down down everybody, so we neverever have to calculate in a template again ;)

autoescape: If set to true, auto-escaping will be enabled by default for all templates (default to true).

As I pointed out earlier, and as @philippejadin said now also, we probably can't use that (or rather, we could use it, but then we'd have to make the themer responsible for unescaping all the things that don't use the default escaping strategy and/or were already escaped earlier).

Again, I don't see how that's more secure than our current system; in our current system, we generally don't ask the person coding HTML in a template file to make these kinds of security decisions at all, but rather we pre-sanitize all common/documented variables for them.

If people are persisting with the idea that Twig is more secure, it would be great to see a realistic code example that can be critiqued to determine if that actually is the case.

I'm going to expand on chx's comment a bit because I agree with him. I think the biggest gain of Twig is shifting the responsibility for the hard security or functional things off of the theme developer and on to us as core developers. We don't have to bang "print check_plain()" and "print render()" and "Read Steven's string theory and understand how it applies to templates" into themers heads and can provide shortcuts and automatic handling which sounds like what we're hearing by things like the token theme suggestion and other discussions. It is not a silver bullet because nothing is but it more closely aligns concerns with the right people.

We also shouldn't confuse increasing the learning curve with adding something for old drupal user's to learn. If you're just working on a template, learning twig is not hard and a similar syntax to other popular template systems. It also only took me about 5min to get the basics and I've never even used those other systems. With a clever doc page we can probably cover the important points and the Drupal specific stuff in about the same time and that's really not to much to ask.

I love the idea of using Twig, while I have not used it specifically I have build themes for Shopify which uses Liquid and loved how powerful and simple that templating language is. I am fine with PHPTemplate and can do with it what I need, but I have thought many times how awesome it would be to have something like liquid in Drupal as it makes theming for Drupal more accessible for people from ruby world.

To me Twig/Liquid to Drupal/Shopify theming is like jQuery to javascript, I can do without it but if I have it I can focus on doing what I need faster.

Also to in regards to having to learn new language/system, I remember when I was starting with Shopify and had to learn it, it was quite natural and easy.

I love where these discussions are going and would love to support the efforts in any way I can.

tl;dr: Twig's great, but not the whole story.

More of the story:
I love all the energy that this issue has raised. It looks to me like a bunch of front-end developers have commented here with a fairly positive opinion of Twig, which is great. That's extremely useful feedback. The comments expressing negative opinions are also useful.

Beyond the syntax advantages/disadvantages of Twig itself, what I think is at the heart of this issue is that for D8 we can move from a push model to a pull model. Our entire #pre_render/preprocess/process/etc. model is a legacy of PHP 4. We have all these functions because we need to provide templates with ready to use variables. But we don't know what the template will actually want to use (Drupal themers are always building more interesting and intricate sites than what core and module developers can predict), so we end up throwing almost any variable we can think of, sanitized in different ways, into our preprocess step, hoping that within that soup, the template developer can find the thing they want. The process step was added so that arrays like $classes and $attributes can be converted to strings after all the preprocessing on them was complete but before they hit the template, so that the template could simply print them, instead of having to do print implode(' ', $classes);, etc. Render API was added because a few years ago, it became clear that we were hitting the limits of the push model, and if we wanted to support things like node.tpl.php or node--article.tpl.php printing a single field somewhere other than with the rest of the fields, or moving some of its $links from the node area to a sidebar, then we couldn't do it by just pushing more and more variable combinations into the template, and that we needed to make a start towards a pull model, where instead of rendering all of the variables needed by a template in the preprocess step, we started exposing unrendered structured variables to templates, allowing the template to call render() on the parts it wants to.

However, we only implemented a partial pull model. Instead of taking advantage of PHP 5 objects to provide unified and consistent variables to templates that could simply be printed as print $content;, print $content->field_foo;, etc., and leverage PHP 5 to automatically render the variables at the point they're printed, we allowed the internal details of strings vs. data arrays vs. data objects vs. render arrays to leak into the templates and burden the themers. D'oh! Sorry. Yeah, compared to that, anything (tokens, twig, xslt) starts to look better. Though for all its flaws, once you manage to understand the D7 render/theme systems, you can create much more tailored themes and websites than you were able to in D6. So the partial pull model did succeed in solving the most pressing limitations from 3 years ago, just at the cost of a very high learning curve.

So here we are with the opportunity to use PHP 5 correctly to greatly simplify all of this by completing the pull model. To a large extent, that's what the Layout initiative is already working on for the largest structures of page/region/block, what Jacine's excellent write up hints at for everything else, and chx's attempt to unify these two levels within his write up of this issue. A token-based approach or a twig-based approach would force us to do so, since they inherently can only work with placeholders of PHP variables, not with PHP variables directly, but we can also do this with PHPTemplate with well constructed PHP 5 objects if we want to. And in my opinion, this is really the key: to map all of our currently sprawled out variables into consistent, clean data structures that remain unrendered until the template decides what to render, and for the syntax for accessing these consistent, clean, structured variables in templates to be front-end-developer friendly.

Doing this will solve many of the problems discussed in #1382350: [discussion] Theme/render system problems and in this issue, including greatly simplifying the #pre_render/preprocess/process pipeline (at least as far as themes need to interact with it), and making working with templates simpler and more consistent, while still having a lot of control and flexibility. The process of making these data structures consistent is also what allows for Jacine's proposal of unifying our current sprawl of templates into a much more manageable set.

So what's the catch? Well, for starters, it's a lot of work, but seeing the enthusiasm captured in this issue and related ones, I'm hopeful that the work is worthwhile and that many people will step up to help, especially once the high level architecture is figured out and we get into the weeds. One way to make the work easier is to just pick a template engine, at least to start with. Possibly, once we can make this work for one engine, we can also find a way to make it pluggable to others, but I would much rather start with one and optimize for it, and only after completing that, decide if it's also worth incorporating others. The fact that we already have PHPTemplate doesn't make it any easier (from a backend perspective) than Twig, because we'd need to prepare variables for it in a radically different way than we currently do, so whether PHPTemplate or Twig, it'll be a lot of work, so if we think we'll end up wanting Twig, now is the best time to make that choice.

My tentative vote is for Twig for 4 reasons:

• The generally favorable reception it's received here already.
• The fact that it's compiled makes me optimistic that we might be able to move most (possibly all?) of our current theme functions into templates, providing yet another level of consistency (no more function/template split). This optimism might prove to be incorrect, however. I haven't yet worked with Twig enough to understand its performance characteristics fully.
• Template inheritance could end up being a great way to reduce the number of templates and reduce the redundancy between variants of a template (node--article.twig could choose to override only what it needs to from node.twig instead of everything).
• Ability to sandbox the PHP variables/functions available in Twig can be great for security, especially for websites that want to allow template editing by not-fully-trusted users (whether via UI or access to a theme only git repository).

What's the other catch? Well, while all this work is happening, we also are simultaneously trying to add UI-configurable layouts and semantic HTML 5 markup to core, and both of those affect each other and the requirements and optimal architecture of this new proposed theme system. Ambitious? You bet. Doable? If enough people and the right mix of people pitch in, I think so.

This is one of the issues that makes me really excited about D8! While I'm all for using PHP as a templating language (as it started out as one, though I'd argue it hasn't been one for a while), the way Drupal currently does that is just plain terrible on a number of levels. As a developer I have seen the kind of preprocess code done by themers who Googled their problems and did a mix of copy/paste and rewrite without actually knowing PHP. And it was NOT pretty. Or secure. And I got bitten by it a year later.

This pull system could be one of the advantages of moving from render array to render object. Hell we could even go as far as lazy loading content on demand from the template. That would shift the responsibility for having the right content from the builder to theme layer.

For example instead of hide() some of the already fully loaded blocks in a template, the {{blocks.block_calltoaction}} will call the render method of that block, gather the data and outputs the rendered content.

@effulgentsia: really nice and constructive conclusion of this discussion!

@effulgentsia Excellent comment! Really sums up the main points of the conversation. I can really stand by that.

we also are simultaneously trying to add UI-configurable layouts and semantic HTML 5 markup to core, and both of those affect each other and the requirements and optimal architecture of this new proposed theme system. Ambitious? You bet. Doable? If enough people and the right mix of people pitch in, I think so.

You really brought an issue that has been also in my mind. I'm worrying we're not communicating enough between the other initiatives. I'm really looking forward for the sprints @ http://frontendunited.org/2012/program#sprint & http://groups.drupal.org/node/219224. I think those events will help us take the next necessary steps. I suggest that everybody interested can join one of those on the spot or IRC. This is excellent momentum we have here and wish it would continue to flourish.

EDIT I understand that this is still a META-discussion, which is why communication with other initiatives is not that relevant. But as it seems our approach is gaining a lot of support, it wouldn't hurt to start discussing.

@effulgentsia
great writeup.

Having read the WSSCI threads in its "wild" phase, they were hitting a very similar problem: the changes were too broad and too much touching other initiatives to be solved by planning and discussing.

So Larry went ahead and did a bold step to create a sandbox that had a hacky, while working implementation that kinda proved replacing very low-level Systems with Symfony components could work.
This again led to that famous weekend of the 15 brave low-level-Ninjas who discussed and tried and came up with the result that the route was absolutely goable and worth it and now the Symfony Train is rolling.

So getting ahead with this issue would require something similar, I think, even if two-parted:
1. Building a working proof-of-concept like implementation of Jacines proposed concept (of which I especially like the Container - Items - Formats - Components part.)
2. Building a working implementation of Twig as a template engine. This may or may be not, but should better build on 1. This could perfectly make use of ReneBs work, who has already quite of a bit of experience with this part.

Maybe a similar sprint like the abovementioned could be organized after this with the 10-15 most clever and committed theming-layer contributors of Drupal to try out if this concept really works out and is worth it.

At the point where this is now, I guess discussion will not get things much further as we need to see to believe.

I personally am a kind of themer who even tries to get around touching preprocess functions. And I consider myself more code-comfortable than most designers I know. So the people we want to get into drupal - people who write themes for drupal without having to become total drupal-ninjas - need something different.

The level we will touch a lot is the one of the actual templates. And for that I can say: give me a Twig implementation in drupal, I will hack away at it and give a view of a non-developer if the system feels any easier for me than PHPtemplate. The discussion has a very high techical level. I doubt a "normal" themer who is also half designer will ever bother with even preprocess functions.

#157 offers great perspective overview of issues/tasks discussed here. Thanks effulgentsia!

I second #162.

At the point where this is now, I guess discussion will not get things much further as we need to see to believe.

I think putting a sandbox together based on Jacines proposed concept is the ideal way to move this discussion forward at this point. Things have been talked out and discussed here a great deal & I think it is getting time to actually get down and look out how this can be implemented. And, I think the sandbox work based on Jacines propositions needs to be done before the Twig implementation happens (as stated in #162). Slapping Twig on to how the existing theme system operates won't fix anything in the long run. Sure, it offers some advantages which have been poinedt out several times this thread. But it does not resolve larger architectural problems with the theme layer currently.

Also I believe putting together a sprint could very well be a wise decision. But only if it is a mixture of core-architect/backend/ninja types and and themers. Though the actually implementation of this might be up to core dev's. The input of themers is highly valuable, to reiterate crell in #17 regarding front-end devs "Quite simply, in this discussion their voice counts more than that of PHP gurus...".

@eigentor : http://renebakx.nl/tfd/d7-twig-distro.zip Drupal 7 with a working twig 1.6 engine inside :)

@ReneB

If we used Symfony libraries to drive Drupal's interaction with Twig would we need Symfony's Templating component?
http://symfony.com/doc/current/components/templating.html
http://symfony.com/doc/current/book/templating.html

#165: nope, you don't need Symfony Templating Component.

*What Fabien said* :)

You need the documentation on http://twig.sensiolabs.org/doc/templates.html and the documentation I still need to write on the drupal specific extension. Most is documented in the TFD classes.

Grrrrreat! I agree 100%! Everything is said that can be said at this point. Now let's start working. :) @chx let us know if you need any help.

jups, feel free to fork my engine code if needed :)

Good to see a decision made. The system is going to compile themes. That means the input to the compile does not have to be locked up in the file system. My suggestion for really opening up theme access: http://drupal.org/node/1510032

no please no... one of the initiatives is to move logic and config from the database into code so it becomes easier to stage, version control etc. etc. Moving templates into the database is a step in the wrong direction. If you desire such power, write some custom code to mimic the way wordpress handles template editing trough the browser.

@effulgentsia:

[...] clean data structures that remain unrendered until the template decides what to render

I wholeheartedly agree with the #157, great summary. I'd like only to add that we should really be lazy-loading data as much as possible: there is no point in loading hundreds of comments if they won't ever be rendered. I think that by pushing the pull-model (pun intended :) to its farthest limits we should be able to do that.

As a themer, personally I would love to have this on D8

Controller - template.php

<?php
/**
 * Get all region properties
 *
 * - Content
 * - Styles
 * - Scripts
 * - Grids, IDs, Classes
 *
 */
function om_region_process_variables(&$vars) {
  global $theme_path;
  
  // Better processing of regions
  if (file_exists($theme_path . '/regions.php')) {
    include_once $theme_path . '/regions.php';

    $regions_system = $vars['page'];    
    $regions = om_regions_get_info();
    foreach ($regions as $region => $properties) {
     ...
/**
 * Process regions.php provided on theme
 *
 */
function om_regions_process_info (...) {
  ..
}
?>

Model - regions.php

<?php
function om_regions_get_info() {
  $regions = array();
  $regions['highlighted'] = array(
    'tag'    => 'section',
    'id'     => 'highlighted',
    'class'  => 'top-region',
    'inner'  => 1,
    'top'    => 1,
    'bottom' => 1,
    'grid'   => 18, 
  ); 
?>

View - page.tpl.php (cleaner)

<?php print $highlighted_region; ?>

Also, automatically reads css and js file names based on region names:
css/highlighted.css
js/highlighted.js

Rendered Output:

<section class="region region-highlighted top-region grid-18" id="highlighted">
  <div class="region-top">
    <div class="region-top-left"></div>
    <div class="region-top-right"></div>
  </div>
  <div class="region-inner" id="highlighted-inner">
    <div class="block ...


  </div>
  <div class="region-bottom">
    <div class="region-bottom-left"></div>
    <div class="region-bottom-right"></div>
  </div>
</section>

That's SCOTCH's layout system in a nutshell only a little more structured to address some developer issues. http://groups.drupal.org/scotch

Apologies if this comment is poorly informed as I am a little bit out of the loop in terms of all the changes being planned for D8.

The Twig approach sounds very interesting, but I'm curious how this will combine with the proposed "blocks everywhere" approach (detailed in http://krisandju.e-webindustries.com/blog/drupal-8-blocks-layouts-everyw...) - is this something that has been abandoned or lost steam? It seems a large amount of fundamental changes to be done all at once (though I appreciate the lengthy release cycles do allow for such things).

@JeebsUK renamed. click the link in my previous post.

I'm still fighting for people learning Drupal, and even though everyone else seems to think twig may be "better" than phptemplate, I'm not sure the "better" solution here is what's actually best for Drupal. It's still yet to be determined if twig is any more secure, and let me remind you that the only people who are complaining about phptemplate are the security team. Everyone else has been happy with this solution for years!

We need to keep the big picture in mind, and the big picture here isn't changing the syntax of template files. The big picture (and how most of this got started - at least for me) ie how *hard* it is to teach people Drupal. (plz actually read http://denver2012.drupal.org/program/sessions/token-templates-new-templa...)

I don't care if we decide to use tokens or not, but adding twig to the mix makes things in this area worse, not better. See #148

To echo Jen's point: The issues we have with the current theming system aren't manifestations of PHPtemplate, but rather an inconsistent API that is implemented via PHPtemplate. Whether Twig, Smarty, PHPTemplate, Token template, HAML, etc, we still need to address the cause of the pain here, which is the spaghetti of inconsistent and varied APIs in core.

I feel like we have been given a poorly written book, and we are debating which languages we should translate it into to make it better. The root of the problem lies elsewhere.

We have varying opinons. That's not a bad thing, but it brings challenges. With regards to this, I believe we have (at least) two primary audiences:

Group A. Those who learn to theme their Drupal site by pressing buttons
Group B. Those who learn to theme their Drupal site by writing code

It is going to be difficult to please both with a system (but not necessarily impossible). Arguably core (or some primary distribution of core) should probably ship with a system that caters well to Group A. Group A is why we have things like Field UI and Omega. We cannot deny this need. For many of us, these are valid, popular and useful tools.

Nor can we deny the needs of Group B, which is the group that I probably identify with more and (when push comes to shove) care more about the markup, and participate in more complicated and challenging front-end work.

In a more ideal world, the needs of Group A should not hamper the techniques of Group B, and conversely the technical expertise of Group B should not overwhelm the desired UX of Group A.

This is not an easy problem to solve, and this will take time.

I'd like to remark that I purposefully used the phrase "*learn* theme their Drupal site" for each of these groups. As Jen has described, the "learnability" of Drupal theming is in a poor state. Those of use who have been using Drupal for years can get around the quirks. I personally avoid render() like the plague and do everything in custom tpls and preprocessors -- but the reason I mention this is not to promote my own personal style, but just to acknowledge that experienced themers have their own ways of dealing with cruft. Newcomers don't have this luxury.

That we have begun to identify the problems of the current theming layer (#1382350: [discussion] Theme/render system problems) I would like us to work toward a statement of goals. This loosely exists under "Problem/Motivation" at the top, but there has been little discussion whether these are indeed valid, or necessary, or sufficient, and what advantages and disadvantages they create. We need a concept to work toward, and I think we've been too focused on schematics. The word "goals" has been mentioned three times in this thread. The word "syntax" has been used more than 40. Schematics, syntax, and implementation will come in time. But we need to know what we want to build first.

In terms of goals, I think http://jacine.net/post/19652705220/theme-system and http://jacine.github.com/drupal/ lays out the underlying issues very well. I agree that focusing only on syntax is extremely dangerous, and have my doubts about that. However, I think people might be doing that because a move to a system like Twig would essentially force these underlying issues to get addressed because we could no longer hack around stuff in various ways; we'd have to define new (and hopefully consistent) systems for overriding and exposing variables, etc.

I honestly don't have a horse in this race, but just to point out that a gazillion more people know (X)HTML(5) and CSS than do (or ever will) know PHP. This is why systems like Twig, Liquid, Smarty, etc. are popular, because they force business logic/display logic separation, and they remove the requirement for a HTML/CSS designer to understand how to write and deal with programming logic; they just put funny characters in their markup and they're good to go. While there certainly is evidence to support that some people have made the jump to module developer from themer thanks to being forced to learn PHP in order to make their sites look as intended, there's also evidence to support that designers stay away from Drupal entirely because of this requirement.

I have been working with people from a design background, and what I have seen more than once is that they will do what is needed to make it work.

For a designer, telling them
- use this piece of code : print $variable;
or
- use this piece of code : [[variable]]
or
- use this piece of code : {{variable|e}}

...is the same.

They will copy paste whatever is thrown at them, wathever they can find in the documentation. They don't see php or template system or wathever, they see special code that must be inserted in their html.

With this said, and with all the respect we should give to those who learned the current system (sometimes with costly mental switches) and are already using it, with all the existing documentation they read, I'm not sure changing the codes used to delimit variables is the most useful thing to make their life easier.

@neclimdul - Thanks for that - though on that group in one breath it says the group is deprecated in favour of this discussion, in the next there's posts about patching core with some of the changes, doesn't seem like the left hand is talking to the right hand there.

Regarding @jenlampton's comment, she makes an Excellent point. We all know that the barrier to entry to Drupal is pretty high, hence the move towards Symfony. If those amongst us trying to pull new people into the community and teach them are also continuously having to relearn the way we do fundamental parts of the system (or too many fundamental parts at once more importantly) then the barrier is only going to become harder to overcome. I'm not against change where change will be beneficial, but anything that is potentially going to slow down / halt further uptake of Drupal is something we need to be extremely careful about.

Moving cheese, or learning new things is a part of life itself. If a developer, themer or something, is unable to learn and move to ewer thing in life progress will come to a absolute stop. It's not that you stopped learning to bike after you got your first tri-cylce? Or did you?

Being afraid of change, is a not to underestimate fear in people, but please.. Do not let this fear become the death of a project. I've seen that happening with other opensource projects before. To many people where comfortable in the way things worked and opposed every change that was proposed. End of the story was, most developers left for projects where new things were accepted, and the project died because of ancient technology combined with a serious lack of developers.

@ReneB - the problem here isn't a fear of learning new things. To use Drupal, you must embrace change, because nothing in Drupal ever stands still. No matter what we do to address these issues, something is going to change, and I don't think that reducing peoples' comments to a "fear of change" is very helpful at all.

One of the problems is that we have is getting data to the templates and overriding markup. Drupal 7 makes those tasks extremely difficult and confusing, even for those of us who have been working with Drupal since 4.7.

Whether we print a variable or use {{var}} is really irrelevant; the point is that $var has to be set and in many cases, massaged, somewhere by some mechanism that in many cases needs to be written. So whether you write a preprocess function to meet the business requirement or you write a Twig filter, the end is the same - someone has to write PHP code somewhere.

What we need is a single unified mechanism for doing that - not a hodgepodge of functions and overrides.

Learning is indeed part of a Drupalites experience and that shouldn't change. I would echo the other comments though that it's not the learning it's the 'what to learn' part. Drupal has a set of coding standards for consistency, yet the theme layer and underlying belly doesn't have any uniformity to standardise - hence why each dev/themer/hack has their own way of doing things.

I do like the look of systems like Twig for Drupal, but again echoing what's been said, that isn't the problem. Painting over the cracks doesn't get rid of them, it just hides them behind a new shiny layer. The core of any new system added should be built on a solid foundation, and I fear that without the underlying architecture it'll just be business as usual and we'll all still be discussing this come D10.

It might be interesting to look at other systems, specifically Wordpress, how they handle templating and theming.
Drupal 7 is extremely granular in what you can do in the theme layer. From what I understand of the entire render arrays, they have been put into Drupal to get this even further.
But essentially, you do not have to use all thie granularity, but can just print your variable and say: well, that is as granular as it can get.
Do other systems even have this granularity? For one thing is sure: more choice, more confusion.

Learning by diving in

But how do people learn drupal theming?
I guess what you would do is take a core theme and have a look in there. Maybe copy things and hack your way to something that looks like you want it to look. That is essentially all a design driven person is interested in. I care about security in themes because my coworkers who are coders else bite my bottom, but for me personally: I don't care, I just want to look it the way my pretty design looks.

And take a Drupal 6 theme like garland or take bartik: you find awful lots of PHP logic. So this is the first impression you get. To be fair: Bartik is a big improvement over Garland in that it looks much cleaner.

But look at something like this:

<?php if ($main_menu): ?>
      <div id="main-menu" class="navigation">
        <?php print theme('links__system_main_menu', array(
          'links' => $main_menu,
          'attributes' => array(
            'id' => 'main-menu-links',
            'class' => array('links', 'clearfix'),
          ),
          'heading' => array(
            'text' => t('Main menu'),
            'level' => 'h2',
            'class' => array('element-invisible'),
          ),
        )); ?>
      </div> <!-- /#main-menu -->
    <?php endif; ?>

Looking from a perspective of a themer new to drupal, this should be

<div id="main-menu" class="navigation"> 
  <?php print $main_menu?>
</div>

Having the entire logic inside template.php or whatever file that the themer rather does not dare to touch is better from an initial impression point of view.

So there is this tradition that every person who uses Drupal learns PHP because else they do not get past basic stuff. Drupal themers are half coders.
It was never in a consistent way thought about: how to supply these people with as little logic as possible, at least in the template files?

This thread and this entire small initiative I understand as an attempt to provide this or at least bring it into focus. So this is very good.
It is also not surprising that we have this weird mixture of logic and presentation in templates, but PHPtemplate does not inherently require it.

Personas?

But maybe the different goals people ask for in this thread should be seperated to get it more clear, because they are totally different:

1. Security team wants unexperienced Themers to stop messing with unsanitized variables in templates
2. Experienced Drupal themers want all the granularity, but have a hard time to get their head round all the changes of D7. They want a consistent concept how all the data gets into the templates
3. Themers new to Drupal: we are not sure what they want, best would be to do some user-testing here and put someone that knows a bit about templating but does not know Drupal in front of a tutorial or give theme a core theme and task them with creating their theme.

If we are talking about Group 3, the perspective is completely different. It would be good to try to seperate it and maybe create personas for it.

Persona 1: Greg
Persona 2: John
Persona 3: Jeffrey
(guess people can put the surnames on that).

As long as the different Persona's needs are mixed, discussion is difficult. Because they have often conflicting needs. If we find a way that helps all 3: heureka. Traditionally Jeffrey is the loser, because he is not here to voice his concerns.

matched more exactly to the issue summary template http://drupal.org/node/1155816

#181 @webchick. Great points. Looking at Jacine's initial post, I've distilled these goals by restating some of her remarks. She delves into implementation, which is fine, but for the record I just want a cohesive list of goals on this issue.

1. Consolidate templates and theme functions ("Too Many Templates and Theme Functions")
2. Develop a consistent API ("Lack of Consistency")
3. Consolidate and simplify data structure ("Data Structure...is a nightmare")
4. Reduce levels of processing ("Too Many Levels of Processing")
5. Allow for a common component library ("Your Output is NOT Special")

I've added these under problem/motivation at the top of the node.

#187 @eigentor Your two snippets of code are prime examples of the two extremes we have. The first snippet represents the extreme where we have a somewhat obscure API function building out links. The second snippet represents the opposite extreme, where we believe we need to just give themers a $variable that they can dump wherever they want. Again, I don't want to focus to much on syntax yet, but I think something like http://pastebin.com/PwcEpBE0 for this particular example makes it easier for both parties: abstraction disappears without sacrificing underlying logic.

@JeebsUK - chx did not "deprecate" the Blocks Everywhere Initiative, he "deprecated" his own post (one of many in that group) in favor of this discussion.

@effulgentsia #157

Ability to sandbox the PHP variables/functions available in Twig can be great for security, especially for websites that want to allow template editing by not-fully-trusted users (whether via UI or access to a theme only git repository).

Looks like twig can introduce a new role, someone who is free to do anything he/she wanted with the layout, but unable to do something malicious to the database, etc. Err, dunno how would that look if put in to practice.

Edit: I was thinking of a role in the development team rather than a role in term of site user role. Also known as themer, bunch of irresponsible dudes with quick tongue and strong evasive manouver, some knowledge of html and css. And the whole VCS (git) process if put in to practice.

But, I think we do need to setup an upload destination where the files uploaded by this role is not executable, even if it has extension like .php. Err, dunno how to do that myself (_ _").

Another point is, if the layout initiative view of the layout plugins is similar to Panels layout plugins, that piece of code where the plugin defines the theme function, css, regions, are written in PHP. This should be a configuration file like xml or other, IMHO, and the theme function could be replaced with a reference to a Twig file.

Edit: looking at Panels layout plugin, having it as a simple configuration file seems possible, but some other type of plugins may need to contain executable php codes, so having plugins as configuration in general, might not be feasible. Or to have the plugin configuration of the assumed layout plugin specifically designed as non executable configuration file :(.

Totally sandboxed 0.0b.

Edit: sandboxed the theme layer along with the themer development group in it.

Edit pt.2: ugh, there's still that thing called javascript. #headdesk.

Edit pt.3:

So, the original idea is to have site user be able to create/upload their own custom theme, or add
modifications to the layout of their pages.

I tried to expand the idea a bit by creating a situation where two workshops join forces to build a single site, one handle the data, the other handle the layout (maniacal laugh... maniacal laugh...). Or when the site builder temporarily hire a remote designer (untrusted), or give quick access to a web consultant to do minor tweak/fix to their site layout/css.

The problem with that idea is javascript. I believe the only people allowed to upload javascript is the site user, or the site builder (whom they trust). Now for a 3rd party (untrusted) to upload javascript to the site, they can impersonate the active user, in the case of commerce or ubercart probably automate a purchase.

If 3rd party does that kind of automation I called them cracker, if the site user want to do that I called them nerds.

Of course there is the chance that the hired goons persuaded to go to the darkside and defaced the website, supposedly fixable using VCS mechanism.

The fix to this I believe is to have the layout plugin has a strict mode. In this mode the twig file is banned of all javascript, layout plugin user unable to upload javascript. Javascript can only be uploaded through the site builder side of the VCS, after going through their review, and used by the themer side of VCS using a sanboxed manner.

Outside of this strict mode is the site user and the site builder, they are able to add javascripts to the layout plugin, or embed them in the twig file.

And that is the sodding preliminary review.

Err, possible hickups: themer upload .htaccess, or upload php file with class in it and loaded by the autoloader, .htc file(?? dunno what this is, actually), adobe flash file, iframes, themer crashed IE6 using css (smirk), IE css expressions(??), html links that lead to a bookmarklet, data-uri that doesn't embed images.

Edit pt.4: which could be mitigated using git hooks and some sodding linters upon the themer sodding git push. Why would the themer has hardcoded url in the twig file anyway, that is sodding data, sodding nuke it! While data-uri in the css file, I think, is safe.

@glennpratt - Ah, reading again makes a lot more sense, thanks for that.

unable to do something malicious to the database

Nice. You can do that anyway with PHP, in a template file, with OO code.

In the current PHP template style, the Drupal "use anywhere" functions still expose too much. Themers need access a level below rendering, below twig, and they instantly get the "use anywhere" functions. A combined recursive templating and token system would still give them access because we would effectively have to give them everything.

This is a classic example i throw into the redesign process. HR. You want everyone to see dates of birth month and day so we can all celebrate everyone's birthday but no one is allowed to see the year because they can inflict ageism. This type of restriction is impossible if someone can print date('Y-m-d, $date_of_birth). Twig is no more secure if it can {{date_of_birth}}.

At the database level, you would add a database view excluding the year. The database can then restrict access to the original data and give you access only to the view that does not include year. Drupal does not have an equivalent. This is a layer below rendering that needs a solution to make rendering secure.

Role A can see Y-m-d. Role B can see m-d. Connect data to roles in a layer below rendering. The renderers can see only the data applicable to the role of the user logged in. The themer needs to see the format of all data available to all the roles who will visit the themed content but should not see the content.

The themer should see date as something like [Y-m-d, m-d] and allow for the two formats that might appear, depending on who is logged in. You can easily do this with tokens. How would you do it in in Twig?

@peterx #193.

To determine different display format for dates, based on role, based on the current site need, look more like a job for a site builder.

From a themer point of view, the psd file said show date here, looks like this, and that is what he/she did, job's done. Ignorance is bliss, and the themer move on to something else :p .

Edit: don't mind this comment, I was arguing for the sake or argument :P.

Lots of technical talk, but perhaps we need to rethink if the problem is actually technical. Can we sidestep creating a new theme system if we redesign to make theme development easier?

1) Is the problem vague variable definitions?
Yes. Developers create variables and arrays that can be themed. Some of the variable output is known in advanced, but most is unknown. This means designers/themers must spend time sifting through data sent to the theme, or asking a developer to create the variable output for them. Both waste a lot of time and make theme development aggravating, especially to newcomers.

2) Is the problem with the templating language or syntax?
No, I don't see this being the issue. The frustrating part is not knowing what it is inside the variable or array, and not knowing what is available to you, without having to rely on messy var_dumps and inspection tools. How to print out the variable is the easy part...

There are 2 ways I can think of to solve #1:

Standardize Drupal output until everyone is using the same set of variables and templates.

This seems pretty unlikely.

Create a GUI for preprocess functions that is part of core Drupal administration.

I propose creating a GUI for exposing preprocess functions stored in code, as well as creating and editing them via a web interface. This would allow themers to actually SEE what's behind the variables and logic they need for their template markup, and it would allow editing and creating new variables via a GUI as needed. This sort of graphical preprocess system would give more logic power to themers without exposing security holes, and free up devs from having to create custom preprocess functions.

To be clear, I'm not talking about implementing a WYSIWYG template system, drag n drop thing, or pluggable architecture. I am talking about creating a UI for preprocess theme logic so that themers can view, modify and create new variables for themselves...a configuration tool to assist with theme development.

For those who are going to be in Amsterdam for Frontend united and wanna know stuff about TWIG.. I will be doing some talking about it, and hopefully going to be there on friday as well :)

Actually, I think you're missing the point. The fact that we would need a tool to be able to navigate the various layers of "stuff" in the theme system speaks a lot to the amount of unneeded complexity of the current theme system.

EDIT: This is @zirafa.

Standardize Drupal output until everyone is using the same set of variables and templates. This seems pretty unlikely.

This is not at all unlikely. Jacine made a pretty compelling case for this, and #1484720: [Meta] Reduce the number of theme functions/templates is a meta issue for some of that, which people can spin off sub-issues as they feel inspired to start tackling that problem. That work can also proceed independently of Twig and Layouts initiative, though all 3 efforts will intersect at some point, but that's okay, it's what code thaw on a large project like Drupal always involves.

@cweagans
My point is that the complexity comes from things being hidden in the dark. There are lots of ways to shine a light to see, but I think it is essential to have theme components / variables exposed in some way. This can be solved with documentation too, but often the documentation is an after thought or incomplete. But anyway, removing complexity doesn't necessarily shine a light on anything or make things easier to understand (although it can certainly help). I just think the experience of theme development is what we need to improve and make enjoyable - that may or may not be related to the underlying technology.

@effulgentsia
Thanks for the link and I agree Jacine makes a pretty compelling case. I just wonder how it will work with contrib modules, and how it could be generic enough to support different Drupal development perspectives. To standardize means having a unified goal and making more decisions and more assumptions, but if we aren't even certain what Drupal IS (framework vs CMS vs xyz) then it will be difficult to reach consensus.

zirafa: It doesn't matter how much light you shine on it. If you have a tangled ball of intestines hanging out, shining light on it doesn't make it not a tangled ball of intestines. Exposing all sorts of internal implementation details in the name of flexibility rather than hinding them is how we got to the current mess, where you have to understand all the internal details to get anything done. Let's not do that again. :-)

Nice metaphore Crell :) And you are right to :) And not that Twig is the holy grail of solutions, it could help to map some of those intestines into some a bit more frontender friendly.

and for those still interessed in Twig in D7, yesterday evening I cleaned and synced both the d.o and github itterations of my Twig for Drupal sandbox project, making the D7 version master and moved D6 into rest.

As a themer new to Drupal (about 5 months in), I appreciate Drupal’s power but no, theming isn’t as easy as I’d like. I’d like to help make my job easier (and make Drupal better!), and my perspective as a beginner might be useful.

In the abstract, what I’m doing as a front-end dev is expressing a design system (with objects made of markup & style). To do that I want control of the markup, but I’m happy to not touch much except class attributes it if it’s lean and has decent semantics. With D7 though, I’m always wanting to override just about everything, and doing that has been a challenge.

I often find that I don’t know what template file to copy into my theme, and might need a second try to get it right. Once I have a template file, it’s an even bigger challenge to just “get this attribute and put it here” or “put some content field there”. I’d welcome a tokenized system like Liquid, or (presumably) Twig. However, what might be awesome as the “clicky/config” way to do this is a Liquid/Twig/whatever code editor within the admin ui, with syntax highlighting etc, but that automatically opened and wrote to the correct theme file for a component and gave me drag-and-droppable tokens for the chunks of data I have access to in that theming context. As a “teaching” tool, it would be important to show the full token syntax once tokens are dopped in. That would be lovely :)

We're all here because we love drupal and because we want to make a change.

Many designers / themers do not use drupal because they don't understand the templating system, and are therefore not here being part of this discussion.

I think it would be great to start a wider discussion not about Drupal, but about templating systems. This will help guide us. Maybe we could reach out to people such as http://www.alistapart.com/, http://www.smashingmagazine.com/ and http://www.netmagazine.com/ to open this discussion.

Twig isn't the solution to the core issue, but it could be very helpful for the second step.

As a designer I love what Expression Engine does for me. So please take a look at a solution that already pleases designers. It offers an accessible, clean, well documented template language. Expression Engine gives me the tools to display sanitized data in whatever way I can think of. It's a clean consistent layer for me as a designer.

For Drupal the first step would be to give the designer an array of sanitized, safe data that I can pull from. Then as the second step I need a way to define the markup how I need it, in a way that is closer to my domain of expertise than to that of a developer. This might be Twig, because it offers a more compact way of combining markup and data from Drupal. I don't want templates cluttered with unwanted classes, ids and tags. It would be nice if I could switch to defaults to get a basic structure if necessary, but otherwise I need tools to create markup the way I need it for my specific project.

In D8 we will have classed entities that will be display through blocks, and can be put in a layout.

In my mind, the only missing parts are:

1. easy ways to add classes and ids to wrap our data (blocks)
2. how html tags are used

My guess is that since all of the layout stuff can be done through the UI, that this would be the most sane place to be able to define our classes and wrappers.

So all is left is a set of theme functions or templates that will give us our data as html, but most of this templates/theme functions should be really simple: give me a list of stuff I will wrap each element with within a some tags; give me stuff, and I will turn it into a link, etc.

What I am trying to say is that, it seems that the WSCCI, Entity, and Layout initiatives are responsible for the data structure problems and should get us as far as showing well organize data in a page (without styling yet), I am failing to see where twig is relevant, unless we are not planning on using most of the functionality given to us by these other initiatives.

The UI might be the most easy place for beginners to define classes and wrappers. But the most effective way would be in a template file.

If I have a working HTML/CSS/JS dummy of my design it should be fairly easy to inject the dynamic bits of data from Drupal, currently it's way too complex and takes too much time.

OK, I'm probably going to hate myself for this, but...

- The biggest challenge is balancing site builders vs. themers.
- Twig files are not executable.
- Twig files can be sandboxed for security.
- Twig files are compiled to executable form no matter what, even if you hand-code.

So...

What if we recast much/most of the GUI configuration not as configuration, but as a wizard? It doesn't save variables. It saves a generated Twig template file. That could be stored in the DB, in CMI (more likely, etc. No matter where it lives we still need to pull the string from a data store, compile it, store the compiled version somewhere, and then run the compiled version. That is, the "storing code in the database" problem is less of an issue with Twig (or any similar library) because the actual code itself is always generated, no matter how the Twig file came to be in the first place. We are going to have to figure out a secure place to put compiled Twig files anyway.

That is... would a Contemplate-like approach actually let us have our cake and eat it too, without all of the myriad reasons why no sane person should ever, ever use Contemplate in Drupal 7 and earlier?

I need to go commit seppuku now for even suggesting it, but...

Theme developers should at least have some working knowledge of HTML/CSS/Javascript/the template system - Adding a bunch of UI on top of that is totally pointless and will add a lot of overhead.

If somebody wants to build a theme and doesn't know HTML/CSS/Javascript, our answer should not be "Oh, here, let me add a configuration wizard that will do it for you". It should be, "Go pick up a book and learn it! It's not that hard."

@cweagans: I believe the point is that "a bunch of UI on top" is already being planned to be added by the layout initiative, and a disconnect is happening between the goal of the initiative and what a lot of themers seem to want: they want to edit templates directly, and not have to go to the UI all the time.
There are advantages to both the UI and the templates so I agree with @Crell that the best of both worlds would be saving whatever is done on the UI as a twig template. I suggested that to @EclipseGC a couple of weeks ago, but I don't have enough knowledge of the internals of panels (what the layout initiative is inspired on) to think through the problems of that approach.

I think more than 50% of the drupal users are in the gray area of being a developer and themer at the same time.

If you're imposing Twig because of non PHP devs, that can be just an option, otherwise why would I need it if do eveything by myself?

One point is to make Drupal custom theme market bloom, I secretly has that in mind.

PHPTemplate is the developer's way to create templates and Twig would be the designer's approach to a template and it's also the safe way. There is always a gray area, but it's not the developer who should design the template, so please give us the tools that we need to do our job.

@dozymoe, please stop editing your comment. Just post new comments.

Perhaps I wasn't clear. Panels/Layout/Blocks is not really what I was looking at. That's perhaps the most easily solved part; A layout plugin defined in code only, and you drop blocks into it in various places in config only. Done.

Views and Fields are the really tricky places, and the things that generate the most markup-hate right now. I'm basically suggesting that Views style plugins, Fields/Fences module, etc. turn from configuration into template wizards that edit a template file directly rather than throwing variables into it. The end result is still a Twig template.

I know, I'm going to hell for this, but still, it's the only way I can think of to not run into the "configurable markup makes for unusable templates" problem.

@Crell Okay, so I’m pretty new to Drupal, but I don’t understand the rationale for building markup with a wizard. Even the select menus in Fences rub me the wrong way. It seems clear to me that the target audience for theming should be 1) front-end devs and 2) admins that have intermediate html/css. Nobody else should be doing anything like building templates by whatever means. Because even now, good markup is crafted by humans (disclaimer: I wholeheartedly support the Goals here, especially #5).

As someone who meets these criteria, the tricky part for me is _finding_ where to write my markup and _getting hold of_ the chunks of stuff to actually mark up. Give me a code editor in admin (with syntax highlighting for whatever template language gets chosen), that opens the template I need (less hunting through the file system), lets me choose a level of specificity and will write the correct template into sites/all/themes/mytheme/templates/. Then this built-in code editor shows me a bunch of little tokens I can use in a palette (like Views field rewrite tokens on steroids) and I can drop them into my markup. Once dropped in, tokens should expand to the full syntax for whatever template language, so I can learn to write it by hand.

Am I crazy to think this would be the bees’ knees for themers?

@Crell On re-reading what you’ve said, perhaps we agree more than I thought - perhaps we simply differ on the UI, and perhaps the audience we’re talking about?

@Crell: what about moving code-driven markup to a postprocess phase? Themers could edit the default markup to their will, and code might inject markup/attributes afterwards through a DOM parser. Obviously injected code should be optional, similarly to what Views allows to do now for, say, the default markup of the fields in the row plugin settings. Variables might be used to identify the places where inject markup/attributes.

@cweagans #214

/me shrugs. It is a single comment meant for people that later came along this thread, having it in chunks would made it unreadable :P. I think now it is completed.

Two points:

• I see the possibility of the core theme functions themselves turned into plugins, probably has performance issue though.
• Just the pull system is what needed, I think, Twig or phpTemplate doesn't really matter, though it has a bit of a bonus (sandboxing, disabling troublesome html tags). I got the feeling that Drupal custom themes risen from the ashes of modules, was built inside out, probably dependent on what the infrastructure currently in used, either it's commerce, ubercart, using views, not using views. I also think user interface doesn't know what variables to throw at the theme functions (and the custom made ones), there is no way for them to communicate.

Err, no rush, one idea can be implemented in multitude of ways, I don't really has preference on which.

ry5n: I've been saying for well over a year that we cannot serve the button-pushing themer audience and the file-editing themer audience equally well. We have to pick on to make our primary target audience. We've been trying to serve both equally for a long time, and the result is that we serve both poorly. Drupalers of course hate to be told that they cannot have their cake and eat it too, so a lot of people haven't been listening to me. ;-) I was suggesting one possible approach to maybe be able to see our cake and eat it too. I don't mean direct in-browser editing of template files. That causes all sorts of issues for version control if noe not perfectly. I was thinking more of a wizard that dumps a template file into config rather than a proprietary array. If you want to hand edit the template file, just damned well edit a file on disk where git can handle it.

plach: DOM-level manipulation of the template file would be an order of magnitude or three slower than just print statements, and would break very easily if anyone causes a parse error. I would be very reluctant to do that at runtime.

@Crell: I thought that compiled templates might alleviate the performance penalty... Surely, as you always say, a trade-off is needed somewhere :)

" I don't mean direct in-browser editing of template files. That causes all sorts of issues for version control if noe not perfectly."

Just playing devil's advocate, but in the age of CMI, why exactly would a UI for editing template files be so verboten? It could conceivably be just configuration data, no?

The CMSes I consistently hear designers saying they like working with are WordPress and ExpressionEngine, and both (afaik) allow in-browser editing of markup. That doesn't necessarily mean we should do it too, but I'm curious why this is considered any worse/more dangerous than any other data we're going to be storing to the config system: Views argument handling using PHP, for example.

Because inevitably, people will edit their crap on prod, break ALL the things, and then whine about how CMI is so broken and doesn't support every use case under the sun. There's no reason to ever expose a user of a Drupal site to that kind of functionality. It's the same kind of thing as php.module - if you want to write code, then write code. Don't expect that you'll be able to just shove things into a text field on your site somewhere.

IMO, code editing should happen in a code editor, not in a browser.

IMO, code editing should happen in a code editor, not in a browser.

@cweagans - I agree, it would be terrible to maintain and troubleshoot if we have people storing their revisions on both file and database for theming purposes.

@Crell @webchick @cweagans As a new themer (not a new front-end dev), I admit I have limited knowledge so far of the theming system. I guess I’m participating to provide that perspective (I sincerely hope it’s useful).

I also understand that I’m not acquainted with all the ways people use Drupal. But if anyone can point me to research on the “button-pushing themer” audience, I’d be interested in reading up.

As far as a text editor in admin, my only interest in that over dedicated apps would be so the editor could talk directly with the theme layer to help me figure out what dynamic stuff is available in a template and how to print it. Note that I wasn’t imagining that this would be stored in the database (just a text editor that “knows about” the template layer; it would still write a file like any other). I even spent some time mocking up what it might look like (attached). After doing so I suspect it’s _actually a bad idea_; I’m not even sure it’s possible.

@webchick The last time I've used Expression Engine having the template in the UI just meant that the code is stored in the database and you can edit it within a textarea element. You need plugins like "It's all text" for Firefox, so that you can edit the code with your code editor and to get syntax highlighting. Expression Engine saves versions of the template so you can roll back, but except from a security or code generating point of view, I don't see any advantages for that kind of solution. Expression Engine also doesn't use file templates and templates in the database at the same time, file templates get pulled into the database, so the database template always has the active template code.

From a theming standpoint Expression Engine provides a very solid theming layer. Each module supplies the themer with a set of variables/tags that he can use. Here's an example for Expression Engine and its comment module. Here's their user guide.

Following on from @ry5n, regardless of the underlying architecture or how it's editable - the main issue seems to be that non-ninja users have no idea what is available to output or how to output. I'm a believer that you shouldn't have to have a degree in computer science to work with Drupal - if adoption is key then you have to encourage ALL types of user - if increasing the number of contributors is key - then you have to provide the tools to make this happen

Surely there could be a simple solution of making that information available, maybe as part of the content type settings?!? Even non devs are able to create a new content type - with it being UI based, and this is good. The missing link as far as I can see is that there is a level of mysticism between the content type creation and what is output. A vertical tab or accordion with theming info could:

1) be a UX win
2) provide the 'safe' output
3) take into account any particular theming engine

Then have the option to turn the info off - probably with a warning about dev/prod so it's not there if not needed.

Of course, this is probably straight forward for field based items, but stretching into the black art of the entity system and other non-field based bits it becomes a bit grey. The ultimate aim is for a unified approach... could this sort of solution cover fields and non-fields?

Good documentation would take us a long way too.

I for one, do not want my templates in the database! That is a DTAP nightmare in quadruple. One of the initiatives is to move out the 'magic' from database into a config format that can be written to disc and then moved into a version system like GIT.

That is one of the main reasons that the Features module was born, so re-introducing code back into the database just because other projects do the same, doesn't automatically mean it is a good thing.

@webchick, editing templates in the browser, is even in these modern times not a good idea and agree with @cweagans on this, it's waiting for disaster to strike and for the above mentioned DTAP reasons.

@dreamleaf I don't want to sound rude or anything. But the names of the renderable fields are available in the node editing screen. It are the names of the field you are creating.

node.field_fieldname is the field called fieldname in your node.

Is there any good reason to have templates stored in the database? I don't see much good in this.

@ReneB not rude at all :) Probably didn't explain what was in my head that well.
I actually meant that explicitly providing this information avoids any guesswork and makes it less of a wtf. My point being that unless you 'know' that, you don't know that. In light of the main topic of conversation (ie. Twig) it would be beneficial to explicitly tell people what is going to be output and what syntax they should be using in tpl's etc

My point is less about fieldnames and more about making the system easier to use.

I'm sure this nugget of of info is in the docs somewhere, but I just looked and can't find it. Something as basic (in theming needs) as this shouldn't be a question that ever gets asked! I would guess that 99% of the people involved in this thread know more than the basics, but they are the 1% in terms of users - and although we all chime in with our selfish wants (rightly so), introducing something to core should be in the best interests of the majority.

True, it is not in the documentation, because by default drupal assumes you want to render the entire node, instead of rendering it field by field.

If core-provided default templates are made nicer, we'll be much closer to the expected goal. I guess/hope the html5 inititive will bring us there.

As for themer ability to discover available options and stuff (s)he can overrides, it would help to have a template generator. For example a UI that would allow to write on disk :

- generic field template to theme all fields / all node types
- field template for a particular field type / node type
- generic block template / template for a particular block
- view / view row / view field

The UI would allow the designer to choose which part he wants to override, and the code generator would write the properly named field_fielname.tpl.php file in the curent theme template folder, or provide it as a downloadable file if there are security issues (+instructions on where to save the file like "Put the following code in a file named "field-myfield.tpl.php inside yourtheme/templates directory).

The file would contain great documentation inside php comments, with all the available variables and functions needed by the themer. The documentation would be tailored to the specific field/node/row wathever item that is being themed. It would also contain the default markup provided by core/module. And this markup would be already good, since it is part of the plan to make it great.

Okay, so block has a URI.

Supposed I want to render content by the client side, using javascript, then I would fetch the twig file(s?) and the block's data from the provided URI, no?

Then where does template_preprocess_HOOK() fit in on this workflow? I imagine this function is the location where data being molded by site builder. Does each version of molded data need their own URI?

Err, I believe someone mentioned this (client side rendering) in #1441320: Evaluate Twig template engine for Drupal core (as an alternative for PHPTemplate and possibly even theme functions), but looking back I can't recall which comment.

Or hook_block_info() is the template_preprocess_HOOK() o_O.

I don't know why, but "view mode" comes to mind.

So....... catching up here it seems like have a UI to create twig files seems like an interesting idea. Please bear with me as I haven't caught up on everything and don't know if that idea has been put down or not.

Outside context: Sitefinity

Another CMS that provides a UI similar to the one we're buildling is the Sitefinity CMS. But the challenge with Sitefinity is creating layouts and modules/widgets that you can use in layouts (see: http://www.youtube.com/watch?v=Go7GiYIyPxg)

The wizard-makes-a-twig idea would be awesome IF:

• you could create a .twig file and give it to the wizard so it can use it as a default layout
• be able to "lock" a twig file so that the layout manager won't alter it.
• be able to use the twig file as a starting point so that other templates could be created using the orginal file as a default.

That way a .twig file would be the irreduceable unit of work for markup makers. It's sole purpose would be to apply markup to raw data structures

Sounds like that would be pretty awesome if we could pull it off.

dozymoe: hook_block_info() will not exist in Drupal 8, at least not in any form you'd be able to recognize from Drupal 7. The Layout Initiative is gutting it. :-)

I temporary re-opened this issue so I can add a quick comment.

I just wanted to let everyone know that I'm in support of rethinking the theme system. I'm also comfortable that the team will come up with a good solution.

I just wanted to add two thoughts.

1. We need to agree on our target audience for this new theme system. Do we want to cater to the "5% themers" (i.e. the most advanced themers, themers that are also quasi-programmers) or do we want to attract many more themers (i.e. less advanced themers that don't want to learn how how to program). My take is that we want to grow the number of themers in the Drupal community, and that we need to come up with a solution that is significantly easier to learn and use than what we have today.
2. I want to encourage all of us to look at competing solutions that are known to have a thriving community with themers/designers. Projects that come to mind are WordPress and ExpressionEngine. I think it is important that we understand what they do as we know "it works". We don't necessarily have to re-invent the wheel. If we want to attract Wordpress and ExpressionEngine developers to Drupal, we should build something comparable.

Glad to see you guys are organizing a sprint. I can't wait to learn more about the results of that.

Thanks everyone, see you in implementation issues.

Before closing this again, it would be useful to have all implementation issues linked to from the issue summary. I just created a "remaining tasks" section in the issue summary and we can link to issues from there.

I've linked to one so far, #1539004: Remove random PHP function calls and logic (other than render() and hide()) from all core tpl.php files (which I just created). Based on the discussion in this issue, that seems like it would be a reasonable one to start with; it's the kind of thing that would be a prerequisite for a lot of the ideas discussed here, but also the kind of change most people would agree with on its own even if we did nothing else.

Adding #1537050: [meta] Should we keep / improve multiple theme engine functionality? as well.

Add a "remaining tasks" section and begin linking to implementation issues

As someone who works at a company that does a lot of Expression Engine the most important thing to achieve is to incorporate the same workflow that EE allows. That is

UX/UI wireframes -> Frontend Developer's Raw HTML/CSS/JS implementation -> Backend Developers's Theme

Instead of Drupal's workflow of:

UX/UI wireframes -> Backend Developer's markup -> Frontend Developer's CSS -> Backend Developer's corrections to fit with the site.

Ideally, I would be able to put a Frontend developer onto a Drupal 8 project that knows absolutely nothing about Drupal and be successful in completing their part of the project. That's what we currently have with EE.

Adding link to #1537050

Updated issue summary.

At the company I currently work at we do

UX/UI wireframes -> Setup backend, create screens with real data and as less markup as possible -> frontend implement design with a little help from backend if needed (read very little help is needed)

Feels like a very natural process to me, most frontenders I know are pretty capable of implementing HTML within a manageable set of templates (with logical names instead of magic names ;))

I think this workflow is in some respects outside of the goals of this issue. The theme system itself doesn't work directly with placing HTML elements(blocks) and placing html for elements that don't exist in the CMS is a layout problem that seems mutually exclusive to the drag and drop in place site builder model in some respects. Placing and theming a view before your build it would lead to some serious development problems.

I don't think the back and forth sort of model ReneB mentions was one of the identified pain points, is fairly usable, and works for modules providing blocks of content. If that workflow is interesting to you though I'd get involved in the Blocks Everywhere/SCOTCH initiative and look over its group on groups.drupal.org where these sort of design decisions are being discussed.

I just tried out Twig and I am extremely impressed.

Seems to me that it provides the perfect balance - It's easy to use, it's powerful and it's extremely flexible.

Totally up for this becoming the default theme engine in drupal 8

The token based idea is good that it tidies up the tpl files, but beyond that i don't really see the point, Twig will do that too and a ton of other great stuff.

Hi there,

I've been involved in the theme sprint at Frontend United, and I've followed up with a blogpost. Basically, it's about the UI for markup. I call it "Markup styles", and in my concept, they're configurable in the admin interface, but furthermore totally disconnected from any particular type of content, object, element or...

When you create such a piece of content (list, node, block,...), you can simply apply the markup style of your choosing to it (and it should of course have defaults).
You should also be able to export/import your styles, so you define them in one project, and just bring them along in any project you take on later...

More information and a few mockups can be found here: On markup styles

I'm really excited about Twig, but also think it would be good to consider an additional UI-based solution to make it super-easy. This was something we discussed at Frontend United – that code-based solutions should also have UI equivalents. Thijs' initial ideas are very interesting.

For a discussion of adding theme to the UI you might want to check out this thread in the Theme Development group on g.d.o. Although the original topic is on a specific implementation it's sort of turning into a general discussion on tpl vs ui theming. And I think this issue was defacto closed in #238.

Updated issue summary.

linked to more, relevant issues

@David_Rothstein, I added some more of the cleanup issues to the summary, but the conversion to twig itself will need to start in one big patch that includes the addition of the template engine, as well as at least one working template as a proof-of-concept. Though that will no-doubt need to be broken into many smaller tasks, I don't think we should create issues for those tasks, since that big patch will probably live on this issue :)

fix typo

move issue to related

added another related issue

add things that can not be issues yet

move todos

add template suggestions todo

added next steps for core patches

reordering issues

wrapper

added required patch

Another note re: Twig's ability to render in both the server and the browser environment: as one of the maintainers of the Backbone module and major advocates of its usefulness in the Drupal stack, the decision to use Twig will make the architecture of that module much more elegant and powerful.

Those JS projects only implement a subset of the Twig language. Would it be possible for core templates to make an effort to stay within one of the JS subsets of Twig?

I don't think it would need to be a hard requirement, but the more compatible the core templates are, the easier front end devs can build server/client apps.

(Ps. see https://github.com/fadrizul/twigjs for another project implementing Twig in JS, though focused on the NodeJS/Express platform.)

The problem with that project is, that it's implementing twig in a server side javascript environment. Rendering twig client side is a whole different ballgame. Not that it is not interesting or should be ignored. But rendering templates client-side brings other problems, like indexing in search engines like google that will fail because they don't parse javascript etc. Not a issues for applications or backends, but try explaining to a customer that their new site won't show up in google, cause google only sees the template engine bootstrap code and not the rendered end result.

@ReneB - time to update your wordview. Googlebot does execute javascript. Nobody knows the details, but they have publically stated this. See http://www.webmasterworld.com/google/4382245.htm

Since the questions around client-side rendering are probably best continued elsewhere, I've started a separate conversation thread around potential architecture and planning considerations for client/server compatible Twig templates here: http://groups.drupal.org/node/232593, in the Backbone g.d.o group.

I ended up reviewing the code in http://drupal.org/sandbox/chx/1541306 in preparation for a talk I'm giving at DrupalCamp Vancouver next week.

I found this perplexing:

Drupal 7 - modules/system/region.tpl.php

<?php if ($content): ?>
  <div class="<?php print $classes; ?>">
    <?php print $content; ?>
  </div>
<?php endif; ?>

Drupal 8 - themes/stark/region.twig

{% if content %}
  <div {{ attributes }}>

    {% block contents %}
      {{ contents }}
    {% endblock %} 
    
  </div>
{% endif %}

Sometimes we do {% thingy %} and other times we do {{ thingy }}. Chx said this is the difference between a "command" and a "variable." Are we assuming themers are going to somehow know this difference though..?

@webchick, there are tutorials for Twig but not in a Drupal context. There is a need for some pages aimed at Drupal theme developers converting to Twig so they can see examples using something they can work on. We can then use before/after examples they will relate to.

I suggest tutorials for Bartik, Zen, AT subtheme, and Omega subtheme would cover 97.38% of all new development work. Perhaps create a starter page then people can fill in the details.

Front end devs will need to learn twig, the same way they need to learn how drupal render works now. If there's a drupal specific application of twig commands and variables we should note that in the docs or the comments in a key tpl, maybe node.

I don't think we need to worry about confusing front end devs -- it can come off as a little patronizing sometimes. Most of us do plenty of js work, and CSS by itself can be dauntingly complex. Instead, let's ask do we as drupal developers have a clear understanding of where and more importantly why each is being used.

Updated issue summary.

Linked to Component Library issue #1804488

DesignDolphin,

Personally, I'm thrilled we're using Twig. Not that I know it, because I don't yet - but the energy and enthusiasm behind getting this implemented and the added bonus of having the Twig community working on it in future bodes nothing but good for Drupal, IMHO. We will be leveraging THEIR work too, not just the Drupal Community's, which is a clear win. I don't mind learning it, if it helps us resolve the mess we're currently in.

Nothing about Drupal is simple anymore, but if we can get this together I think we'll be making huge progress towards a sustainable theme layer.

Yay chx! Yay Jen, jacine, greggles, c4rl, eclipsegc, et al!

Thank you all so much for your hard work. You are my heroes.
-Kelly

re: Dolphin in #262:

See theme system cleanup issues.

@DesignDolphin,

...I just started learning Drupal 7

Since you have a lot of catching up to do, and you voiced concern about making Drupal design easier, I referred you to a list of issues which seek to clean up the theme system.

Communication is hard. It's difficult to hear constructive critisism in the context-free nature of these comments and not read into all the possible frustration coming from both sides of this issue. Let's take a step back and actually talk about this instead of verbally combating. To me I don't really see a lot of disagreement. Here are the points.

1. The focus on Twig (as a backend system for implementing the themeing layer) seems to be misplaced.

But what DesignDolphin doesn't realize is that in order to make the Frontender's developer experience better, we need to simplify and delegate development of this component to a more dedicated team.

2. We need to make it easier for Front end developers to inspect and implement templates.

And we are. We're implementing a completely new way of thinking about themes with the layout initiative. Also we're reducing the markup in specific templates. And we've already spent effort re-factoring and separating css stylesheets.

Using Twig gives us a great opportunity to touch on every template and reduce the markup that is output. Using Twig's syntax and functions gives us a non-Drupal-specific language to implement these templates. Over time, that implementation may be enormously valuable as we won't need to hire Drupal specialists to get the job done, while at the same time allow Drupal veterans the ability to implement themes in other systems.

And let's not discount the value of having a dedicated Twig development team working with and for us. We'll have a team focused on making Twig stable, robust, and better.

3. Twig seems to be a risk when we could instead focus on improving our current system

As chx mentioned, we tried that. And yes, we could continue fighting that fight. But what would the cost be? What do we lose if we use twig?

• Drupal specialists will have to learn a new system
• Drupal 7 themes that want to transition to Drupal 8 will likely need to re-implement their templates for Drupal 8
• Since Twig is new, we might not get it perfect the first time.

Sure those are all risks but I think Twig is worth it. Twig will give us a mechanism that we can use to improve the DX experience for themers, for site builders, and for backend developers.

Every front end developer that I talk to is excited about this opportunity. That's mostly because of the frustrations they current experience when dealing with our current system.

@DesignDolphin: if you have the time, please engage in thetheme system cleanup issue. And help clean up these templates.

So, whatever the new theme system, to stay on topic, please keep that in mind. Because all this work going into these complicated theme functions I'm doing everything to work around them. Seems like we could both save ourselves some time here.

Precisely. It sounds like you're running into the wall of pain that is Drupal 7 theming. That's the result of us trying to implement Ultimate Flexibility(tm) at all levels ourselves, in pure PHP, guessing at what "themers" (whoever they are) would be able to use. And doing it about 4 times, layering on top of the old system that was there. What you see in Drupal 7 is the product of 6 years of trying to "fix" the theme system. We apologize for the inconvenience.

A ton of work is going into Drupal 8 to make that experience suck less, not by trying to layer more abstraction on top of what is already there (that hasn't worked out so well), but by removing our over-abstraction and switching to a library that is already field-tested, already debugged, and already shown to be pretty damned good and popular, even if it's not perfect. With that in place, we can, hopefully, rip out the layers of unnecessary abstraction that we no longer need and make a less confusing, less self-recursive, more themer friendly system. Twig is only one part of a larger effort to resolve the exact problems that you describe, and that are felt by, well, anyone who's been trying to theme Drupal 7.

(Using Twig also means that it's harder for us to change the template syntax yet again every version, since we'll be sharing a library with many other projects and cannot just willy nilly change things. That should in the future lead to more stability and consistency, not less. We just have some bandaids to rip before we get there. :-) )

@DesignDolphin

As a designer and front-end dev, I share a lot of your concerns. I understand; I wish theming Drupal 7 was as easy for me as hand-coding markup. However, as chx points out, Twig is well-engineered, and, as far as I can tell, that extends to its syntax and features. As a front-end dev, I for one am excited about it. From what I’ve seen (FYI I have yet to build anything in Twig), I expect it will make my work easier.

No, Twig isn’t perfect. But also, no, we can’t change the syntax. The community adopted Twig and Twig is Twig. Please be aware there are reasons for the syntax being what it is (e.g. the double braces have meaning – they are shorthand for printing values to the page). I suggest you take a look at the Twig for Template Designers page inthe Twig docs for a primer. I find it both concise and readable, especially considering everything it’s actually doing behind the scenes.

As far as the Drupal theme system itself and how to make it easier for front-end folks, I’m sure there are still many things to improve with the new Twig setup. And as a community, we get to do that! That’s pretty cool, IMO.

@DesignDolphin, you're welcome to write your own theme engine (or use any of the existing ones) at any time. Nobody will stop you.

The bottom line is this: there's always going to be some complexity around writing templates for a CMS. This is true for any CMS. For the past 5 years, we have worked with our own template system that is unique to Drupal. Over time, PHPtemplate has evolved into a convoluted mess.

As a community (with nearly unanimous buy-in from frontend developers), we have made the decision to move away from our own solution to Twig, which is in use in many places (or at least, more places than PHPtemplate). Since Twig is so compartmentalized, it can be dropped into nearly any PHP project. There's also a Ruby version called Liquid (as noted in the issue summary).

So maybe Twig is not the perfect solution. But it's damned close, and at the very least, it's 1000000x better than PHPtemplate just because it can be used outside of Drupal and because our front end development community has so strongly supported it's adoption.

Dolphin: The hostility you're reading in various people's posts is because you're arriving about a year late to an ongoing conversation.

Imagine if all the presidents of all countries in the world had finally managed to get a summit meeting where they all agreed to finally fix global warming by moving the entire world over to wind power, and had even managed to get the oil companies to not block it. Then you show up and say "wait, what, wind? This oil thing I'm finally using sucks, we should fix it. Wait, why three blades on the wind turbines, four looks better to me, I guess. Pfft, fine, whatever, don't listen to me, I'll probably just go become a hermit because oil sucks so much."

You'd probably get a lot of hostility there, too. :-) On a smaller scale that's what you're running into here. If we want D8 theming to not suck as badly as D7 theming, we cannot be distracted by revisiting decisions made half a year ago or more.

If switching from the fugly that is D7 PHPTemplate to Twig makes you want to leave Drupal... well, not to put too fine a point on it, but the cost/benefit analysis says we switch to Twig. I know that sounds harsh, but this is the first time we've had this broad a consensus on anything front end since we adopted jQuery. One late pebble cannot stop an avalanche.

That said, if you want to see the theme system move forward there are still plenty of tasks left to convert templates to Twig, clean up markup, etc. Who knows, working with Twig a bit to help push D8 forward you may find that you like its syntax after all. :-)

Some clean-up of issue summary

Move the *.tpl.php render() clean-up task to a related issue since it isn't explicitly related to Twig.

Reformatting

Consolidate documentation to Twig sandbox http://drupal.org/sandbox/pixelmord/1750250

Add link to Drupalcon Denver core conversation

Add link to core queue issues

1. Twig is the new default theme engine for D8.
2. All theme templates have been converted.
3. Most theme functions are converted into templates; the remaining ones are actively worked on.

Thanks everyone for your passionate input.