Technical discussion on Drupal themeing.

This is the follow up to Open discussion on Drupal's themeing capabilities and templating engines., and is more technical in nature than the previous article. This article intends to go into depth about the nature of the current system, and what possible improvements could be made in the future.

It will attempt to refer to as much actual code as possible, so much so that I am starting to wonder wether it would not have been faster for me to just write the code and get it done with, however.. I also feel that it is important that contributing designers have a reference point for the nature of these changes, hence this discussion.

I am only indexing the current Drupal 4.4.0 contributed themes, as they are the only ones which would be able to support all the drupal features. Unsupported themes that should be indexed additionally should please be mentioned in the following discussion.

Proof:

Drupal themes currently reimpliment a lot of functionality that could be centralized, and allow for themes to be more easily constructed.

Experiment:

Research all of the currently available themes, and index the features they support which are either inherent in Drupal, or configurable in the theme setting page. Also take note of all the theme_ hooks they override.
Method:

I synched with the current status of the DRUPAL-4.4 contributions cvs branch, and copied all current contrib themes into my current stable testing directory. I was left with the following themes after modifying or removing the incompatible ones : adc, chameleon, goofy, interlaced, marvin_2k, phptemplate, slurpee, sunflower, unconed, xtemplate

GWorks is named xtemplate.theme, and is such incompatible with the default XTemplate theme, and as such was removed. Marvin_2k is the predecessor to the PHPTemplate theme, and uses the same template library , but doesnt perform a conditional include check for the existence of the library, this was added. Since Example is not a complete theme, I have also not included it in this discussion.

I then enabled all of the themes, and went through each of them to verify which features each of them supported, and then used the resulting information to compile a feature matrix.

Default functionality is defined as settings that are configured / required by the drupal core. The theme supports the feature when it either displays, or allows the site owner to display the contents of these fields.

1. Site name
2. Site slogan
3. Primary navigation
4. User navigation block
5. Mission
6. Footer
7. Sidebars (left, right or both)
8. Breadcrumb Trails

Additional functionality is defined as any features that the theme has above and beyond these.

Results :

The complete feature matrix hopefully gives us an overview of just exactly what all the current themes can do. Where menu support is noted as partially , the stylesheet provided with the theme was causing the indent on menu items to push halfway across the page when in the admin menus. Not a big fix needed, but it still wasnt working out of the box. PHPTemplate and Marvin_2k are directly related and as such share most of the same supported features.

The great majority of contributed themes do not seem to have their own settings page, and as such do not allow much modification from with Drupal itself without modifying the theme. A great improvement from previous versions is that all the themes test support both sidebars without any hassle. A lot of them , most specifically xtemplate (!) do not support the primary navigation links as defined by the modules, or the primary_links variable in the configuration file.

It is telling to notice that the mostly it is the template engine supporting themes that define settings pages for switching templates, and chameleon for switching stylesheets.

Visual overview

The following Current theme system overview was designed taking into account that there are 4 main components of the drupal theme system.

Internal theme API

When a new module is developed, all of the markup generating functions are (should be?) seperated into their own theme_hook($parameter) functions, and called using the theme("hook",$paramater) function.

Public theme API

When a page is displayed, Drupal attempts to load the specified theme file, which contains functions in the format of themename_hook. Internally in the theme() function, it checks for the existence of a themename_hook() function before defaulting to the module specified theme_ function

Markup / Skeleton layer

The output generated by the themeing system in the form of (hopefully valid) html. In certain themes (phptemplate and xtemplate) the markup and core theme logic are seperated to allow for easier customization by designers and other non-technical folk. The template selection generally takes place in the theme specific settings screen.

Style / Skin layer

Currently only chameleon allows the user to select the template they wish to use with the site, so as to allow varying designs that do not need to modify the markup/skeleton layer

Benefits of the theme system

It is incredibly customizable, to the point of making many things not normally possible, easy (to your average drupal developer). It is quite close to the Drupal core API in implementation, and thus allows module developers a similiar interface to providing markup for their contributions.

Problems with the current approach

High barrier of entry

Not everyone is a developer, and the barrier for entry to normal designers is artificially high. It is considered the norm that most people should probably use xtemplate to design their site, as it allows easy copying and modification of the template (to a point).

Users wishing to customize their site are expected to learn php, the basics of the drupal theme API, and very likely the xtemplate library , where an intelligent and user-friendly set of customizable options could instead be defined.

No consistency across themes

There is a marked lack of consistency in supporting even the most basic Drupal requirement in all themes, and there is also no way for novice users to configure a lot of very common display settings without modifying the template, and this breaking compatibility and causing themselves upgrade problems in the future.

Lack of configuration options / deluge of incompatible configuration options

Some of the most common actions, such as uploading your own logo image, is still not supported from within the interface.Different themes have different settings, and they do not respect each other's settings. Configuring the primary links for example is different on xtemplate and phptemplate, and phptemplate doesn't even support a secondary set of navigation links, much less allows them to be configured. A complete set of requirements to allow for consistent user experience is not in place. Some themes will not even load without modifying the .theme php code , which in turn makes the theme incompatible with future versions.

Technical concerns

When it comes down to it, most themes really are not that complex to begin with, they override only the most basic set of hooks available in drupal (page, node, comment, block and box) , and most of the common ones do not have any additional settings, but the settings that are presented to the user should very likely be available to all themes. Also, because of the structure of the theme system at present, we consider the technology used to define the markup , more important than the markup itself. The user should not neccasarily be confronted by the fact that his site is generated via the xtemplate library, however knowing that he is running the Pushbutton design is pertinent to his needs. Since each theme has it's own set of supported features , even the same template for a different template based theme will likely be markedly different in capabilities. If a template theme developer wanted to enable the templates to take advantage of configurable stylesheets (like chameleon does) , the resulting path (both on the file system, the menu and regarding the basic logical depth of perception) will be between 4 to 6 levels deep.

Unneccesary code duplication

The Phptemplate and Xtemplate themes are not very different in nature, yet the current system leaves us no choice but to maintain completely seperate and subtly incompatible versions of these themes, along with the additional work that supporting additional template engines in their own themes,three levels down in the directory tree, five levels down in the system menu, and many levels down in the logical system view.

Supporting additional templating engines with the current system , requires wholesale duplication of many hundred lines of code, specifically template selection and basic template configuration code.

Proposed solution

Centralized configuration

Define a set of configuration settings that should be configurable from within the administration system, that handles some of the more basic display configuration requirements. Exampled of this would be enabling the user to upload his own logo to use instead of the default, and optionally toggle the display of avatars or the site name in the header. Users who do not have a graphical logo, and lack the abilities to create one themeselves, are currently forced to modify their chosen theme , simply to be able to display the name of their site.

Required features for any contributed theme/template

Maintain a checklist of required supported features for any template or theme that applies to be distributed on Drupal.org , that is designed to define a mimimum level of configurability across all themes or templates, to allow new users the ability to modify their site without needing to modify anything other than the settings in the presentation menu. This might not seem important to the average developer, but being asked to open an xtemplate file to change something menial, can be incredibly daunting to a very novice computer user.

Implement a template layer

This template layer is intended to allow developers to modify their site in any supported templating engine, while also not requiring the core Drupal project to designate a primary template library for use with the system. This template library is a simple abstraction layer that allows developers to target the library that suits their project. This does away with layer of complexity in the file system and administration trees, and is more userfriendly in that it gives the user a more consistent view of the available options. This should allow us to allow configuration of drupal .theme's and template files , from within the same interface.

Require style/skin layer support

Since templates and themes are now on the same level, it doesn't add too much complexity by adding stylesheet switching as a requirement to all themes/templates. This would go a long way to helping the user configure and customize his site without needing to break compatibility with the theme or template in future versions. additionally ..

Configurable overriding CSS

Additionally allow the user to specify additional style rules for each template, directly in the administration interface. This would allow for small modifications ,such as padding the space around the logo that has just been uploaded, without requiring ftp access to modify files which may or may not be accessible to the user. These rules should be saved in a custom.css file in the template directory

Easy copying of templates and styles

Provided the markup is not implemented using a standard drupal .theme module, it should allow direct copying of templates and stylesheets for configuration purposes. So the user can copy and modify the pushbutton template without having to open a php file and executing a global search replace, and then hope it worked. Additionally this should allow for easy installation / distribution of Drupal templates / styles. I intend to teach the install api to maintain the themes and styles available along with contributed modules.

Base theme to be supported across all template engines

To fully allow the utmost configurability and customizability of Drupal to as wide an audience as possible, we should maintain a 'default theme' , which represents a 1:1 ratio to the required features for that version of Drupal. This markup will have to be generated exactly by any templating engine hoping to be distributed on drupal.org, as a functional unit test that ensures a minimum level of compatibility with the defined standards. Because the markup should be identical across different implementations of the theme, the css should be perfectly cross-compatible between said versions. As the bulk of the customization process will hopefully be happening in css applied to this default theme, it creates a 'known good' starting point for designers who have started to require modifications to the actual markup. Because this theme is implemented in every templating engine, the designer will only be required to choose which engine he is most comfortable with, and need not be required to learn any (or much, depending on the situation) php to accomplish his task

Document the stylesheet capabilities of the default theme

Extensively document the tags, and possible designs that can be achieved by applying css to the default theme. Because this theme is being bred to be the starting point for most designers, the existing stylesheet framework will hopefully deliver a lot of value to them , in that they will likely already have memorized the class and id names of all the main elements. A forum dedicated to little snippets of CSS for customization would also help a lot of users apply their own 'fingerprint' on their sites. Moveable type already has such a framework in place, and along with decent documentation and support channels, the stylesheet capabilities will likely be able to satisfy most of the Drupal customization requirements from your (a bit more experienced) average user.

Maintain compatibility with Drupal theme API

Current themes should not be completely broken by the new advancements, but should still work with the system, as the theme api is still probably the most flexible method of modifying the generated markup , and any system that extensively modifies the output of Drupal beyond the capabilities of the template system, will still need to be able to define theme_ overriding functions. An average user should however not be required to develop extensions to the theme API unless there is a really special requirement that could not be solved from within a templating framework .. which means.

Optional additional logic to be loaded for templates

Since templates can in many cases need direct access to the drupal theme api, allow the template to contain an additional php file that will be included if found. Any additional code will be executed upon loading up the template, and the file can contain, for instance, overriding theme_ functions that would still be honored by the theme system

Move/rename theme menus

The admin/system split which puts your average theme configuration page four levels deep in the tree requires too much navigation to reach, and even then is still too deep down into the tree to deliver any more meaningfull branches (configuration settings per stylesheet for instance). The configuration should be moved to admin/presentation , which allows you to select the active template, and there should be a menu entry for each template, where you are able to override some settings specifically for that template ( a different logo might be needed for instance) aswell as selecting the stylesheet for those available for that template. Adrian Simmonz suggested that we name the template/markup layer the 'skeleton' layer, as it provides the framework for the html that is to be generated, and furthermore name the stylesheet layer the 'skin' layer. This would be accesible to a great many novice users who will be able to relate to the functionality offered by the system.

Impliment new menu editor

Leverage the new menu editing functionality in HEAD to allow a minimum of 2 sets of navigation links to be configurable from within the user interface, and also allow the template users to easily extend the sets of links in as many ways as possible

Attempt to provide painless template extension functionality in most cases

This is likely applicable for just about every templating engine but xtemplate, in phptemplate for instance, you should be able to create a user_list.tpl.php template file within your template directory, that will be loaded and used to override the theme_user_list() function. This will require some presentational logic to be used within the template files, which xtemplate does not allow, so you will always need to create additional templatename_hook() functions to drive xtemplate. The benefit of this additional override functionality is to hopefully allow as much of it to take place without requiring modification of the default template. Once again , I can hear many people puzzle as to why I don't want people unneccesarily editing the templates. It comes down to a simple support question. If you change xtemplate default or any other theme extensively, keeping track of your changes can be a very taxing prospect. By trying to keep the only really required template free from mods, it will allow for a clean upgrade cycle almost every time. see. method. madness.

Visual overview of proposed templating system

The following graph illustrates the basic logical structure of my proposed template api addition to the method Drupal themeing system. While at first glance there might appear to be more complexity than current theme system overview, it should be noted that the complexity is because of an entire 'layer' of the previous system being integrated into core , to simplify the interface that the user experiences. The user should now be able to simply think of the template being used, and not be worried about the .theme file, until such a point as the user needs functionality that can not be offered from within a templating framework.

The previous phptemplate/xtemplate themes have been rebuilt into 'template engine glue libraries' , by seperating the logic actually used for generating the output, from the logic that was used to configure the system. These functions have been centralized to allow all templates to be configured centrally.

The style layer has been extended to cover all contributed themes and templates, so as to allow for the maximum amount of configurability. The style layer uses most of the same code that the template switching does, except applied to switching available stylesheets within the selected template.

The configuration layer has been added to the theme api internally as to allow for extensive and consistant configuration options. Because these options are now centrally configurable, you are able to override them intelligently , like for instance be able to specify a custom stylesheet for every user, or tie the template/stylesheet used into the path currently being displayed. This would allow sites to create very customizable front pages using push/pull hybrid templating with phptemplate or the like. They would simply be able to create a template to be used for front page

Final words

Since I have now probably confused a bunch of people again, and I feel there is enough information here for the general populous to digest now, I will take my leave and continue with the spec tomorrow. But i just have the following to say :

YES current drupal themes will be mostly completely supported. They will masquerade as a template type called '.theme'. The theme.engine glue library will simply load the theme on template inilitialization, and since theme functions still take presedence over template functions, there should be no change in the rendered code, unless you specifically require the theme to be in themes/ or some other specific little bug. Settings might also be unavailable until a consistent way to integrate them is found

YES very little modification of the core system will actually be required, the bulk of the changes involve changing the theme system to a) check for a valid template based solution if no overridden theme_ function was available, and b) removing the theme configuration / loading system and replacing it with a template based solution that loads the right glue library, which may or not be loading a real.theme file. (if it sounds complex, it isnt. really it isnt)

YES you will hopefully soon be able to install templates and styles from the Drupal website using the install api. And css zen garden is actually a very apt example of what is capable of the proposed solution

NO no, i don't think this is adding too much complexity to the default system, I believe that from a useability perspective, the proposed changes have a very real benefit for novice users. The new system centralizes a lot of things that are currently not well supported, or do not have the neccesary documentation written.I believe that one of the major methods of allowing a much wider audience to find Drupal is to stop making all the sites look like xtemplate/marvin hacks (which they are) , and to grow the designer community interested in doing Drupal projects.

YES I still expect the first reaction of Dries/Unconed to be related to capitalization or spelling / grammer. At 4am on a tuesday , those facets of my brain tend to fizzle out and die.

YES I do sleep far too little , and I really should be getting into bed soon