Chatham House, home of the Royal Institute of International Affairs, is an independent policy institute and world-leading source of independent analysis, informed debate and influential ideas on how to build a prosperous and secure world for all.

The World Today magazine issue

They decided to rebuild their website to better promote their independent analysis and new research on international affairs, engaging with their audiences and disseminating their output as widely as possible in the interests of ultimately influencing all international decision-makers and -shapers.

They wanted a modern, responsive website with a clean design that would provide a better user experience and increased traffic. Additionally, one of the requirements was to integrate the website with their events and membership management software.

Chatham House have used Drupal as their CMS of choice since summer 2011. Their previous Drupal 6 based website was suffering from intermittent performance issues, a dated, non-responsive design and did not integrate with Chatham House’s internal membership management software.

Chatham House worked with Torchbox to produce a new, modern and responsive design on a new Drupal 7 instance. A lot of previous custom code was replaced with tried and tested combinations of contrib modules and a large, rolling content migration (more than 10,000 items) was required to move content from the old Drupal 6 site to the new Drupal 7 site.

Chatham House’s membership management software was also integrated with their new Drupal 7 website for user authentication and events content.

Why Drupal was chosen: 

After extensive research, Drupal proved to be the most powerful and customisable platforms that would cope well with their complex content structure and processes.

Describe the project (goals, requirements and outcome): 

Expert comment article

The previous Chatham House website was powered by Drupal 6. In January 2013, Chatham House commissioned a discovery phase with an external agency for the rebuild and relaunch of the site in Drupal 7.

Chatham House and Torchbox built on this work and undertook the development of a brand new Drupal 7 website to help Chatham House meet their objectives to:

  • Promote new policy ideas – publications and research projects – to our target audiences
  • Measure and be seen as a primary resource for international affairs analysis, including being regarded as a source of thoughtful, insightful commentary on relevant external events
  • Operate as an effective and efficient e-commerce site (for servicing conferences and membership in particular)
  • Provide an engaging and valued portal for members.

The work for the project could be split into several distinct phases:

The discovery and design phase

An initial discovery phase was conducted to reaffirm the information from Chatham House’s initial discovery work and agree the objectives, scope and budget ranges of the project.

Chatham House held regular steering group meetings with internal stakeholders to confirm the requirements and ensure transparency.

The outcome was a series of requirements documentation that later fed into the design of preliminary wireframes. The requirements documentation and the wireframes were then handed over to Torchbox who used these to produce a project scoping document.

The development phase

Data driven section index page

Starting in January 2014, the development of the new website was broken into 94 individual features. The development of these features was managed using Kanban. Each feature and ad-hoc request was managed with a matching ticket in Codebase, and the overall development schedule, budget and progress managed and tracked using a Google spreadsheet as the project dashboard. This dashboard was intentionally made available to all members of the project team and could be accessed at any time. Kanban provided a means to visualise the development process by showing work in progress, limiting it to focus attention on open features and making the processes of each feature explicit and easy to follow.

Working with an agile, feature-driven approach allowed both Chatham House and Torchbox to quickly and easily amend the development schedule when necessary. This flexible approach meant any changes to the priority (or even necessity) of features could be managed without throwing the project schedule into disarray.

Content migration

In conjunction with the standard project features, a concurrent series of Drupal to Drupal content migration features were scheduled to run from the very beginning of the project and intended to compliment other ongoing features in the main development stream. This approach was intended to to add immediate value and content from the start of the project. Given the content heavy nature of some of the research topics on the new website, having real content available made both prototyping new features and final acceptance testing far easier than relying on placeholder content. This content focussed process also highlighted difficulties early in the project and avoided adding unnecessary pressures towards the launch date. The migrate and migrate_d2d modules were used to drive this process.

Building the responsive, mixed-content menu

Chatham House’s main navigation menu mixed both traditional navigation items and content. A standard Drupal 7 menu system would not permit this, focussing instead on rendering a pre-defined hierarchy of menu link items.

The proposed navigation menu was not just a mixture of menu links combined with dynamic content and a membership login form, but was also a sub-set of the underlying Drupal menu structure. Fortunately, both the links to research departments and other parts of the site were relatively static - only requiring a few minor changes throughout the year. This made the process of building it a little more straight forward.

This highly unusual, but visually appealing navigation menu was built with a relatively low-tech approach. A theme region was defined to provide a container for content blocks. Most are static and can be managed by HTML literate content editors at Chatham House and the other blocks are dynamic content blocks exposed by views. All of these blocks are specifically arranged, styled with a mixture of CSS3 with media queries, custom JS and jQuery.UI tabs (part of D7 core). It is true that the links in the menu do occasionally require manual maintenance by a content editor, but where possible links to system paths (eg: node/123) are resolved to their aliased versions when rendered which does help matters.

Integration with membership system

Chatham House's membership system is the backbone of the organisation for it's members online experience and in the past version of the site integration between the two systems was incredibly limited. The new website sought to improve the member experience by:

  1. Providing a logged in member dashboard to showcase featured content, events and profile management. This was provided by a custom content type for the dashboard with lots of views, all underpinned by profile2 to power the user entity's profile data.
  2. Providing authentication support against the membership system. This was built using entirely custom code, but heavily inspired by the existing ldap module.
  3. Providing a feed of events to pre-populate website content based on known, approved events data from the membership system. This was built using feeds, feeds_tamper, feeds_tamper_php.
  4. Use the membership system as the authoritative datasource for member data, asynchronously pulling in changes from an API on the membership system every 15 minutes. This data was added to profile2 profiles, which are fieldable and readily available via other Drupal components such as display suite and views. The functionality for this 'feed' was entirely custom code, as a drush plugin, invoked on system cron instead of Drupal cron to avoid congesting the main cron process.

At no point were the website and backend membership system synchronously linked to ensure redundancy/resilience of the website. The website will not go down or become unresponsive if the back end data store becomes unavailable. At worst, if the membership system goes offline members cannot authenticate but they are notified via a message on the login form and their profile data is simple re-synced with Drupal once it does come back online. Website staff (eg: editors/admins) still authenticate using Drupal's own users table and so can still use the site in the event of the two systems being decoupled at any point.

Chatham House required integration with a number of multimedia content providers. embed.ly was used as a common interface to connect Drupal 7 file entities to multimedia assets hosted on Youtube, Brightcove and Soundcloud.

After 5 months of intensive development, rolling content migrations, testing and final content entry the new Chatham House website was launched on 2 June 2014. The adoption of Drupal as a CMS platform provides yet another example of the capability of this Open Source platform as a highly capable solution for other similarly sized and high profile organisations throughout the world.

Modules/Themes/Distributions
Why these modules/theme/distribution were chosen: 
  • Apachesolr: Drupal's core search is well understood to only be suited to very basic search requirements. It cannot provide fuzzy search results, spelling suggestions or faceted search to drill down from general search queries into more specific ones. Apache Solr runs as a stand-alone service and provides a searchable index of data supplied by Drupal. The apachesolr and apachesolr_autocomplete modules were used to push content changes to the Solr index on every cron run, replace the core search pages with more useful ones and to provide autocomplete suggestions when entering text in the search form. A custom module was also written to allow editors to override the search result snippet with a custom snippet.
  • Display Suite: Often seen as a more lightweight alternative to the Panels layout management suite, Display Suite allows site administrators to define and customise different view modes for entity types. In the case of Chatham House, we wanted publication nodes to have view modes for full content (ie: when you're viewing the node in full) as well as teasers for any listings. A restricted content mode was also defined that was intended to be used when non-members try to view premium content. Instead of showing a harsh access denied page, a teaser of content was provided along with an advertisement for the benefits of membership of Chatham House. Panels was not chosen as the comparatively steep learning curve required and features it provided exceeded the requirements and budget of the editorial team at Chatham House and therefore would not provide a worthwhile return on investment as a whole.
  • Election: a feature rich and highly extensible module to allow Single Transferrable Vote and Referendum style elections on a Drupal site. Chatham House use this to power their council elections and collect votes from members in a single STV election. Chatham House's specific implementation of this module did away with the front end display of entity pages (eg: candidates/elections/posts) and only exposed the voting form to end users. Site administrators were still able to set up the election (to provide the data model) but all other landing pages for election posts and candidates were managed as standard page nodes as these were already available, themed and easy to work with. The relevant pages were then linked together by content editors with a final link to the voting form. Results were analysed using software such as OpenSTV, which is easy to install and is complimented by a handy contrib module - election_openstv.
  • Features - used to power the more complex project features as well as a basis for the initial whitelabel site Torchbox provided to speed up the development process. Mostly used as a container for exportables to reduce the amount of manual functional work when deploying new work.
  • Linkit: Augments existing link fields to allow references to internal and external resources. This is great for editors as it means they can reference internal assets by entity id instead of absolute URL paths. It also provides a great content search to allow editors to track down the content they want to link to. It also supports external URLs which is a great benefit to cut down on the number of fields on the site.
  • Media: Provides centralised asset handling for images, documents, videos and audio files. Chatham House had a huge amount of legacy assets from their Drupal 6 site as 'Resource' nodes with CCK fields. During the migration, these were converted into File entities with appropriate fields.
  • Oembed: Chatham House had existing multimedia from a variety of providers ranging from Soundcloud to Youtube to their new Brightcove video platform. A common interface was needed to allow these assets to be presented as stand alone file fields as well as embedded assets in WYSIWYG rich text fields. Most prominent media providers implement the oembed protocol which, when combined with a presentation layer provider such as embed.ly, allows all sorts of assets to be managed as File entities with Drupal but without the hassle of providing custom display code for each provider.
  • Nagios: The contrib module allows the Nagios daemon on the web server to become aware of certain Drupal-specific issues such as over-due cron runs, pending security updates and pending database schema changes. Custom plugins were also provided for basic server and data feed monitoring.
  • Rabbithole: Allows site administrators to customise how certain entity bundles react when viewed in full display mode. Useful for controlling access to secondary content types or supporting content not intended to be viewed as a full page in itself. Chatham House make extensive use on this for content types such 'Contacts' and 'Section highlights', the latter of which is often used as a proxy node on key areas such as the homepage. In that kind of scenario, the node can be used to present a title, image and teaser and when viewed will redirect to the URL defined in an underlying link field. This provides a huge amount of flexibility for manually curated content without the overhead of having to theme a teaser display for every single entity type.
  • Securepages: Allows site builders/administrators to define which, or all, pages to be served over HTTPS. In an ideal world, all traffic would be run over HTTPS (with SSL/TLS termination) but the use of third party services such as embed.ly and ShareThis was incompatible with this approach, so only key URL paths are protected. Eg: administration pages and all webforms.
  • Workbench: permitted easy content revisioning for content editors as well as content authoring workflows for the site. The following sub-modules were used: Workbench media, Workbench moderation. Considerable effort was made from the start to keep the editorial roles and workflow as simple as possible and avoid the pitfalls of complicated content access/moderation . Sticking to a standard Draft > Moderate > Publish workflow with editor and moderator roles makes it really easy to understand the publishing workflow and possible states revisions can move from and to. Selected staff could also have the moderator role which allows them to publish nodes marked as in need of moderation. The revision scheduler was also employed to allow press releases and other time sensitive revisions to be published at pre-defined times. In the past, Chatham House editors would stay awake until midnight in order to publish or unpublish time sensitive material. This may be the first Drupal site that has contributed actual sleeping hours back to end users :)
  • Mothership: A useful starting point for the Chatham House theme that wasn’t as bulky as Omega nor as minimal as Zen. The Chatham House theme itself was built as a child theme of mothership and comprised of HTML5/CSS3 preprocessed with LESS CSS. Additional libraries such as modernizr and respond.js were employed to provide additional browser capability detection and media query handling for earlier version of Internet Explorer. Flexbox was also used to quickly theme key page layouts such as the International Affairs and The World Today magazine pages.
Community contributions: 

During the course of the build Torchbox contributed patches to other drupal.org projects, such as:

Project team: 

Organisations involved

Chatham House logo
Torchbox logo

Torchbox team

  • Ben Enright: Designer
  • Edward Kay: Discovery process
  • Ged Barker: Project management
  • Johan Gant: Technical lead
  • J-P Stacey: Content migration and development
  • Justine Pocock: Front end lead
  • Olly Willans: Design/user experience direction
  • Paul Gill: User experience/discovery process
  • Simon Minter: Front end development

Chatham House team

  • Agnieszka Grychowska
  • Jason Naselli
  • Josie Hock
  • Lee Tickett
  • Marcus Cummings
  • Mark Horrell
  • Michael Farrell

Comments

chanakya’s picture

The revamp work is great! The website is faster and more user-friendly.

Thanks for sharing the development details.

As the website is very big, the Server/hosting plays a crucial role in website performance. Could you please put some light on the Server setup for the website?

Thanks
Pushkar

johan.gant’s picture

Thanks for the positive feedback. It's a fairly standard architecture and spec for Drupal, with Varnish for anonymous traffic and a suitably tuned hosted server. I can't give you any specific details I'm afraid, but with enough careful configuration you can a surprising amount of performance out of a relatively modest spec.

chanakya’s picture

Thanks Johan for your reply.

Do you prefer Varnish (https://www.drupal.org/project/varnish) over Memcache (https://www.drupal.org/project/memcache)?

In some of the case studies, I read about creating separate cache, database and web server. Do you think it is required for a website of the size of Chatham house website?

Best
Pushkar

johan.gant’s picture

Varnish and Memcache play different roles so a comparison of the two wouldn't be useful. Where possible, any non-trivial D7 site would do well to use memcache (memory's quicker than db access) and Varnish. That said, D7 does come with db based caches for all sorts of things... but you'd need a pretty hefty server to get the best out of them during heavy periods of traffic. Putting web/db servers on different machines can help in terms of resilience but it would always add a slight overhead for network I/O between machines and services. Each site/case is different - you really have to look at the requirements in each case and apply an appropriate architecture in each case.

chanakya’s picture

Thanks Johan for the reply.

duntuk’s picture

Out of curiousity, during development, have you tried using the Bootstrap theme? Asking because, like Mothership, the latest version of this theme does a pretty good job at stripping out excessive markup, plus it uses a well polished framework (Bootstrap 3.x)...

johan.gant’s picture

Hi duntuk,

Haven't used Bootstrap theme to date, but I'm sure we could have. Most theming projects we've done to date have been great to prototype with CSS frameworks such as Bootstrap, but for a custom design/theme we often find it more productive to start from a very minimal, clean slate and build upwards instead of starting with and sometimes later overriding existing framework elements. Thanks for the recommendation in any case! Pretty sure @WigglyKoala (Front end lead on the project) could answer any further detailed questions you might have.

diomede’s picture

Awesome website, congratulations!
I have disabled javascript in my browser to test your responsive layout; take a look at your main menu, list items overlay other elements.
Thanks for sharing your project.

www.concertpass.com - the secure way to buy concert tickets online!

andyaharons’s picture

This question has been asked before, but I would like to emphasize it:

As the website is very big, the Server/hosting plays a crucial role in website performance. Could you please put some light on the Server setup for the website?

Could you shed some light? I'm think of building a similar website.

----------------------
I'm currently working on babybumble.com