After winning control of the New York State Senate for the first time in almost 50 years, the Democratic party turned to Advomatic to quickly deploy a new website powered by Drupal. There were many challenges to be overcome, and their requirements pushed the envelope in some areas such as permissions and work flow. In the end, this site stands as a testament to what can be accomplished by a focused and experienced development team using the power and flexibility offered by Drupal and its community.

Multi-Site Structure

Previously, each Senator controlled their own little section of the web with a proprietary ASP-driven site, with little continuity between them. The Senate wanted to allow each senator to continue to control their individual sub-sites, but wanted content to be more easily shared between the sites, and for desired articles from individual senators to occasionally be featured on the main section of the site.
This obviously required a multi-site solution, and this was the first issue to be tackled, since its implementation would inform the rest of the development.

We looked at several proposed solutions and modules that help with deploying such a site, and in the end decided to create a custom method (similar to what we deployed for Air America Radio), because of the unique requirements. This consisted of creating a 'Senator' content type, to which every other content could reference, which would be used for grouping. For instance, Senator Malcom A. Smith's section groups all of his content, with a unique header, a specialized menu navigation, blog, etc.

The design (created by our very own Amanda Luker and Jack Haas) called for two basic themes, dependent on party affiliation. We simply added a drop-down field with a 'Blue' and 'Red' option, originally with the intention of switching the global $custom_theme as needed. In the end, we decided to simply print that in the body class, as it was easier to simply use a CSS switch rather than an entirely unique theme. We used Cufón for image replacement throughout the site.

Menu Navigation

Menu navigation was far more challenging to implement. They wanted each senator to have its own menu for primary navigation and have the menu automatically pre-populated, but to also allow each senator to override the menu as desired. We quickly dismissed the standard menu as an option, as we did not want to open up the can of worms of allowing full access to the system menu by a disgruntled office editor. Even if we modified the access to only allow access to a specific menu, that wouldn't solve the need to automatically create a suitable menu from a template, which would serve 95% of the senators.

In the end, we opted to use a Link field for the menu, which would be printed by overriding $vars['primary_links'] in phptemplate_preprocess_page. Then we simply checked the path to see if we were on a page referencing a senator, in which case we would do the override. We also implemented a custom token for this, so that we could fill it with such useful default paths as [senator]/blog and [senator]/gallery.

As there were many other cases for which we needed to know if a specific page load referenced a senator, we put all the arg sniffing in a static variable during hook_init, so that we could easily discover that: a call to nyss_senator() would return either FALSE or the referenced senator's node.

Data Migration

After creating the basic content types, with required senator node reference fields, it was time to populate our site. The original vendor had provided us with a backup disk with about 12,000 pieces of content. We wrote a quick migration script to map the original data with our new content types. In the past, we would simply do this over the command line. This time, I wanted to dive into Drupal 6's batch operations, which offers a slick interface, so I put the scripts into a special one-time only admin callback so I could watch a druplified progress bar while I ate popcorn.

The original data being in a Microsoft Access database presented several problems for migration. Firstly, the original dump was corrupted, as the original migration returned about three quarters of the nodes with odd characters. After investigating why apostrophes were being turned into ’ (among other problems), we learned the data had been corrupted when the original vendor backed up the database. Normally, PHP's iconv would have been able to convert from Latin-1 character encoding to the required UTF-8. However, the corruption was in the data itself, which meant we had to get a new backup. A few hours later, the new backup was delivered, which was properly encoded.

There were other problems with the original data: there was no unique field on the rows, images and pdfs were stored inline (which meant we had to modify those on the fly to match our new consolidated /files folder, and also turn them into suitable file objects for storage in a filefield). Then there was the problem of editing the 300 page State Constitution as a single node.

These issues were easily surmountable, and soon we had a working site. Interestingly, the original proprietary CMS didn't have anything approaching Drupal's check_plain, which meant that in the past, some innovative senatorial editors had gotten away with such fancies as displaying a page's title as a JS marquee...

Editor Permissions

With 12,000 nodes filling in the sections for 60+ senators, not all of whom play nice together, we next had to tackle the roles and permissions for the site. This was not a straight-forward matter: they all had to be able to edit content from their respective senators, but not that from other senators. Additionally, if a senator had multiple editors, they would need to be able to edit each other's work, and we had to account for the possibility of some editors being shared amongst senators.

We made use of hook_node_access_records and hook_node_grants to handle the grunt work for this. We created a grant realm for the possible field_senator's nid on a node, and mapped this to the office editors assigned to a senator. This in turn was assigned with a user_reference field selecting from a view of users with the Office Editor role. Finally, we altered the menu callback for node/[nid]/edit to check the permissions, to keep folks from being mislead by an edit tab to nowhere.

Flag FTW

We made heavy use of the Flag module throughout the site. This included not only promoting content to the front carousel (which was created using jQuery Cycle) and to other blocks on the front page, but also to a similar carousel and tabs on each senator's page, galleries, and other places throughout the site.

Of course, this presented its own set of problems for our unique permissions structure, as a senator should not be able to flag content for another senator. To circumvent the problems, we altered the flag links so they would not appear if the editor did not have permission to edit a specific node.

Bread and Butter

Views, of course, filled the site with our content. After creating and finalizing the View blocks and pages, we exported them all for better performance. There were no real problems with this stage of things, except for a few view queries that were not possible to create with the standard UI. In those few cases, we made use of hook_views_query_alter. Additionally, because many of the views required an argument of a referenced senator, and we had a custom token for our senator's path that didn't necessarily match up to the title, we had to write our own views argument handler that we stuck in the argument PHP validation block when required.

As a site of any complexity would need, we needed to implement several custom blocks. We have refined a system over several sites that we continued here, that allows for the flexibility allowed with Drupal's custom block system, while extending it to allow things to be easily controlled through code, rather than embedding such things as a block's per-page appearance on the administration form. Basically, when creating a new custom block, we define its visibility as 2 (so the pages are controlled with a PHP callback), and set that to return a custom callback with the $delta. Then if we decide in the future to move one or more blocks to another page, we can change the settings in the callback, rather than submitting each block configuration page individually. To further ease configuration, and take advantage of Drupal's just in time file loading, we also create theme functions automatically when a new custom $delta is added, loading the new theme_custom_block_$delta.tpl.php file as needed.

There are a lot of little things I would love to tell you about, and there was much more that went into creating this site than I can get into without turning this case study into a novel-length exposition. I haven't even discussed Committees, Legislation, or their Open Data project.

Advomatic's development team was superb! I was assisted by Jonathan DeLaigle, Amanda Luker, Jack Haas, and Marco Carbone. Overseeing the project were Fred Gooltz and Dylan Clear. A hat tip to EchoDitto who was responsible for the early project management and first draft of wireframes/specifications. Craig Leinoff also jumped in feet first from the Senate's CIO office to help develop the site and get it ready to launch.

I am excited to see more of our government extolling the virtues of Open Source, and am proud to have helped to deliver a robust Drupal site to the New York State Senate.

Aaron Winborn is a developer with Advomatic. Besides development, Advomatic also offers a wide range of other Drupal services, including maintenance and clustered hosting. In addition to helping roll out sites such as the New York State Senate, Air America, and Mozilla, Aaron also contributes heavily to the Drupal community, including such modules as Embedded Media Field and Views SlideShow. He has written Drupal Multimedia, published by Packt Publishing, and is mentoring a Google Summer of Code project to help roll out the upcoming Media module. You can read his blogs at Advomatic and


plutado’s picture

This is fantastic! I know Jonathan DeLaigle, who worked on this site. It's super cool to see some very talented developers I know make it to the front page of d.o!

ahoppin’s picture

Great post Aaron-- thanks. It has been a pleasure to work with Advomatic (and EchoDitto) on this extremely complex project. From the perspective of the Senate, we rolled out ~95 related websites in one fell swoop! Each Senator and each Committee considers themselves to have their own "website" as part of this rollout, and each had different ideas about what a "good" website ought to include, and each needed staff with varying degrees of technical competence to be trained to be able to manage their new web presence prior to launch.

We had to deal with immense technological and workflow complexity as a result, so we were lucky enough to be able to build a great in-house team led by Offlein and nonecknoel to take over primary development and QA responsibility from Advomatic for the final two months of the project, so that upon launch we were fully empowered to support our own site, and to immediately begin work on v2.0!

Advomatic was fantastic about collaborating with us on this hand-off process, including continuing to work through last minute bugs with our in-house team right up through launch.

This experience is in stark contrast to the old Senate website(s), which was built using a proprietary CMS from an external vendor such that the Senate Technology staff had little ability to support itself or to quickly implement feature requests. As a result, these websites were an afterthought in the Senate's communications and operations.

Our new Drupal website has put technology squarely at the strategic center of the NYSenate's communications and operations.

Thank you Drupal! :)

justageek’s picture

Could you elaborate more on your custom block methodology, I'm new enough that I didn't follow the whole custom callback thing using $delta, and I really didn't follow the automatic theme functions.

if time permits, would love to know more.

aaron’s picture

OK, I'll blog about that in the next day or two and link that here.

aaron’s picture

drewish’s picture

aaron’s picture

Nice! Thanks for the link!

vertazzar’s picture

how many tables does database contains?

dman’s picture

Thanks for that write-up!
Normally I advise against enforcing user permissions too heavily - but I can see that this example is certainly a case where you can't trust your neighbouring editors!!
I think that's a great result - sounds like it was handled effectively. I hope you didn't have to go too far into custom code - it sounds like there are many unique tweaks. The custom menu management looks fascinating.

iantresman’s picture

Good write-up. It would be nice to see various modules have feature requests where programming was required to circumvent their limitations. e.g. the Views module, which was unable to implement certain requirements.

skullcap’s picture

Nice site, complex, kind of make my head hurt think about it. lol

It is nice to see open source been used in the main stream.

sefcy’s picture

Thanks, it's cool !

vr_mex’s picture

How about releasing the permissions system talked about in this article as a module, right now, admin of new users as is in D6 is limitted, for instance, when a master admin (user=1) rules and needs sub-admins to help him, say approve new users without giving sub-admins full admin permission of the site...

aniediudo’s picture

Hi aaron,

...the front carousel (which was created using jQuery Cycle)

Can you be kind enough to walk us through the integration of the jQuery Cycle plugin?


nmridul’s picture

Thanks for this great writeup. It is always amazing how drupal provides us all the tool we need to create what we want.

While I liked the design, I feel that the background gradient does not merge slowly into white, but there is an abrupt ending to white from the gray shade. So the page background suddenly becomes white, giving a feeling of broken halves. This is more prominent when we scroll up/down on longer pages.

Appreciate this great writeup.

wmclark’s picture

This consisted of creating a 'Senator' content type, to which every other content could reference, which would be used for grouping. For instance, Senator Malcom A. Smith's section groups all of his content, with a unique header, a specialized menu navigation, blog, etc.

I am developing a website that will require similair grouping characteristics shared between several cck content types.

I am torn between using a node reference field vs. using Organic Groups Module. Why did you decide to use node reference instead of using OG for grouping your content types under the Senator content type? Perhaps your insight will help me decide what would be best for my implementation as well.

It sounds like you could have used OG for your permissions handling of editors that work under more than one senator just by adding those editors as a certain role under each "senator" group you wanted them to work for. I have not used OG but I am considering it for the project I am about to begin.

MiMe’s picture

It's really nice to see that more and more websites are using Drupal as their platform! Great job!!

Hitta massage

escoles’s picture

Good case-studies of challenging projects like this are very helpful to me, and I think to others as well. Often the approaches that are considered and rejected provide insights and hints (what didn't work for you might work for me).

I'll second the request to learn more about the block system, but I'd also love to hear more about the editing permissions. In your CST*, I hope you'll consider contributing whole articles on those topics.

*Copious Spare Time

aaron’s picture

OK, just used some of that CST to blog about -- I'll see about the other requests in the next week or two...

codebutcher’s picture

What module did you use for the home page with the rotating text and graphics?

Advomatic’s picture

The Views Slideshow: Imageflow module, developed by Aaron Winborn specifically for the NYSS. It's already got over 270 users! WOOT!

codebutcher’s picture

Great! [looks up] NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO! It's only for Drupal 6.x I'm running 5.x. Hmmmm. Is it worth converting a site now from 5 to 6? Hmmmm.

aaron’s picture

At the NYSS, we used Views to create the listing, and themed it with the jQuery Cycle plugin. There are several other modules giving this functionality, and a plan to unify some of them with an API using Views Slideshow.

cmgui’s picture

Great site. Very polished!
Did you say you use Views? No wonder the site is a little slow.

iantresman’s picture

Is there an alternative to using Views?

cmgui’s picture

yes, write your own module.

dzrobarge’s picture

Thanks for this article - I'm trying to learn drupal and keep getting stuck on how to structure & our site requirements are very close to what has been done here, this is very helpful!

Regarding the multisite "custom method" solution you spoke of - is that an advanced drupal user thing to do? I completely get "creating a 'Senator' content type" - that would be using CCK correct? Is there more to it than that?

My plan right now is to use mainly sections and taxonomy to handle the multisite functionality we need (more like "sub-sites" similar to NY Senate).

And 12,000 nodes -wow. How does that look in the admin??!

Other than structure I'm stumped on set up for easy managment. We will have tons of blocks and haven't a clue how to keep that all straight.

vinayras’s picture

Thank you for sharing the points with us. It will surely help me a lot.

Thank you

rockitdev’s picture

I have a similar requirement with taking a custom token from the URL and injecting it into the view. would it be possible to share the argument handling code used in this case?

SamTN’s picture

I can write node, block, taxonomy,... module, but I don't know how to put it together :(. I don't know how to put a form with a list of nodes in one page.’s picture

Nice work. Great site. Very polished!

Thums up :-)