The Vintage Aviator - Front page, on release, May 2008

Last week, I released thevintageaviator.co.nz. It was probably one of the most challenging Drupal sites I've ever done, but also the best so far. It's a presentation of a local WW1 Warplane workshop, containing build stories, reference material, and thousands of great images of plane reproductions and archive materials.

The build story is a long one, mostly because the requirements shifted along the way quite a bit. This story is targeted at current Drupal site developers who are interested in the tools and techniques used. Mostly to head off the "so how did you do that?" questions we get whenever we do a write-up :-)

However among the interesting thing for developers and designers will be the complex layered look, the way this layout was optimized to scale to all screen sizes, and the surgery I did to Drupal to make this all happen.

Initial feedback we've received so far has been overwhelmingly positive (save some justified grumbles about the page weight) and I'd like to share this case study.

Build Process

An inside page, rich with clutter

The development environment went through many stages. Syncing a working database with working development is not a job for the faint-hearted.

For almost all of the project, the tech side was me and only me. Plus one Graphic Guy, and a part-time wordsmith. Over a year an a half, mostly in small, concentrated bursts with long gaps.

In that time, the theme and purpose of the site went through many changes. I won't go into detail, but at one point (version 15) I actually made a clean break and rebuilt from scratch, transferring the content by hand and with views_bonus_export, node_import into a new sitemap, theme and architecture, and re-importing the galleries (image_import) into their new (named) locations. That was a good and necessary move! Once you've made enough mistakes, you've gotta know when to start with a clean slate!

Due to the extended build period (and to our own fluctuating circumstances) the site moved around a lot. First locally on XAMPP under my desk at the office, mirrored (working(!) on a thumbstick for offsite work). Then I took it home and ran off my local dev machine (Ubuntu). When we got to preview stage I pushed it up to a hosted server (Mediatemple) we could all get at a bit easier (My ISP wasn't really up to serving from home).
Then we shifted office and I had the Dev site running on a real Ubuntu server, while still using the hosted one for content development.
Eventually we could push it to its intended destination on a local helpful hosting provider (http://face.co.nz) but that was PHP4 only (some rework required, small stuff).
And once that went live, the content requests just crushed that machine (the site is HUGE file-wise), and they immediately offloaded it to a dedicated box. Which I was able to upgrade to PHP5 again in the process.
If nothing else, this was stress-testing for platform compatability!

Development environment throughout was The Eclipse IDE with PHPEclipse, The Web Dev toolkit, and occasionally RDE (which allows remote-site project editing over SSH etc). I really like Eclipse.
And on Windows as the desktop (plus a little from my macbook). Firefox (with of course the web-dev toolbar) for most dev, with frequent IE6/7 switches (and its developers toolbar, which isn't as cool, but still helps heaps).

Content management continues on the live site, while remaining code and theme development happens on our office dev box.


As you can imagine, keeping that all in sync was a tedious undertaking. I used and improved some scripts and tools I'd developed earlier to assist in site synchronization : sitesynch.module as well as a bunch of commandline shortcuts (rsync and mysqldump)

Standard Modules

The site is Drupal 5 (now 5.7). It was first scoped and prototyped a year and a half ago. Even now in the middle of 2008 I don't think I could get the right results up and going in D6, although I'm making an effort to do so with current projects, and intend to drag any medium-sized modules I find lacking into the brave new world.

The Usual suspects

CCK of course, although I avoided it for almost the whole development, and only pulled it in at the end to add a credits field to the images.

Views I use mainly for some admin screens, like a 'show all content' all-in-one audit page for proofreading. I also use a fixed (modified for table layout, taxonomy terms and image thumbnails) version of edit_view for bulk image caption editing, as managing thousands of pictures with several clicks each is painful.

coder and devel I kept in the background during development.

Other standards include:

  • site_map.module (mostly unmodified, just minor theming)
  • xmlsitemap.module (worked out of the box, which made me and Google happy)
  • nodeteaser.module for custom intro text, as the first paragraphs were not helpful enough in general, and the design really constrained the size of the text we could use.
  • htmlcorrector.module ... because there's a WYSIWYG :(
  • dhtml_menu does the expanding main menu, although I had to break it a bit to change the behaviours where it was 'unintuative' or redundant when clicks didn't actually go anywhere. I believe this could have been solved through design (showing an expand/contract icon) but that never came through. It still needs to be broken a little more, as currently it loads the full menu tree even when not required. Earlier I tried the jstools activemenu also, but the speed/efficiency tradeoff for AJAX didn't work out for me.
  • taxonomy_multi_edit.module for some admin stuff during development, shifting images between galleries etc. I can probably remove that again now.
  • pathauto.module - pretty standard stuff, although I'm not sure it's working every time.
  • sifr.module for the page header font. It causes a little page flicker however, but that's something to do with margins in the themes. It does show up a little slow on some connections, which is disconcerting, but the typography won out.
  • fckeditor.module We had to have a WYSIWYG, even though I find them all various flavours of evil. Once I and the content editors had got used to the pain, and I'd inserted a handful of custom styles and stylesheets for it... It's OK. After much cursing, our designer was able to do some pretty cool things with the type.
  • imce.module For image insertion. The file browser is horrid, but it works. Navigating thousands of images over a net connection is never going to be easy, but after I'd ranted about the importance of good file name conventions, and added tokens to create custom directory paths it became almost manageable.

Image Galleries

Themed image_gallery

Choosing an image handling solution for Drupal is always a challenge, because there's so many of them.
I choose to stick with the simple ones I know and understand, so I did not use any of the fancy 3rd-party gallery extensions, just image.module and image_gallery.module. Importantly, I always think of my resources (images) as first-class citizens, with their own metadata, tagging, and place in the content management. For this reason I couldn't use any method that did not treat images as nodes. This eliminated the attachment-based ones.

We worked up a themed combination of fixed, proportional and floated displays for different contexts, as well as several extra additions (see below). Basically, there were many iterations and many styles before we got to this styled solution.

I worked in an image gallery ordering option, so I could list by filename-alpha (which was just a requirement for bulk-uploaded galleries, reverse by date just was wrong). Image+views came along later, and makes this option slightly less neccessary.

I used image_attach.module to select a representative image for each story in the teaser, and taxonomy_image.module (which Nancy has been steadily improving recently) for the topics.
Both slightly modified over time to integrate with imagecache.module for the custom thumbnails (more on that later).
I considered imagefield for that, but it wasn't mature enough at the time, and (I think) didn't create image nodes, just 'fields' to do it. And image_attach, although it's being left behind a bit now, just felt more integrated into the old Drupal UI and theme system.

I modified image_attach so it could re-use existing images from our library, rather than requiring a fresh upload (and probably duplication) each time. The drop-down list is huge, but with good naming conventions it's still OK for us so far.

Hm, has that not been contributed back yet? Or maybe someone else did the same recently. Ah well, it's there now.

Modified Modules

  • taxonomy_context.module

    Trying to display taxonomy-ordered structure, pages, teasers, and menus and breadcrumbs still worked out much harder than it should be. I worked with taxonomy_menu for a while but just found it incompatible with too much of the normal Drupal-way of doing things, and the URLs were horrible.
    For a while I used my own version of a taxonomy_menu module that rebuilt menus on-the-fly every time taxonomy terms were modified, but that got too complex. As it is, we still add items to the menu manually as needed, for control. With some help from my updated term_edit.module.


    I hacked taxonomy_context a lot to theme it and get it to display items in the structure I wanted (it was severely lacking in good theme hooks, and still pushes 'content' into the 'help' zone!) and to support recursion.

  • video.module

    Although it worked fine in prototyping, we ended up making a totally custom themed FLV player, so I combined video.module with a new theme_video_play_flash() override.

  • imagecache.module

    result of an imagecache process, resize, alpha, background, crop, flattenscaled,cropped original

    ... I did a lot of things to imagecache. A big shout-out to dopry, Dimm and team for the work done there, especially the move into imagecache-2, which happened over the course of my work.

    First, I glued it into image.module presets - So I can now use imagecache presets (a nicer scale&crop, and watermarking) on normal image.module derivatives like 'thumbnail' and 'preview'. That kept everything nice and Drupally and together!

    Then I worked through the watermarking and extended color effects, first in imagecache-1, then imagecache-2, and now in its own contrib module : imagecache_actions.module. This was used over time to create many different effects for our graphic looks - only a few of which have ended up in use in todays site. Currently, its main job is watermarking.

  • lightbox2.module

    lightbox

    I severely hacked lightbox.module. Our current version is quite customized to please the designer and the site theme. As all lightbox code is written on-the-fly via javascript, there was some hairy code and javascript changes buried in there. But after a week or two of reworks (because of several totally different designs, arrg) we got it stable again. What we have now is actually one of the closest to the original ... after all that :-}.

    Later, I reverse-engineered a little more of the lightbox.js to launch it on demand from within a Flash image flipbook. Gee, that was fun. Try it here.

  • piclens.module

    piclens

    The new guy on the scene, and I love it! DO try piclens! I fell in love with it, considered doing a module, then three days later saw swentel had a working implimentation ready for me. Thanks!

    Although we are not publishing the piclens slideshow viewer directly, our galleries are piclens-feed enabled (with many experimental tweaks, like 'next:' links). PLUS, once I discovered how simply right the piclens/RSS feed method was, we reworked and developed all the flash gallery extras to read the gallery feeds! Versatile, re-usable, semantic, easy, portable... I rave!
    I think I'll be using this methodology for ALL my client-server communication from now on.

Custom Modules

Note: Most of these modules are not yet contributed back to Drupal.org. Some may, but please don't bug me about it unless you want code you may cut yourself with.

  • edit_term.module

    edit-term module

    I put together a handful of enhancements to term management pages and we bundled that into this module. I don't know how we used to manage without it. It saves clicks! It's a few small changes that makes Drupal a better UI. This project IS available for download

  • imagecaption.module / lightbox_everything.module

    lightbox_everything in action

    I took a look at imagecaption.module, then hacked it beyond recognition and came up with something quite different. This is a filter which finds raw images embedded in a page (as in via the WYSIWYG) then backtracks to discover what (if any) node this image represents.
    It then reformats the embedded image to add a few classes and css stuff (for alignment and scale) and adds the caption, border and hooks for the lightbox and imagecorner effects.
    Pretty tricky all 'round.

    It MEANS that we can use the WYSIWYG to just dump images into a page, and they end up looking like you see here, resized to the screen, and lightbox-enabled.

  • gallery_attach.module

    Gallery_attach embeddedGallery_attach UI

    After a few attempts at deciding whether our stories should have all their images embedded in the text, or link to the full gallery, or as a child page, or a combination, I came up with gallery_attach.module.
    This in the first instance works a bit like image_attach. On the node edit screen you choose an available gallery to 'attach' as supporting illustrations. During development I extended this from the basic "show all thumbnails below the story" to add a "carousel/preview" view, a "flowview", and a "flipbook" view. And eventually, an arbitrary block view to add anywhere, and a gallery-attach-block-filter so these views can be embedded in the text also. The code is cool, but unpolished. The gallery feed data draws somewhat on the piclens feed mentioned above.

    By using this utility, I was also able in some cases to create page nodes that contain nothing but a gallery thumbnail view (indistinguishable from a normal image/tid/n page) that was easier to work with in the CMS, and could have a descriptive paragraph, term and teaser assigned to it. This is easier and more consistant than trying to tag and position real gallery views.

  • spacefiller.module

    This is a custom module I wrote to leverage the jstools dynamicload(block). It incrementally requests content from a pool of (imagecache-processed) gallery images to fill up gaps in the page. It does this post-load, as it's only then that we can calculate the page dimensions, and also so that requesting all those images doesn't slow down page loads.
    You'll see it in the left column on a huge page

  • phraselinker.module

    Phraselinker gets liberal with cross-references

    I know that hyperlinks are easy, but even (especially?) with a WYSIWYG, finding the target you want to link to is a chore. I know there are a few existing markdown filters (wiki) to make it easier, and probably a few more bite of this apple (like glossary.module?), but still...

    What I did was add an extra keyword filter to auto-link every occurance of a keyphrase to an associated URL. Now some pages are liberally scattered with cross-references automatically, making some of them look a bit like wikipedia or Everything2.
    It's a cheap win so far, but I'm not sure how this trick will scale. So far we just have to be careful to not add too many generic phrases to the list, and ignore the fact that sometimes it gets a bit self-referential. (Pages don't link to themselves, but they are very likely to link to parents liberally.)
    It was a fun module to write, but it needs some better tuning to be safe in the wild.

  • The ACDSee image viewer assists quick captioning before you import images in bulk

    mediadescriber.module

    You won't see this in action on the site, it's an admin/archiving utility I added to help manage image galleries. Once I found myself dealing with hundreds of captioned images (supplied on CD, as uploading individually was impossible) and changes to them requiring exports, dumps and replacements, I started using a simple metadata file - descript.ion text files (as seen in ACDSee and others) to read the file info in. And, soon, to write it out again alongside my image galleries as they are stored on the server. Based on experience with site migrations, I hate my cannonic info to be tied in to any one system or database, even Drupal.
    This tool lets me save, archive, zip and send full image galleries with their titles and captions in a format I know will still be useful in a decade. But for now it just meant I could communicate meaningfully with the photographers and indexers who were supplying the content I needed to manage.
    It integrates directly with image_import.module when reading, and saves to a text file when updating images.

  • sitesynch.module

    I mentioned this earlier. It's a tool I developed over several sites to provide one-click 'snapshots' of code, files and database from any dev site, and also uses RPC calls and other tricks to download, unpack and sync between two mirror sites. I'm also developing page-compare utilities to assist in one-click one-page 'updates' from site A to site B. In Drupal 4.7 I even had it working to directly compare any Drupal admin form with the same page on a remote site and submit changes to it - useful for syncing admin changes I'd made on dev to the staging server. Unfortunately, the Drupal5 API with its extra security checks broke that feature, and I've not been able to upgrade to that again.

    If that sound interesting to you, sorry, it's got too many dangerous corners to be released yet, but it may happen.

  • pathfinder.module

    Another tool to make migration less painful.
    Pathfinder is an URL-rewriter that works both as an input and output filter.
    When developing content on one site that's going to end up on another host, and trying to embed images or paste links ... it's easy to accidentally find yourself with full URLs that just won't work when live. Pathfinder scans links on submission, and modifies the source before it even gets saved to remove full local http: links and other site specific (eg drupal-in-subdirectory) artifacts. At the other end, it has an output filter that checks for some common errors and repairs them. In my case I tuned it to detect missing embedded images and find the real location if possible. Neccessary when I applied my grand file-renaming patch.
    Now I (and more importantly my content editors) can copy and paste URLs with abandon, knowing that those landmines are being diffused when I click 'save' and won't suddenly bite me when I switch from test.example.com to live.example.com!

    This one is a candidate for drupal.org release, although it's highly tuned to my development style and requirements. I'm sure it will need more exposed options (which will be complicated) before it's much use to other folk. If I can get it workable, I think it will avoid a lot of newbie gotchas.

I apologize if this list is a bit of a tease, but no, you can't have them in the forseeable future. It's just here as a run-down of the challenges I encountered and how I solved them. I don't have enough unpaid time to commit to more support requests.

Theme customizations

Sorry... there is one table holding the layout together.

Although I'm a semantic snob, by the time we got to this (Template version 17) with its transparent, overlapping corners, tiled and positioned backgrounds, Fully liquid displays and everything... Pure-CSS just couldn't cut it. Not with reasonable support for IE6.
So there's one old-fashioned table there to hold the corners out, although all the imagery is pure-CSS, and switchable too! There was at least one other 'page' background with different textures and colors that uses the exact same HTML hiding within my theme dir.

Floated CSS divs just had no concept of 'align:bottom' or 'height=100%' for the elements I needed.

Visuals

The gorgeous design is all down to my man Mark Williams here at http://gomi.co.nz/ . He put up with my curses against Flash and drop-shadows while I put up with his always putting elements outside of the rectangular spaces I'd allocated for them.

Stand-out achievements include:

  • Finding a way to get full floating alpha-transparency working in all browsers (even IE6) and at the same time reducing page load by like 400K over what it was taking using PNGs!
  • Coming up with a dozen radically different designs over the course of the project, each of them perfect in their own way, and still ending up with this magnificent look.
  • Jumping on the boat with the semantic RSS feed concept and developing our custom page flippers in a way that makes my semantic pedantics rejoice at the thought of using Flash sensibly!
  • Check out what happens when you resize the banner on the front page! I actually got this effect working first with pure-css and gifs first (!) but this is glorious.
  • Of course, all of the image-prep and effects, using Photoshop to do his thing but still leaving layers in a way I could slice transparencies out of later.

In broader Kudos:

  • The image resources we had to work from were outstanding! Both the modern photography and the scanned archive imagery was of the highest quality, and inspiring source material. Our client was very forthcoming with sourcing and preparing these for us, with some fast turn-around from "what if we had these sort of images to work with?" to "Here's a CD with 60 images to choose from!". That phase was great.

    We have a library of goodies to be prepared to add even more texture and variation to the pages as we go on.
  • Our client-end content manager had to deal with a truly miserable internet connection and a small-screen laptop for much of the build. Trying to preview, and upload huge images - using the standard Drupal interface - was tiring.
    Heaps of respect to Gene, our brand-new webmaster and his newfound ability for finding interesting captions for hundreds of photos, and his patience in believing me when I made excuses for stupid browser behaviour. "Have you tried a hard refresh? I just changed the CSS again!"

Tricks & Unique stuff

Throughout the process, a driving requirement was that pages should fill the screen at all times. No fixed-width here. And have no empty space. And the screen it was being previewed on was a maximized HD widescreen running IE6 (!). Yet I needed it to be actually usable for human beings... and our content editor was on a 1024x768 laptop. Tricky.

Resizable layout

Screen at 1024Screen at 1280

After much work, I got our thumbnail resizing magic happening so that in every resolution, our page layout looks as near as can be imagined to a properly formatted magazine layout. Lots of fun, lots of pain and testing. Fully degradable to no-script, no-flash, no-images (although I gave up on some esoteric effects for IE6).

High-res

The source images for our site are huge, high-quality, and they just kept coming. The file directory is 2.5 GB so far! We needed to have the 'view-full' images really worthwhile, plus the watermark changed a few times, plus there's a possibility we want super-high res zooms for the archive photos, so there's a backup of the full res originals indexed also.
What this means is that backups and transfers ... and especially site synchronization - could take ages. Learning how to do rsync effectively was vital. As was having decent SSH access to my hosts and ssh-key support. This tool let me only update changes that NEEDED to change, not just blindly copying an FTP dir and crossing my fingers. I also often configured it to ignore my derivative images when I didn't really want all of them to update due to a small watermark change just yet.

Clutter

Our margins are variable, our page heights are variable, but we wanted no gaps. With a bit (lot) of jquery magic (spacefiller.module), and some help from dynamicload.module, you will see that the margin gets decorative content inserted according to page size every time.

Transparent clutter element

Our decorative elements are all managed through the CMS (as specially tagged images or custom content types) so they can be managed via normal node creation, and extracted with views.The 'clutter' elements scattered on the 'desk', the topic banners, and the side images are all handled this way. It's a little bit more complex than just custom-coding some CSS or PHP directory scanning, but has paid off and I'm sure will help as the site grows.

The clutter is mostly placed using views blocks (list views), but unthemed a lot to remove 90% of the markup. It was tricky finding exactly the right theme function names to over-ride, but eventually I extracted them from the views theme_wizard and discarded all the wizard-ness.

Yes, half the image elements are Flash! WTF?


Basically, we had them in PNGs for most of the duration, but when I found I was looking at 1.4 MEGs per page load I screamed. Mark declared Flash was better at compression and did full transparency with no problems. I said "Prove it". And he did. It dropped many of our lovely photo-graphics from 200K to 20K, and indeed layered safely, even providing a win on IE6. I conceded, and we got a result that works! It's all non-sematic stuff anyway, so I was content.

Image Borders

image borders

No, that's not done with image processing. Those image corners are layed on with some css (and jquery) magic. Partially as CSS Sprites where I could.
At one point I did get imagecache to do the job, but I had to drop that because it looked silly when scaled. After I got that down, I was (mostly) able to add the right bits to the theme and use this same effect and script throughout the site for taxonomy_image, image_attach, image_gallery, imce and teasers! This did require heavy theme overrides in places, but it's working.

Image Menus

Using a combination of CSS sprites for the hover effect, and some fancy theme code (theme_menu_item, theme_menu_item_link) and a small script that looks for pre-prepared images in a resource directory named according to menu text the main navigation has groovy text. CSS tricks are supposed to ensure that it degrades to plaintext OK, but the jury is still out on exactly the best image-replacement technique.

quick edit

Short-cuts

Just to speed things up, for editors, images get a small [edit] button overlayed so they can hop straight in and update captions and make correctins from where they see them. It looks pretty trivial, but was really neccessary once lightbox started taking over all the clicks. This is done in the theme (because that was quick and easy) adding an node/n/edit link, plus a little style.

Editview mass editor

As mentioned above, I used also editview to display a mini-version of the image node edit forms. This was handy for some bulk updates.

CSS

Early on in the last remake I tried using a reset.css to see if that would reduce my cross-browser pains. It did and it didn't. I'm not sure I'd use it again.

In development, I tried breaking off several of the tasks into their own stylesheets, often custom ones named after the module it was to affect. That was mostly a win, but got tricky sometimes when deciding where some directives should go.
It did make some of the tasks easier, eg all the menu theming in menu.css, and the gallery frames in image_gallery.css. More formality, slightly more complexity, possibly better maintainability, and certainly better for change control.

The theme you see here (vintagedesk) is based on almost no available theme I can remember. I tried zen in one iteration, but it just broke a lot of the methods I was used to from Drupal theming. I think this was started from cleanslate or something similar, as that far down the track I knew I had to do 90% from scratch.

No, it will not be released. I hope you understand :-)

A taxonomy_overview page

Lessons Learnt

Stress-test

I proposed a quiet launch to give the search engines time to seed before announcing the site. I went live mid-week and started submitting and remote testing. A friend of a friend got a link, posted it in a forum, and it entered the blogosphere.
The next morning, precisely when google decided to start downloading my huge images, and some dweeb decided to run a bad-behaviour HTTrack mirror on all of out 2.5 GB, people started finding it, and not getting in because the server was unhappy. Those pages are heavy, I know, but it seems it was the sheer bulk of the images and videos that took it down to a crawl. Having a mega-connection actually worked against us, as the suction from the available pipe was bigger than the Apache server could keep up with. Or something like that. Drupal/PHP/SQL performance was also probably doing the first cache build of thousands of pages of requests, as I'd dropped the cache at transfer time, but not spidered it myself. So that did slow it down too.

I blocked the HTTrack robot, but our host got a scare. Thankfully they immediately enacted a couple of Apache throttles, and by close of day we had co-ordinated a transfer to a dedicated server (it was taking down its neighbours). By that time, pretty much, the bot-fight had settled down, but it was a non-optimal user experience for those that got the leaked URL. Of course those folk were some of the inner circle, so that was a bad look.


Basically, by the time everything else was done, I didn't have a contingency or scaling plan in place. :-(
Partially because the scope of the site had changed radically into a media server from the brochureware we had planned, and I'd not re-evaluated requirements since.

Quick tips:

  • Make that redirect from www.example.com to example.com early - before you open the site up to spiders or start taking logs. Or else referrers will fill up with crap
  • Do warn your hosting company that the site you are going to make live is going to be 100x bigger than what you told them it was going to be when you signed up for the contract.
  • Figure out a scalable method of storing images and files as soon as you can. Just dropping them all in one big folder just doesn't work beyond a few hundred. User quotas were not answer for me. The worst bit was breaking everything (all embedded image references) and having already-proofed pages fall apart following this global change. You need some serious fu to fix that in a hurry.
  • (Something I didn't do) Figure in some process flow for checkin, proofing, sign-off, design review of your pages. I know there's a few taskflow modules around, but I never found the right time to turn it on and learn. I enabled 'revisions' at the half-way point when our editors multiplied, but never really made use of them.
  • Check your printable pages each time the design changes seriously. I've been pretty good about this, but not exhaustive. And basically it was mostly turning things OFF in print.css. Given a spare day and lots of scrap paper I'd like to start ADDING some print-only effects like a B/W letterhead. If you don't already know it, good printable page breaks in browsers are still freaking hard to do right, even with the available CSS 'hints'.

Disclaimer

Yes, you are now free to poke around the code and tell me what I could have done better. The CSS (cached version) is probably not much fun to look at, and I know there's some confusion in there, but we had many revisions and quirks layered on, so not much of it can be held up as a shining example to work from. You are better off using firebug.

I know it's graphic/pageload heavy the first visit, but CSS/style caching makes up for that pretty quick. I was constantly warning about page sizes, but the mandate was simply "Don't compromise just for old machines, this is a design for next years computers". So there!

In the pipeline is probably some performance optimization (which I've done before with APC) and improvements on that side (I got bitten, as described above), and there's a small list of 'phase 2' tweaks yet to make, like RSS promotion and ongoing newsletter-style archives.

Dan Morrison, (dman) @ Gomi

Comments

JohnForsythe’s picture

This site looks amazing... I love photo-realistic designs. And it's a liquid/resizable layout, too!

Really awesome work, congrats.

PS: For CSS resets, I prefer Yahoo's version: http://developer.yahoo.com/yui/reset/

I use a modified copy for most of my designs now.

--
John Forsythe
Need reliable Drupal hosting?

dman’s picture

The resizing is the "fun" part. Over transparent textures yet. Optimized for 1024 to, um, 1800+
and OK with scroll to 640 :-} sorta. Still not recommended for hand-helds tho - even if they could do the bandwidth.
You'll see the embedded images scale as much as they can. Interesting exercise that one :-)
I may revisit resets, but it was a new way of working for me. And the Eric Meyer one I tried was a bit vicious to 'normal' styles like, um, bold so I lost a little faith there.

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

Haza’s picture

Wow, amazing work on theme and modules. It really look beautiful !

(unuseful comment, but i had to say it)

---
Web-42, a french try...

Fred125’s picture

very beauty appearance for this site

styro’s picture

Nice write up Dan

This post probably had more effort go into it than went into the actual building of most of the other sites being showcased in the forum :)

I'll have to sit down and read through properly later...

--
Anton
New to Drupal? | Troubleshooting FAQ
Example knowledge base built with Drupal

afandyag’s picture

agree with you
sit down and read, then try :)

dman’s picture

Hey, after a year of development, it deserved a serious wrap-up! I know it's verbose, but that's what happens when I attempt documentation :0}
Also, as a team of one I guess I didn't get enough of a chance to bounce off other folk that would appreciate the code tricks. So it's out there for you guys instead. Lucky for some?

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

styro’s picture

Also, as a team of one I guess I didn't get enough of a chance to bounce off other folk that would appreciate the code tricks.

Yeah, I know what that feels like.

Hopefully I'll get the chance to pick your brains at a future Welly PHP meetup (I've been too busy the last couple of months).

--
Anton
New to Drupal? | Troubleshooting FAQ
Example knowledge base built with Drupal

robertDouglass’s picture

Great work. Thanks for taking the time to share so much of the insider knowledge that goes into building a great site.

- Robert Douglass

-----
my Drupal book

my Drupal book | Twitter | Director, Product Operations Commerce Guys

Kuldip Gohil’s picture

Classic site..
Good Work........

--
Kuldip Gohil

Jonny St.’s picture

This interesting story. Nice work.

__
I wont all.

Ujval Shah’s picture

awesome !!!!!!!!!!!!!!!!!

hypotheek’s picture

Yeah, really great!

masterkreed’s picture

thats fantastic!
this work is very great

no longer active 6’s picture

Hi Dan,

This is a great site. I really liked this. A fabulous work of drupal and css.
Even the build process you wrote is really detailed.

Thanks,
Rakshit Maniar
(Sr. Software Engineer)
Gloscon Solutions Inc
Open Source Web 2.0 Development Company
8988 Lindsay Pl, Surrey, BC, V3V 6E3
T : 604.630.4292 (Canada)
T : 323.319.3609 (US)
F : 323.337.8330
Gloscon: Email : sales
@ | Gloscon.com | Portfolio | Youtube | Drupal SEO | Drupal Themes
Training/Incubation/Dedicated Resources : OpenKick.com

eorr’s picture

dman’s picture

Drupalmuseum?
Wow, I'd not seen that before. Very nice.
It's certainly in good company, by the looks of things! Thanks!

[edit] Oh man, there's some nice content in there! Thankyou x2

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

Ujval Shah’s picture

really you did the job, very good collection.

Ujval Shah’s picture

i have list of few drupal site having awesome layout.

joep.hendrix’s picture

Very well done and thanks for the writeup.

-----------------------------------------
Joep
CompuBase, Drupal websites and design

-----------------------------------------
Joep
CompuBase, Dutch Drupal full service agency

aterchin’s picture

Probably one of the best looking designs i've ever seen. Well done, dman and crew.

Boris Mann’s picture

Great work as always, dman. On sitesynch.module, it sounds very similar to http://drupal.org/project/deploy -- I would consider joining forces with heyrocker on that...

--
The future is Bryght at Raincity Studios

dman’s picture

Good suggestion. I'd not seen that.
When I started this tool (back in 4.7) I referred somewhat to backup.module, and hoped to find ways to integrate with that, but we never got a common API together.

deployment.module certainly seems do be doing similar things (and more). It's got a heck of a lot of moving parts! I'll try to see if it can work with my workflow sometime. Auditing? deployment profiles? Wow! All I was doing was snapshotting and replacing!

Sounds like he got on top of the x-site form submission challenge under FAPI also. My (pre-FAPI) solution was to transparently switch the form action to the target site and press go :-)

And, FWIW, I pushed my authentication through as XMLRPC parameters. Which was, admittedly, not audited for security exploits - the main reason I never released the code! No clue how that API key thing works. It's probably better.

Judging by the comments, he also quickly encountered the same problems I did re id synchronization when trying to do incremental updates of different parts of the database ... like Node content :-)

Yup.
Maybe I'll just keep mine as a site freeze and restore util!

Thanks for the kind words!

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

gmclelland’s picture

Have you tried the Pathologic Module as an alternative to your Path Finder Module?

I think it might perform the same thing.

Great write up by the way.

-Glenn

dman’s picture

I did see pathologic appear (and the original forum discussion that sparked it) and was going to chip in with what I'd done about then. Neither my nor their ideas were very firm at the time IIRC.
At that point it appeared to be coming from the opposite direction to mine (but towards the same destination)
The current write-up already looks much more mature than its first steps, So I will certainly give it a review when I have time. My own pathfinder module is pretty damn small, and, as I said, has some code that was pretty specific to my task at the time.

The sig-diff from what I can see is between an input filter (more properly termed an output filter ;-) and my approach which was to stop inappropriate stuff from entering the system, rather than fixing it each time it went out (ambulance/cliff scenario).
Because (with an eye on migration issues) I didn't want my live site to be supporting legacy patches that were the fault of the dev site.

But yeah, it basically looks to solve the same problem. Could probably be combined.

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

Garrett Albright’s picture

I'm the developer of Pathologic. From the description of Pathfinder, it looks like the most significant difference between Pathfinder and Pathologic is that Pathologic does not modify node content before it is saved to the database; it only functions as the node is rendered. It would have been a lot easier if I could mangle paths before they are saved to the DB, but the immensely invaluable Pro Drupal Development stresses that input filters should only work on content as it is rendered, so I stuck to that.

I encourage those who were intrigued by the description of Pathfinder above to give Pathologic a try -- I'm really proud of what it has become, and I think many others will find it useful as well.

dman’s picture

Well, as Granny Weatherwax would say: That's what rules are for... to make you think really hard before you break 'em.

The common expected behaviour of filters is just that - an effect that gets applied to the source, which stays the same.

Me, I see hard-URL-references to the dev site as an error, so this utility is just a bit of self-correcting validation. Broken links in my source code are just that - broken, so If I'm gonna fix them I should make it permanent.

It's like a spell-checker. Is fixing spelling errors something the editor should do before saving, or should the renderer correct it on-the-fly when the document is read?

Still, there's value is many of the other repairs you can do at output time. :-)
I think a combination of these approaches is optimal.

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

drubeedoo’s picture

@Garrett, please consider Dan's point as a valid option to enhance your module, as leaving broken links in the source is not necessarily a selling point for your module. In other words, let the user choose between render/permanent in the module setup. The two of you should really get together on this. ;-)

@Dan, great site and fantastic write-up. Thank you!

dman’s picture

Minor errata:
It wasn't image_attach I modified to re-use existing images (as claimed in the image galleries section)
It was taxonomy_image which I modded to COPY image_attach functionality. That was my confusion. :-)

.dan.

spreto’s picture

The design is fantastic. I am in awe of how you managed to scale the images and never leave any white space. This site must have required so much work!

dman’s picture

Interestingly (or not), I first used that technique in (IIRC) Netscape 3. Without captions and corners of course, but you could still set width=50% and see what happens.
In those days, of course we had no css, so no min-width, no max-width, not even floated blocks (although images and tables could be aligned somewhat). No DOM rules either, so 50% generally meant 50% of the whole page width, even if nested.

However, PCs didn't do scale-smoothing the way they can now, so you'd usually get some very chunky pixels, and I think IE4 or something came out and didn't scale proportionately - I got 30%wide by n high all the time, where n was its natural full height. Sux, so I stopped experimenting.

Resolution was 640x480, with a few high-tech folk running 800x600, so pixelation be damned, really ;-)

This year I put some hard CSS-fu against it and achieved an effect I'd never seen before. It's thrilled a few developers/designers so far. Maybe I'll do a write-up and get it named as a technique (like the "holly hack", or "sliding doors") and become famous on the interweb :-) Hm. The "dman squeeze"?
Yes, Maybe I can even Module it. ... although it's more of a design principle than a plug-in effect.
Have you noticed it also applies to the full-size high-res embedded images too?

It WAS fully IE6 compatible until I was required to stick a max-width on it to prohibit any pixelation ever (on Maxi-width HD screens). I didn't want to scale up the thumbs filesize just for that limited audience (a useless pageload bump for the rest of the world), so a week before launch just turned it off for IE6 and left them static. In sane screen-sizes, I'd be happy to use it in IE6 also. Well, not happy exactly, but you know what I mean.

.dan.

protoplasm’s picture

It really is an amazing site. Thanks for sharing all this information!

protoplasm

hass’s picture

I'm very interested to give this module a try... doesn't have someone else written such a filter module yet?

dman’s picture

I would have thought so, and did go looking.
I could have used an arbitrary text/token replacement process like reptag.module to sorta get the job done, but I didn't see one that was dedicated to just making cross-references and hotlinks.

As I said, it seems to be working, but already it's bitten me twice in being a bit too keen with its links (first they started coming in teasers, which I didn't want, then I had to adjust the regexp a bit to stop them appearing inside tags and tag titles :-})

The concept as a whole needs a bit of thrashing out and could mess up peoples sites if used rashly - which is why I'm hesitant to release it.

.dan.

swentel’s picture

Greate write-up and very glad to see that you like my piclens module :)

rouand’s picture

No one can imagine at first glance that the site was built around drupal, it is amazing theming and module customising project. I hope my sites will look as good as this one.I think you should get a Master's Degree at least for such project.... if there is any degrees for Drupalists.

SocialNicheGuru’s picture

nice work all around

Chris

http://SocialNicheGuru.com
Delivering inSITE(TM), we empower you to deliver the right product and the right message to the right NICHE at the right time across all product, marketing, and sales channels.

mrgoltra’s picture

Congratulations.... simply wonderful....

physiotek’s picture

WOW! awesome!!
i love the way the pictures auto-adjust to the width
so many details.
really awesome!

klinsek’s picture

Thank you so much for taking the time to put this together for the community and readers.

I found it very informative. It is always interesting to see the way other people approach the challenges of design and development. All your hard work shows in a very fluid and dynamic site presentation.

dman’s picture

Thanks everyone for the feedback. I'm glad you liked the show & tell.
It is interesting to see how other folk approach development, so I tried to give as much "why" as "how". Of course there's many more layers of detail behind some of the decisions, but thankfully not much that I'd call a compromise or 'beyond my control' so no excuses :-)

Yeah, there was a lot of work at some stages, but oddly there's not any serious architecture issues there. Normally I do information-driven sites, so although I tried to engineer in a bit of knowledge management and super-structure .. basically it's a pretty shallow site organisation-wise. But I have got the pieces in place for scaling and searching later.

.dan.

mightypile’s picture

I've done only very amateurish module development as needed, and never felt my efforts mature enough for sharing with the world. I was intrigued by so much of what you've done. I just finished reading and I now have 7 tabs in Firefox to read through from all of my "open in new tab" action to delve deeper. Thanks for the information and the inspiration to take my efforts further.
-Mike

NicolasH’s picture

Great case for the fact that there is still a place for heavy artistic themes with lots of layers and textures - if done for the right reasons and followed through properly. Yes, this would have been unusable a few years ago, but now that the tubes are bit wider for most of us, why not indulge a little?

Also some pretty bold moves in regards to the usage of Flash - but again, the particular use-case makes perfect sense and is implemented for all the right reasons. One immediate question: Why did you not wrap the other image treatments, such as the photo album corners, also into the Flash approach, but instead chose jQuery? Simply a case of how it turned out or a technical reason?

The content is amazing! I can imagine it would have eased the pain a little at times working with this stuff ;)

Do you know anything about the copyright arrangements? Especially in regards to the advertisements...is that all public domain by now? Given that most of those large German industrials would have not simply disappeared, but merged into other ones would make clearing a nightmare I'd imagine.

What I love about these write-ups are all those "damn, I know what THAT's like"-moments...a had quite a few of those reading through this great example.

ourbrisbane.com

dman’s picture

Although I tolerate Flash for non-semantic stuff, I don't see the need to go overboard.
Partly it was a time thing. We spent ages getting a good x-browser result layering corners on with CSS (and then with JQuery). And it got to work just fine.

The idea to switch to Flash in places was simply size-driven. File size was not a problem for our corners, so no win there.

Also, once I started embedding multiple Flash objects, I saw noticable performance bumps. At first the three sliding elements at top left, top middle, top center were there in three CSS regions (floating overlays worked fine) but stepping back I saw we had like 6 or 7 separate Flash objects (written on-the-fly using x-browser activation script) in their own Virtual Machine initializing each pageload.
This (plus the other onload events like the lightbox init) caused visible page freezes on anything but the fastest platforms.

I can't see the advantage in using Flash for the image corners. Beyond the initialization overhead, the designer would have had to integrate with the lightbox trigger or come up with a total lightbox zoom alternative.
And it would have interfered with right-clicks.
And I'd also lose totally on the resize effect. Embedded objects don't play rescale too well. It's sorta possible but we'd have to capture events all over the place. Not a go.
And, of course, overall I don't like tying content up in a Flash interface. I suppose we could extend the jquery to find embedded img elements and replace them entirely with some magic object passing the img src to it as an argument, but seriously!?

I'm sure the archive posters are long out of copyright. The heritage trust behind the collection owns the actual items, and we of course have full clearance for use of the original images themselves. The watermarks are not exactly claiming ownership of the art, just about preventing them being stolen and republished without attribution. A huge amount of effort has gone into the scanning and preparation. And somewhere behind the scenes there are huge copyright lawyers (much more experienced in IP than you would imagine ;-) so I'm sure it's all on the up and up.

.dan.

dgorton’s picture

Great work & great site. Excellent job sharing back with the community!

Drew Gorton
Gorton Studios

themegarden.org’s picture

You have created great site with nice designed drupal theme.
But, you have created really amazing article on building site.
Thanks.
---
Drupal Theme Garden

mokargas’s picture

Fantastic Drupal site, I love that lighbox2 transition and enlarge function. The design is very appropriate for the site and I get a feeling of the era from it. I wouldn't worry about heavy pages, to be honest this is the 21st century and I left my 28.8k modem behind a long time ago. I'll be checking out those contrib modules of yours.

Excellent work!

markgriffith’s picture

Looks lovely.

I'm still struggling with my very first Drupal site, so I guess I'll buy a book.

Encourages someone like me though.

Mark

oyunindir’s picture

Best Regards

Looks lovely.

I'm still struggling with my very first Drupal site, so I guess I'll buy a book.

Encourages someone like me though.

oyun indir

jeannea’s picture

I think you are so amazing! I wish I could make something like that in the near future! good luck to your next project!

NicolasH’s picture

...and I know this discussion is in a million places here on drupal.org, but I'm interested in your particular choice.

You glued imagecache and image together, while imagefield would have worked out of the box. The reason seemed that you wanted an image to be a node. If you would have created a new CCK type 'my_image' and would have added an imagefield and whatever else you want, could you have achieved the same thing?

Also, you mentioned that you "tagged-on" CCK late in the process for some extra fields. I never considered that, sounds really interesting - a custom node type with a bit of CCK. Can you explain that a bit more?

Many thanks...

ourbrisbane.com

dman’s picture

Yeah, I could have created an imitation 'image' node by cobbling one together in CCK, but I didn't see any advantage in that over the normal version.
From what I knew, imagefield didn't have the imagecache integration last year either, and it was incompatable with image.module so I couldn't combine them. I wasn't aware it supported size-derivatives in the same easy way either? I'd have to do something extra to emulate image_gallery and then tweak it again for lightbox integration, taxonomy_image, image_attach (re-used) etc, whereas they were already there and published for image.module. And I'd end up with something totally unique to my site and unmaintainable. Image.module is simply simpler and more ubiquitous. Oh, and then there is image_import - which I'd have to re-write from scratch for my new made-up format! How many reasons do we need?
imagefield has probably got a bit smoother since, AFAIK, (some of those features may now exist in one way or another) but it's out on a bit of a limb when compared with existing popular support.

There are no custom node types. The information architecture is very shallow. I didn't have any need to complicate the core CMS.
I didn't want or need CCK until the last - when I just added a textfield to the image.module image node. No biggie, I probably didn't even need to mention it in the write-up.
CCK nowadays just enhances ANY content types with fields - core or module-defined - you no longer have to define it explicitly for custom types, you know?

Page, Story, Image, [Image Gallery], [Topic Overview]
- that's it. Not even any public Views (apart from some built-in to image_gallery)

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

NicolasH’s picture

As in providing out of the box imagecache support for quite a long time as well as that being carried through to views. But really, there's no debate if you're familiar with one approach and know it'll work for your case.

I just like to read real world scenarios to get many angles.

Cheers

ourbrisbane.com

Sree’s picture

awesome work & great write up too ....

-- Sree --
IRC Nick: sreeveturi

miadeos’s picture

Really nice work

http://miadeo.com

ass45sin’s picture

The site is amazing with all the attention to details. Your verbose writeup is also something else. It's already in my scrapbook. Thank you very much for sharing it here. And congratulations to a work well done. And.. er.. Drupal Rocks.. ^_________^

Ed

hadingrh’s picture

I'm impressed, really am
I like your gallery. A lot of image but it load fast.

Bandung Tourism

ramnam’s picture

I ended up spending a lot of time on the site. The short clips are wonderful and the pics load up very quickly. And your article was a great read. Congrats Dan!

Bindhyeswari Furniture, Nepal

josephgut’s picture

Just wanted to say that you've done a tremendous service to the Drupal community by publishing this tome of information. It is extremely helpful! Thank you for being so considerate to all of us - this will help many people move to the next level in their Drupal skills and knowledge.

Sincerely,
Joseph

Free antique, collectible, and art appraisals
http://www.instappraisal.com

dman’s picture

Thanks for all the feedback folks! It's much appreciated.
For those that are appreciating the big writeup, I hope you didn't miss The PopSci case study earlier this year. That one is a great read that makes even my story look pretty lightweight!

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

webdev2’s picture

Site looks terrific and one of the best descriptions/How-Tos I've read - great job.

32i’s picture

http://thevintageaviator.co.nz/projects/reproduction-guns/gun-gallery
look at bottom right corner, alot of rss feeds here

luco’s picture

this article really did the trick for me. thank you so much for sharing your thoughts and modules.

_________________________

"There is no off position on the genius switch."
- David Letterman

socceruci’s picture

This is one cool site. I love the stemless edges, I love the ordered loading times, I love the graphics, and everyone loves the screen size scaling.

So I am assuming you put the site up here to help others AND get advice but so I noticed some issues from a business outlook. What is the purpose of this site? To help vintage aviator get more business through the web? If it is to have a site that people go ooohh and aahh to then you did great. But if you are trying to get traffic and business there are some pivotal things.

SEO: you need to have a very specific targeted message on your meta tags. You need internal linking that promotes all that wonderful content you have. There is more if you are still working on the project...

Selling the company: The About Us page is the only place you talk about the "business" really, everything else could be considered a distraction. I am sure people looking for work done on old planes are looking for quality, experience, turn around times and price. I hate to say it but a much simpler site might have done this better for them. So that homepage has to do more than awe, it needs to funnel the customer in the right direction.

Now I could be wrong on some of this. Maybe they have too many people calling and only want people really serious to call, I do not know these things. But what I do know is if they want more business from the web a few of the above things will help. Thanks again for the awesome work up.

dman’s picture

Yes, thanks, I am soliciting constructive criticism. It's appreciated.
You've certainly noticed something about the site I'm aware of. It currently serves no commercial purpose!

To help vintage aviator get more business through the web? If it is to have a site that people go ooohh and aahh to then you did great.

It's number 2. :-B

Or more accurately, the guys that did these projects are in a position to share the resources and stories that they think others will find interesting, so they do. It's more like a hobby site with a budget. They are passionate about the craft they do, and the site is just a "Hello world"!
Target audience is hobbiests and other industry folk, including internal stakeholders.

And it's to show off a bit too :-)
No, they are not (at the moment) actively soliciting business from the web. And yes, they are actively discouraging 'casual' enquiries. :-}
At one point the FAQ boiled down to: "Can I have a workshop tour? : No"
Serious inquiries will have to be serious indeed - those in the know are probably able to count on one hand (or less) the workshops capable of attempting these jobs worldwide. So it's a portfolio for them as well. Nothing wrong with a bit of buzz, even if they are not taking contracts off the street right now. As you may tell from the writing in the stories - the teams choose their projects, not the other way around.

There is effectively no SEO beyond the best kind of all - making valid content that people want to look at. If we redefined the purpose of the site (which could happen) then the audience targeting, and especially the front page, may be revisited. There actually are a few business objectives in the wind, but not much more I can discuss.
I think the slogan is performing pretty well in search engine listings, but I'll look the metas again.

.dan.
How to troubleshoot Drupal | http://www.coders.co.nz/

socialtalker’s picture

you truly are "d-man" !
great site, i dream of having talent and skills like you.

protostos’s picture

Excellent! Your work is like an artist.

internal’s picture

It seems it's your own private module. All I can find here is the Flash gallery module using simpleviewer. I like your flowview very much, what is the relation with piclens.module? Thanks.

Designers.pro’s picture

This is awesome design!
Also, thanks for very informative post.

www.designers.pro
"Design Professionals Community"

gmclelland’s picture

Any chance you would release the gallery_attach.module? Would love to see it on Drupal.org.

dman’s picture

Actually, I was tuning it up for release a short while back.
It's all good for 5, but I thought I'd do a D6 version to go alongside it to release together ... and encountered headaches with the new CCK-widget API. (It adds the 'gallery' as a CCK select field)
Um.
I'll see what state the code is in and stick it on the TO-DO list.
Many folk are turning away from image.module and image_gallery.module in favour of image_field or something, I don't know why. So my approach is a little old-school now ... but it's damn simpler!

.dan.
if you are asking a question you think should be documented, please provide a link to the handbook where you think the answer should be found.
| http://www.coders.co.nz/ |

PetarB’s picture

Wow. This is such an achievement. I wish we had clients with the stamina and vision to do something like this. You really did a marvellous job.

enviarflores’s picture

Incredible high quality images and fun to see. I think my grandfather clicked trough the whole website hehe, good old times for my old man.

nice work