Subscribe to CiviCRM Blog feed
Posts related to Drupal
Updated: 1 hour 15 min ago

Load test Drupal and CiviCRM with LoadImpact

June 10, 2015 at 10:28am

This has been my approach (together with CiviCoop) to load test a big site with CiviCRM where most visitors where expected to login.
Let me know if you would agree with this approach or if you have a better alternative.

Every big drupal site needs load testing before going live.

These are the key questions you should have answered in the final stages before deployment:

  • How does your infrastructure handle the expected amount of visitors?
  • how does it perform with maximum amount of visitors?
  • and at what amount of visitors does it start to crumble?

For anonymous load testing there are a number of tools available.
For logged in users there are not so many available.

But what about the sites where logging in is secured using unique tokens per user visit like drupal?

How do you load test those?

The problem is that you often can record or script what you need to post during login.

But sites like drupal secure their login pages with an unique token so you do not know what you will need to post beforehand.

With that problem is solvable.

LoadImpact automates load testing and gives graphs like:

Example of LoadImpact graphs

To setup a load test on Load Impact for logged in users you can do:

Step 1: Record one or more user scenario's with the Load Impact crome plugin:

Step 2: Export user scenario to Load Impact.

Step 3: Look into the generated LUA code and find the GET request to the login page.

Step 4: Change it so the form token is gathered and placed in variable:

For example:

http.page_start("Page 1")
local pages = http.request_batch({
    {"GET", "", response_body_bytes=10240}

local body = pages[1]['body']
local token = string.match(body, 'input type="hidden" name="form_build_id" value="(.-)"')

Step 5: find the POST request to the drupal login page and change the "form_build_id" with the token value.

if token ~= nil then
    {"POST", "", headers={["Content-Type"]="application/x-www-form-urlencoded"}, data="form_build_id=" .. token .. "&form_id=user_login&name=<username>op=Log%20in&pass=<password>", auto_decompress=true}
  log.error("failed to find token" .. body .. "");

And you're done. Now load tests can be performed with thousands of concurrent logged in users on your drupal site.

If your user scenario contains other form submissions you can repeat this for the other forms as well.

Using CiviCRM as an example: someting similar is needed if CiviCRM searches are performed.

CiviCRM adds a session dependent qfKey to every search. Without the right qfKey a search will not be executed properly, harming the load test.

To solve this you have to execute the following steps in the Load Impact user scenario.

Step 1: Find the GET page for the search and place the qfKey in a variable

local pages = http.request_batch({
    {"GET", "", response_body_bytes=102400}

local body = pages[1]['body']
local token = string.match(body, 'input type="hidden" name="qfKey" value="(.-)"')

Step 2: find the POST request to the search page and replace the qfKey with the token

if token ~= nil then
http.page_start("Page 5")
    {"POST", "", headers={["Content-Type"]="application/x-www-form-urlencoded"}, data="_qf_Basic_refresh=Search&_qf_default=Basic%3Arefresh&contact_type=&" .. token .. "&sort_name=&tag=", auto_decompress=true}

http.page_end("Page 5")
  log.error("failed to find token" .. body .. "");

And you can also do proper CiviCRM searches in your Load Impact user scenario and load test your Drupal+CiviCRM site before deployment.

Originally posted on


Categories: Planet Drupal

16-19 July 2015: NYC Drupal Camp, Aegir Summit and CiviCRM turn-key hosting

June 5, 2015 at 1:44pm

For those of you in the New York City area, 16-19 July 2015 is NYC Drupal Camp (pronounced "nice camp"), an annual grassroots non-profit conference run by volunteers. The event covers a broad range of topics related to Drupal. As part of the camp, the developers of the Aegir hosting system have organised the first Aegir Summit, 16-18 July.

For those not familiar with Aegir: it is a control panel based on Drupal and Drush to help automate the installation of Drupal, typically in a multi-site architecture (1 code base, many independant sites). With the provision_civicrm module, Aegir can also automate the installation of CiviCRM. This means that with a few clicks, you can create a new database, install Drupal and CiviCRM, configure the web server and optionally manage the SSL certificate. It also helps to automate other tasks, such as backups, upgrades and cloning. Need to create a new testing site for a client? Two clicks and it's ready. If you often create the same types of sites: create a model site, then clone it everytime you need a new instance.

There are also plans to support WordPress in Aegir. I have been working on a prototype that uses the command line tool wp-cli instead of Drush. If you would like to try it, please keep it mind that it is highly experimental and requires patching Aegir 3 (which is still in beta, although there are Debian/Ubuntu packages). The code is available here: hosting_wordpress (please read the 'Readme' file for installation notes).

If you are into Docker and other types of farming, you might find the Aegir Summit interesting as well. There has been a lot of talk about moving Aegir to a more Docker/container-friendly architecture in the next phase, for example.

Long story short: if you are in the NYC area, it would be great to see you there. If you cannot make it and you are interested in any of the above, feel free to leave a comment on this blog post or contact me by e-mail: mathieu at

If you are a CiviCRM hosting provider and you would like to provide a self-serve online form so that your future users can test your services and create a new CiviCRM instance, that's possible too. Hopefully we will have a demo on time for the Aegir Summit, but in the mean time, I will leave the following teaser below (and yes, the sign-up form is a CiviCRM form!) :-)

Categories: Planet Drupal

Give CiviCRM on Drupal 8 a test out

May 27, 2015 at 9:17am

Thanks to generous contributions from Fuzion and Beat Schnyder from basx GmbH, I was able to put in a few days work over the last month into the Drupal 8 integration module, allowing Drupal 8 and CiviCRM to interoperate.

I had done a large chunk of work of the initial work for this during Google Summer of Code last year but as Drupal 8 has been under heavy development, so much had changed that the module needed to be carefully picked over to bring back into a working state. Over the several days of funding I had, I managed to get both the module and install working against the latest Drupal 8 beta (at the time, beta-9 but now 10 too) and Civicrm 4.6.3 in time for the Denver CiviCon sprint.

You can check out the current state of things (along with all the bugs and so on) here: (login: demo password: demo). If you would like to explore deeper, such as testing how it works with Views, and later on with other Drupal goodies that aren't yet available in D8, then just ask us for a user account with more permissions. Fuzion is asking for contributions to help support this work - they are encouraged but not compulsory. Funds will go towards Drupal 8 support.

Any bugs you come across, please feel welcome to open an issue on JIRA and assign to me (Torrance).

For the adventurous at heart, I’ve documented the installation procedure, so you can have a go at installing a local copy to play with (and I will update this if it's not clear enough or there are issues in the process).

There’s still plenty of work to do, but the more eyes looking, the shallower the bugs. :)

Categories: Planet Drupal

Easier creation of email newsletters - New version

May 24, 2015 at 2:24pm

When preparing an email newsletter, one part of it that is time consuming is gathering together all the content that is needed. In my experience, virtually all the content already exists elsewhere, such as in the local CMS, in CiviCRM, or on a blog, or some other online source.    So I was thinking how can I make this process easier.  What I did: I created mail merge tokens for CiviCRM that autofill a list of recent blog posts, stories, or any other type of CMS content.  So the end-user sees a list of tokens, one for each content type, each term/category, each aggregator feed,  and for each date range. Such as "Content of type 'blog' created in the last 7 days" .  What is particulary powerful about this approach, is that if you are also using a CMS aggregator (such as the aggregator module in Drupal core) then virually any external RSS feed is turned into CMS content, which is now available as a CiviCRM token. (The original blog post about this extension is at: )

Thanks to community involvement (specifically thanks to, there is a new version of the Content Token extension.  This version now supports Joomla, in addition to Drupal7, Drupal6, and WordPress.

The lastest version is 2.9 and can be downloaded from:

I am looking forward to getting feedback on this.




Categories: Planet Drupal

Using Views Bulk Operations to alter CiviCRM data

April 14, 2015 at 1:10am

We have started a small collection of blogs about using Views, Views Bulk Operations (VBO) and CiviCRM Entities to provide a Views-based interface for bulk updating Civi data.

This opens up a number of possibilities for bulk data updating which may not yet be straightforward in CiviCRM itself.

The recipe is basically

  • install CiviCRM Entity version 2.x & Views Bulk Operations VBO
  • create a View based on the Civi entity in question eg Contact or Participants or Relationships
  • add the VBO field and configure the desired 'bulk operation'

Setting up a View with a bulk operation to modify the entity in question is pretty straightforward. The one 'catch' at this point is that as far as VBO is concerned, we have to enter the CiviCRM 'value' not the 'label', so if you are wanting to update a Participant Status to attended you need to know that is a '2'.

Scenario 1 - updating a Participant Status to 'attended' as people come through the door

So in this case the use case was that an invite had gone out from civi, some had responded and had been set to a Status of 'RSVP-yes', others had not responded and were hence still Status of 'Invited'.

The View then provides the client with a simple interface where they can tick off names as they come to the event, and at the end of the evening, set them all to 'Attended'.

You can see a longer explanation with many screenshots etc here and grab a txt version of the View from here. A screenshot of the way to set up the VBO field is below.

Scenario 2 - setting End Dates (or whatever) on Relationships in bulk

In this situation we needed a bulk updating interface for Relationships so that relationships can be set to inactive along with the End Date.

Eileen has set this up as a Feature you can try out here.

Categories: Planet Drupal