Subscribe to Acquia Developer Center Blog feed
Updated: 15 min 6 sec ago

Staying Ahead on Security: Why Acquia is Disabling PHP 5.3

September 2, 2015 at 2:46pm
Isaac Sukin

The biggest risk for any website is security. Research has shown that the average cost of a data breach is several million dollars. The last two years have seen a number of high-profile security breaches affecting millions of people, including:

Tags: acquia drupal planet
Categories: Planet Drupal

Helping the Drupal Community with Drupal 8 Training

August 31, 2015 at 6:24pm
Kent Gale

Training Acquia employees how to use Drupal 8 had two purposes. First, there was the obvious need for our company – one that specializes in building and delivering a digital experience using Drupal – to get everyone up to speed on the new version.

But we also felt our training sessions had a larger purpose: to inform the larger Drupal community on the ins and outs, not to mention joys and pains, of the updated version. So as we embarked on our long training program of nearly 50 employees, we documented our progress.

Tags: acquia drupal planet
Categories: Planet Drupal

10 Ways Drupal 8 Will Be More Secure

August 27, 2015 at 4:20pm

Security is very hard to bolt on to any software or product after it has been built. Building it into the core of the code helps to avoid mistakes, and thus the upcoming release of Drupal 8 tries to build in more security by default, while still being usable for developers and site builders. This list of 10 security improvements is not exhaustive - some are just a line or two to handle an edge case, and there are others I may have overlooked. I've contributed to a number of these improvements, but they reflect overall the community consensus as well as reactions to problems that required security releases for Drupal core or contributed modules in the past. For each point I've tried to include a link or two, such as the Drupal core change record, a documentation page, or a presentation that provides more information. Some of these may also be possible to back-port to Drupal 7, to benefit you even sooner. A "7.x back-port" link indicates that.

For context on why these 10 improvements are important, I looked at past security advisories (SAs) as well as considering the kind of questions we get here at Acquia from companies considering adopting Drupal. In terms of past SAs, cross-site scripting (XSS) is the most commonly found vulnerability in Drupal core and contributed modules and themes.

  1. Twig templates used for html generation

    This is probably first on the list of anyone you ask about Drupal 8 security. This is also one of the most popular features with themers.



    One security gain from this is that it enforces much stricter separation of business logic and presentation – this makes it easier to validate 3rd party themes or delegate pure presentation work. You can't run SQL queries or access the Drupal API from Twig. 




    

In addition, Drupal 8 enables Twig auto-escaping, which means that any string that has not specifically flagged as safe will be escaped using the PHP function htmlspecialchars() (e.g. the same as Drupal 7 check_plain()). Auto-escaping of variables will prevent many XSS vulnerabilities that are accidentally introduced in custom site themes and custom and contributed modules. That fact is why I ranked this as number one. XSS is the most frequent security vulnerability found in Drupal code. We don't have a lot of hard data, but based on past site audits we generally assume that 90% of site-specific vulnerabilities are in the custom theme.


    To see why themers love Twig, compare the Drupal 7 block.tpl.php code to the Drupal 8 Twig version.

    Drupal 7 block.tpl.php:

    Drupal 7 block.tpl.php

    Drupal 8 block.html.twig:

    Drupal 8 block.html.twig

  2. Removed PHP input filter and the use of PHP as a configuration import format

    OK, maybe this should have been number one. Drupal 8 does not include the PHP input format in core. In addition to encouraging best practices (managing code in a revision control system like git), this means that Drupal no longer makes it trivial to escalate an administrator login to being able to execute arbitrary PHP code or shell commands on the server. 


    For Drupal 7, importing something like a View required importing executable PHP code, and for certain custom block visibility settings, etc. you would need to enter a PHP snippet. These uses of evaluated PHP (exposing possible code execution vulnerabilities) are all gone – see the next point about configuration management.


    Now that we have covered the top two, the rest of the 10 are in rather arbitrary order.

  3. Site configuration exportable, manageable as code, and versionable

    The Configuration Management Initiative (CMI) transformed how Drupal 8 manages things that would have been represented in Drupal 7 as PHP code. Things like Drupal variables or ctools exportables (e.g. exported Views).



    CMI uses YAML as the export and import format and the YAML files can be managed together with your code and checked into a revision control system (like git). 


    Why is this a security enhancement? Well, in addition to removing the use of PHP code as an import format (and hence possible code execution vulnerability), tracking configuration in code makes it much easier to have an auditable history of configuration changes. This will make Drupal more appealing and suitable for enterprises that need strict controls on configuration changes in place. In addition, configuration can be fully tested in development and then exactly replicated to production at the same time as any corresponding code changes (avoiding mistakes during manual configuration).
 Finally, it is possible to completely block configuration changes in production to force deployment of changes as code.


  4. User content entry and filtering improved

    While the integration of a WYSIWYG editor with Drupal core is a big usability improvement, extra care was taken that to mitigate poor practices that adding a WYSIWYG editor encouraged in past Drupal versions. In particular, users with access to the editor were often granted access to the full html text format, which effectively allowed them to execute XSS attacks on any other site user.



    To encourage the best practice of only allowing the use of the filtered HTML format, the Drupal 8 WYSIWYG editor configuration is integrated with the corresponding text filter. When a button is added to the active configuration, the corresponding HTML tag is added to the allowed list for the text filter.


    Drag a new button from the available to enabled section in the editor configuration:

    WYSIWYG editor configuration adding underline button

    The corresponding HTML tag (the U tag) is added to the allowed list:

    U tag is allowed in the filter

    An additional security improvement is that the core text filtering supports limiting users to using only images local to the site which helps prevent cross-site request forgery (CSRF) and other attacks or abuses using images.

  5. Hardened user session and session ID handling

    There are three distinct improvements to session and session cookie handling.

    First, the security of session IDs has been greatly improved against exposure via database backups or SQL injection (7.x back-port ). Previously in Drupal, the session ID is stored and checked directly against the incoming session cookie from the browser. The risk from this is that the value from the database can be used to populate the cookie in the browser and thus assume the session and identity of any user who has a valid session in the database. In Drupal 8, the ID is hashed before storage, which prevents the database value from being used to assume a user's session, but the incoming value from the value is simply hashed in order to verify the value.


    Next, mixed-mode SSL session support was added to core to support sites that, for example, used contributed modules to serve the login page over SSL while other pages unencrypted. You will have to replace the session handling service if you really need this. This encourages serving your entire site over SSL (which is also a search engine ranking boost).



    The final change is that the leading “www.” is no longer stripped from the session cookie domain since that causes the session cookie to be sent to all subdomains (7.x back-port)

  6. Automated CSRF token protection in route definitions

    Links (GET requests) that cause some destructive action or configuration change need to be protected from CSRF, usually with a user-specific token in the query string that is checked before carrying out the action.

    

This change improves the developer experience and security by automating a process frequently forgotten or done incorrectly in contributed modules. In addition, centralizing the code makes it easier to audit and provide test coverage.

    Drupal 8 makes it easy. A developer merely needs to specify that a route (a system path in Drupal 7 terms) require a CSRF token. Here is an example of the YAML route definition for a protected link in Drupal 8 entity.

    entity.shortcut.link_delete_inline:
      path: '/admin/config/user-interface/shortcut/link/{shortcut}/delete-inline'
      defaults:
        _controller: 'Drupal\shortcut\Controller\ShortcutController::deleteShortcutLinkInline'
      requirements:
        _entity_access: 'shortcut.delete'
        _csrf_token: 'TRUE'
    

    Only the one line in the requirements: section needs to be added to protect shortcut deletion from CSRF.

    Shortcut inline delete link and corresponding URL with a token in the query string:

    Drupal page showing shortcut

  7. Trusted host patterns enforced for requests

    Many Drupal sites will respond to a page request using an arbitrary host header sent to the correct IP address. This can lead to cache poisoning, bogus site emails, bogus password recovery links, and other problems with security implications.

    For earlier versions of Drupal, it can be a challenge to correctly configure the webserver for a single site that uses sites/default as its site directory to prevent these host header spoofing attacks. Drupal 8 ships with a simple facility to configure expected host patterns in settings.php and warns you in the site status report if it's not configured.

  8. PDO MySQL limited to executing single statements

    If available, Drupal 8 will set a flag that limits PHP to sending only a single SQL statement at a time when using MySQL. This change would have reduced the severity of SA-CORE-2014-005 (a SQL injection vulnerability that was easily exploited by anonymous users) (7.x back-port)
. Getting this change into Drupal 8 meant I first had to contribute a small upstream change to the PHP language itself, and to the PDO MySQL library that is available in PHP versions 5.5.21 or 5.6.5 and greater.

    There is also a patch in progress to try to enforce this protection regardless of which specific database driver is being used.

  9. Clickjacking protection enabled by default

    A small change, but Drupal 8 sends the X-Frame-Options: SAMEORIGIN header in all responses by default. This header is respected by most browsers and prevents the site from being served inside an iframe on another domain. This blocks so-called click-jacking attacks (e.g. forms or links on the site being presented in a disguised fashion on an attacker's site inside an iframe), as well as blocking the unauthorized re-use of site content via iframes. (7.x back-port).

  10. Core JavaScript API Compatible with CSP

    Support for inline JavaScript was removed from the #attached property in the Drupal render API. In addition, the Drupal javascript settings variables are now added to the page as JSON data and loaded into a variable instead of being rendered as inline JavaScript. This was the last use of inline JavaScript by Drupal 8 core, and means that site builders can much more easily enable a strict content security policy (CSP) – a new web standard for communicating per-site restrictions to browsers and mitigating XSS and other vulnerabilities.

A final note of caution: The substantial code reorganization and refactoring in Drupal 8 as well as the dependence on third party PHP components does present a certain added risk. The code reorganization may have introduced bugs that were missed by the existing core tests. The third party components themselves may have security vulnerabilities that affect Drupal, and at the very least, we need to track and stay up to date with them and fix our integration for any corresponding API changes. In order to try to mitigate the risk, the Drupal Association has been conducting the first Drupal security bug bounty that has been run for any version of Drupal core. This has uncovered several security bugs and means they will be fixed before Drupal 8 is released.

I am excited that we've added more “security by default” to Drupal 8, and I hope you download and try it out so you are ready to start using it for new projects as soon as it's released.

Blog series: Drupal 8Workflow: PublishedFeatured: NoTags: acquia drupal planetSecurityDrupal 8 related: YesAuthor: Peter Wolanin
Categories: Planet Drupal

Securing Your Site’s Communications with SSL

August 27, 2015 at 4:00pm
*/

Securing your siteThese days, using SSL with your website isn’t just a good idea — it’s essentially a requirement. Encrypting the information that’s sent between visitors’ browsers and your website is the new price of doing business.

Fortunately, Acquia has a lot of experience helping to secure websites, and we’ve collected several of these tips into a best practices article on the Acquia Help Center.

Want to know about using Varnish with SSL (and you’re an Acquia Cloud user)?

It’s in there.

Want to ensure that all of your website’s assets are served to visitors using the HTTPS protocol?

It’s in there.

Want to set up your 301 redirects from your HTTP pages to HTTPS?

Surprise! It’s in there.

For more information on how best to use SSL with your website, visit the article and read for yourself.

And for even more information that you can use with your Drupal website, feel free to browse the articles and resources on the Acquia Help Center.

Workflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Lynette Miles
Categories: Planet Drupal

Terrific Tools for Drupal 8 Training

August 26, 2015 at 5:20pm

We couldn’t have trained dozens of Acquia employees on Drupal 8 without using various applications to schedule, measure our progress, and follow communications about the massive undertaking.

Let’s take some time in this, the fifth blog in a series about Drupal 8 instruction, to give a quick overview of the tools we used to train.

We used the collaboration software Confluence because we could set it up quickly. Kent Gale, the co-author of this blog and my partner in the training project, and I would have preferred to have built a tool within Drupal. But we went with Confluence because we were able to deploy it quickly and edit content. It’s where we placed project documentation to enable in-place editing.

Calendars kept in Google Docs spreadsheets accounted for every employee’s time. Every trainee had time blocked off each week for training. It was protected time, so to speak, agreed upon by employee and supervisor – time that was free of any other obligation. Each week, I’d also schedule 30 minutes with each employee – although meetings didn’t last that long – to talk about training. I wanted to see if they faced any barriers or had questions they were too embarrassed to ask in training sessions. We found this hands-on check-in – which was brief but frequent – kept everyone on track.

We also used email and a chatroom to communicate. Initially, email was the preferred route, but as more people enrolled in training, the chatroom was better able to handle the flow of communications. The spontaneity of a chatroom allowed a lot of questions to be answered quickly. It built a resource as well as type of behavior.

On our tracking sheet, we followed the time each employee spent on training and could see how much time was left. We also tracked the lessons they completed and could see if a team fell behind and could quickly address why.

When you’re ready to train, find a tool and use it a lot before starting the program to ensure it will accomplish what you need. Then pick an end date and work backwards. That will tell you how much time you’ll have. From there, you’ll be able to determine if you should compress training to reach a deadline or if you can spread it out over time.

As organized and motivated as we were, and despite starting early, it was still difficult to carve out time for nearly 50 people, and keep our end-date within reach. It was a commitment. So keep that in mind: Your managers and employees have to commit to protecting training time to make it all happen. Otherwise, it’s very easy to let the opportunity slip away.

Blog series: Organizing to Rock with Drupal 8Workflow: PublishedFeatured: NoTags: acquia drupal planetDrupal 8 related: YesAuthor: Thomas Howell
Categories: Planet Drupal

Training Your Team on Drupal 8: Balancing Classrooms and Individuals

August 24, 2015 at 4:03pm

Not every student learns the same way, so teachers consistently have to find a way to instruct a classroom while also reaching students individually.

We were reminded of a teacher’s greatest challenge when we trained dozens of Acquia employees on Drupal 8. As my co-author Kent Gale and I detailed earlier in this series on Drupal 8 instruction, we separated employees into groups of two, with one person having some knowledge of the new platform and the other having no knowledge. Once their instruction ended, they split up and each teamed with two other employees – our version of chromosomal mitosis.

Our approach to training was structured. We had goals to achieve. But we also had to stay flexible throughout. Because experience, knowledge, and skill set differed with each employee, we had to connect with them individually while maintaining the larger class structure.

We had people with deep programming experience. We had middleware folks. We had site builders. We had front-enders. Because of that, the training program had to present a lot of material, but not so much that individuals wouldn’t learn. We trained with the expectation that not everyone would, or even needed to, become experts.

Consider the training of our “explainers,” the employees who explain our products to the public. We had to figure out what they could easily learn in only one- to four-hours of training. They needed to know enough to promote and answer questions about Drupal 8, but didn’t need to know as much as a support person, who received anywhere from 40 to 80 hours of training. Figuring out what the explainers needed to learn took some effort, but there was ample material to help us determine which path to follow.

Speaking of paths, your team doesn’t have to follow ours. Mitosis worked great for us, but it may pose a problem for your program if you have fewer employees, less time to train, or other considerations.

You need to find out what works best and that, as we’ve mentioned, takes time and effort, success and failure. Some employees like to be lone wolves and learn everything on their own, for example, so our process may not work for them.

Tools that track progress will help you ascertain what works and what doesn’t. Every company, no matter how large or small, faces time constraints, so these tools will guide you through the unknowns.

We used training as a key performance indicator (KPI) for employees. Shared ownership in this big Drupal 8 training project made sense if we all had to make a big leap together in understanding the new platform. Sometimes employees will sweep training under the rug because they believe putting out other fires is a priority.

We knew learning Drupal 8 would be a significant commitment; it’s a significant change, after all. But we couldn’t delay training. Drupal 8 was coming out and there was no time for delay. KPIs helped motivate and get everyone on the same page. There was a vested interest in making progress.

Blog series: Organizing to Rock with Drupal 8Workflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: YesAuthor: Thomas Howell
Categories: Planet Drupal

Now or later? Weighing the benefits of early adoption for Drupal 8

August 21, 2015 at 3:36pm

If you’re considering a switch to Drupal 8, why not become an early adopter? Becoming an early adopter has some risks — and Acquia will work with you to mitigate those risks — but it also has huge benefits.

In this post, I want to talk to you about those benefits and also share with you my experience with Examiner and its early adoption of Drupal 7.

If you’re not familiar with it, Examiner is a news company powered by thousands of self-contributing writers. Currently, it’s read by 22 million people a month. But back in 2009, the company was having problems with its ColdFusion CMS, and those problems were hampering its growth.

Examiner decided to move away from the legacy homegrown platform to Drupal. So they acquired NowPublic, a citizen-journalism company I founded, for its Drupal expertise and leadership. That’s how I became the CTO of Examiner (I later joined Acquia in 2012).

Moving ahead, the big question we faced at Examiner was: Do we go with Drupal 6, a stable but mature technology? Or do we take a bold leap and implement the yet-to-be-released Drupal 7? Ultimately, we chose to become early adopters, going with Drupal 7.  

Here are the reasons that powered that decision:

1) You stay in front of the technology wave

While a good product at the time, there was no denying that Drupal 6 was closer to its end of lifecycle, while Drupal 7 was just taking off. We already understood the costs involved in supporting a legacy product. And we knew any extra investment early on would be offset by things like a longer lifecycle.

As a side note, unlike previous versions of the platform, Drupal 8 releases will come out every six months. So if you plan to become an early adopter of Drupal 8, not only are you taking advantage of the latest and greatest today — but you will continually upgrade to the latest features over the lifecycle of the product.

2) You differentiate yourself from the competitors

At Examiner, we wanted to set ourselves apart from the competition and we knew D7 would give us that edge. AOL was starting to invest in Patch at that time. And we felt that if we wanted to grow our audience and draw the best journalism to our site, we needed best-in-breed tools.  

3) You can attract the top talent

Great developers want to be on the cutting edge. Who wouldn’t want to jump on an opportunity to work full-time on their passion and be able to contribute back to Drupal? When we brought in a great platform at Examiner, we attracted the best Drupal developers in the world. Very quickly, we hired 15 of the top 50 developers in the Drupal community.  

4) You have the opportunity to shape your investment

Getting in on Drupal 7 earlier put us in the driver’s seat with the technology. That was important. We weren’t looking to adopt just any set of tools. We wanted an opportunity to shape the next generation of a platform. And we knew Drupal was going to meet our needs better than anything else out there. Plus, it’s a lot less risky than building your own CMS, because you are not going it alone. You’re going in as part of a community.  

5) It forces you to develop best practices

Being an early adopter forces you to use best practices with respect to software development. At Examiner, we were able to participate in the community and contribute code to the platform. So for us, being an early adopter forced us to do things the right way — and that set a standard within the company moving forward.

After a year of work, Examiner moved to a new platform built on Drupal 7. Thanks to Drupal 7, Examiner went from not being able to meet the needs of its users to exceeding them. We were able to deliver new features on a faster cadence. We had the best-in-breed platform, the easiest to use interfaces, and ultimately those features accelerated the growth of the company.

Examiner launched on Drupal 7 six months before the official release of the platform. We started developing on it almost a year and a half before the release. So we were really early adopters. Examiner faced tremendous risks, because at that time, Drupal 7 was nowhere near as put together as Drupal 8 is today, but we still decided to do it — and it paid off.  Today, Examiner is a top 60 website.

Being an early adopter is definitely an investment. It will cost more to be an early adopter of Drupal 8, but as Examiner has demonstrated, those costs are set off by several factors. And if you are concerned about the risks, keep this in mind: more than 400 sites are already running Drupal 8. And Acquia has already announced we are ready to support anyone with Drupal 8.

What are your thoughts? Do you have any experiences on being an early adopter of Drupal 7? And how do you feel about the risks/benefits of being an early adopter for Drupal 8? We’d love to hear back from you and get the conversation going.

Workflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: YesAuthor: Michael Meyers
Categories: Planet Drupal

Five Ways to Leverage Third-Party APIs: The Drupal-Zendesk Integration

August 20, 2015 at 6:36pm

When Acquia’s Global Support Team outgrew their ticketing system in 2013, it was time to make a change. An outdated ticketing system was taxing their team and compromising their ability to support customers. In addition to lacking the core functionality required to meet increasing customer expectations, the third-party vendor lacked visibility and integration with existing systems like JIRA and Toggl, reporting was slow, and SLA was waning.

The Global Support Team decided to look for a new, flexible API that would deliver tight integration with existing systems and generate responsive channels for quick, direct and clear communications. Reporting needed to be real-time and fast, and the customer and agent UX needed to be streamlined. Acquia needed a new system.

In Walks Zendesk

After systematic vendor vetting, Acquia’s Global Support Team quickly determined that Zendesk’s documented API provided the flexibility needed to do things the Acquian way. Zendesk is a customer service platform that provides the ideal framework for an enterprise environment. Zendesk offers an out-of-the-box solution, which provides a front-end customer interface and a back-end agent UX. Instead of just “drinking their own champagne,” Acquia decided to split a bottle with Zendesk’s REST API and develop the front-end of their Acquia Help Center in Drupal.

Drupal-Zendesk Integration

With a Drupal-Zendesk solution, Acquia built a powerful ticket request system that provides unparalleled support to their customers and internal teams. Here are five ways Acquia’s Support team leveraged a third-party API to build a new ticketing system.

1. Using Zendesk’s API to create customer requests in Acquia’s Help Center on Drupal

Acquia needed to migrate nearly 100k pre-existing tickets into Zendesk. This kind of overhaul required some reconciliation. Reorganization consisted of deleting completed tickets, cleaning up the open ticket queue, and configuring data into Zendesk.

The new Acquia Help Center was built using Zendesk’s REST API in Drupal, providing a Customer UX that is easy to navigate. The Agent UX, utilized internally by the support team, is outfitted with all of Zendesk’s built-in functionality. Zendesk also offered Acquia’s Global Support Team the ability to customize their apps to guarantee top performance.

customer UX create ticket .png

2. Additional Info Block Application

The flexibility of the Zendesk Apps Framework allows companies to extend the capabilities of the framework to leverage tickets, users and knowledgebases. Acquia customized their solution with an Additional Info Block Application, embedded in the Agent UX. The info block provides a global and integrated view of the customer.

info block 2.png

The info box displays information such as the product the customer is using, the number of application support tickets their subscription enables them to register, what networks they are connected to, special handling notes and their account management team.

“This heightened customer visibility allows diverse members of Acquia’s Global Support Team to best support the customer’, says Jeannie Finks, Director of Global Support Systems and Programs at Acquia. “This supplementary ticket data is a necessity for our team to provide customers with the personalized assistance they need and now expect”.

3. Time Tracking App

By leveraging the flexibility of the Zendesk Apps Framework, Acquia was able to aggregate all of their systems in one place. Existing systems like JIRA and Toggl are essential to Acquia’s workflow, and needed to remain accessible in the Agent UX. Toggl is a time tracking app that allows you to sync your entries in real time. Toggl’s cloud based framework is Acquia’s default time tracking interface. Acquia’s custom Toggl-Zendesk app pushes ticket time to a central repo of daily agent activity:

toggle time tracker.png

Additionally, Zendesk’s partnership has enhanced the view of the customers through expert reporting. The Zendesk toolkit allowed Acquia to track tickets rolled in by account, customer backlog, and a root cause report. The introduction of expert reporting offers support teams a comprehensive overview of the customer. Real-time reporting provides Acquia’s Support Leadership with the resources needed to proactively identify critical issues and solve them quickly. This Info Block increases customer visibility, allowing Acquia to see what their customer needs, right when they need it.

4. Custom SLA Monitoring and Notification within Zendesk

The ticketing system also monitors the status of tickets based on a customer’s Service Level Agreement. Acquia continues to take advantage of Zendesk’s flexibility by configuring SLA data from a central customer data warehouse. This customization generates alerts that flow into all key communication channels, such as mail and chat. This custom monitoring system notifies teams when SLA expiration time is appended to a ticket, providing support teams with the visibility needed to best assist the customer.

SLA .png
 

5. JIRA and Zendesk Linked Tickets

In addition to Toggl, JIRA is a ticketing system that Acquia’s Global Support Team utilized internally. It was a workflow necessity to have continued access to JIRA, and Zendesk’s robust API enabled Acquia to do so. Acquia further customized their API with a mini app that linked tickets filed in JIRA and Zendesk.

The system scans Zendesk ticket comments, subject, and internal URL fields. After scanning, it will match any Acquia JIRA project keys. The system will then display the JIRA key, subject, status, time created, updated time, reporter and assignee. Comment links can also be added to any JIRA ticket.

“The benefit of these customized applications is that all of Acquia’s support systems are connected in one place”, says Finks. “The convenience of having JIRA, Toggl and a customer info block in the Agent UX relieves the major pain points that were taxing our internal teams. Through our integration with Zendesk, Acquia’s Help Center is able to offer unparalleled global support to customers 24/7”.

The next installment of our series will examine best practices when integrating with a third-party API.

Blog series: Integrating Drupal and ZendeskWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Georgianna Anderson
Categories: Planet Drupal

Avoiding Fuzzy Math When Load Testing

August 18, 2015 at 3:42pm

No one likes fuzzy math. It’s especially problematic when you’re conducting a load test and can’t accurately gauge concurrency.

In this last blog of a series on load testing, here are some tips on how to avoid the fuzzy math that can distort your expectations of how a website will perform.

Shaky math can happen when using Apache JMeter, which is the most commonly used application to load test the performance of an open source site.

JMeter breaks it down into three categories: the number of threads that are happening at once, the ramp-up period from zero requests at a time to the max number, and the number of iterations. This typically is the way to determine expected concurrency: the total number of requests divided by average response times over how long you test.

There’s a problem with this method, though. When a site becomes overwhelmed, the response time actually increases, which means concurrency drops. So with such a test, you’re actually simulating the exact opposite of a normal site and not even close to seeing probable concurrency.

But there is a proper way to solve this, a way to get a true glimpse of possible concurrency – it’s something called throughput shaping. You can find it on jmeter-plugins.org. With this tool, you only have to simply say, “I want a thousand requests at once,” and it’s not only accurate but will save time that’s ordinarily lost on JMeter as you first try to figure out things like how many requests will hit your site at once.

Another speed bump to consider is how difficult it’s becoming to properly determine how many requests hit your site. That’s because not every every bot and not all of the “noise” on the Web crosses the path of Google Analytics and Omniture. Likewise, looking at the number of requests to your Web servers on Acquia Cloud, for example, doesn’t take into account if you’re using a CDM. (But if you are using a CDM, that’s the most likely source where you’ll have an accurate number.)

So here’s the last lesson of this series: Don’t extrapolate results. If you’re testing with 150 connections, don’t assume 300 will be exactly twice the number of resources required. Your test will tell you what 150 did. Load testing should be conducted in an environment that’s exactly the same size as what your production environment will be. Sure, it will cost a bit, but it will tell you how things will actually behave.

If your site faces contention – when many visitors simultaneously compete for your application’s attention – it will perform the exact same way if you have 150, 300 or even 600 connections. But the point is, you don’t actually know that; extrapolating results won’t provide an accurate number. Most often people look at the numbers, testing exactly to the numbers they have today. They’re missing more than just surges.

Consider must-visit websites for special events like sporting events and live award shows. They have one or two huge days of traffic a year, but the rest of the year, they’ll have a totally different number of visitors. So, when load testing, it’s not just a matter of looking at numbers. It’s understanding what those numbers actually mean and recognizing your end goal.

I recommend that once you have numbers, always test about 50% above that. Not necessarily because you’ll see that kind of traffic at launch, but if the site’s successful and growing over time, it’s rare that you’ll take the time to go back and run more load tests. By initially testing well above what you’re expecting, you’ll have a buffer and won’t have to worry about how the site is going to behave six months from now.

Hopefully this series has prompted you to think carefully about the nuances of load testing and will help as you prepare to launch a site. Load testing done right can help achieve optimal site performance. And, as I mentioned earlier in the series, your users define where load testing should take place, so you can’t go wrong. If you have any questions or suggestions, please drop a note in the comment box. Thanks for reading.

Blog series: Load Testing Your Drupal WebsiteWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Erik Webb
Categories: Planet Drupal

Learning Drupal 8: Mitosis Isn't Just for Science

August 17, 2015 at 9:05pm

You probably learned about mitosis in high school science class and have since forgotten all about it. Mitosis is the scientific term for when chromosomes in a cell nucleus separate into two identical sets of chromosomes, each in its own nucleus.

We performed our own type of mitosis when preparing the staff at Acquia for the release of Drupal 8. We divided, conquered and divided all over again – accelerating the pace of training that otherwise would have plodded along and drained internal resources.

This is the third installment in a series on Drupal 8 instruction and it seems appropriate at this point to look at how our spin on mitosis not only helped us stay focused on the ordinary flow of business – employees needed anywhere from 10 to 80 hours of training, depending on their job duties – but also had people benefitting from the power of collaboration.

As we started the training program, we had already read a lot of documents to prepare fro Drupal 8. But as helpful as they were, we knew nothing would beat the advantages gained from hands-on training. It also made sense to train in bite-sized fashion so no one would feel overwhelmed by learning the more than 200 new capabilities of Drupal 8 while also trying to stay on top of regular work.

So we first paired four people into two groups and gave them – and me and my co-author Kent Gale – two months to port a module from to Drupal 7 to Drupal 8. Unfortunately, we didn’t have a thorough, consistent stream of documentation to lean on, so our pairs immediately went off in different directions. At that point, we changed tracks and reorganized groups: teaming one person who knew even a little bit about Drupal 8 with another who knew nothing about it. Right away, there was a constructive back and forth with each pair. Trainees did spend time reading materials on their own, but they clearly benefitted by working together. Between sharing a computer and spending a lot of time together, it became a lot easier for them – and us – to ask the right questions and solve problems.

When that round ended, both members of each team understood Drupal 8, and each then paired with employees who needed instruction, again performing our version of mitosis. Four became eight; eight became 12. We eventually dropped the original four trainees and kept moving along, with a steady rate of 12 trainees. Twelve was the ideal rate of training we could support.

Once we gained steam, we recognized each training session had to last three months rather than two so we could provide 40 hours of instruction to our higher level people, the “code makers” and “fixers.” Three months enabled us to handle vacations and sick days, and gave us a firm idea of the speed we needed to maintain when training.

It goes without saying that when your outfit starts planning training, it’s crucial to determine how quickly you can do it. Only when managers and teams collaborate will you be able to truly learn what can be done.

Blog series: Organizing to Rock 'n' Roll with Drupal 8Workflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: YesAuthor: Thomas Howell
Categories: Planet Drupal

What Drupal 8 Means for Load Testing

August 17, 2015 at 2:30pm

If you’re already working with Drupal 8 (no excuse, now that it's supported on the Acquia Platform), you're probably still exploring what it can do. There are more than 200 new features, so it may take a while. 

But since this is a series about load testing, let's narrow our focus to what Drupal 8 does during load tests. 

You probably know that Drupal can be used as a back-end framework for managing content such text, media, and geospatial data, while AngularJS and ReactJS can be used as the front-end from a JavaScript perspective. Drupal 7 can do this, but it’s a lot easier with Drupal 8.

When load testing, the biggest difference between the versions is that with Drupal 7, you’re expecting pages that come back to HTML as something a Web browser can understand. In Drupal 8, what comes back will be something like JSON or XML, something that a Web browser doesn’t care about as much. Now, this won’t change the performance aspect of a load test, but it will be interesting to see how people adapt to these different patterns as they write tests.

Enhanced caching is another big advancement of Drupal 8. From its first iteration to the seventh version, Drupal was geared out of the box for developers, so nothing was cached. When you were developing a new site or module, everything you did was shown immediately. With Drupal 8, out-of-the-box performance is a lot better, and page caching is great example of that improvement. In Drupal 7, page caching was disabled by default; in Drupal 8, it’s enabled by default. It’s one less thing you have to do by hand. That also means it should catch things that maybe you’ve forgotten about during development. It should catch things that Drupal 7 wouldn’t have caught.

One other key difference is how Drupal 8 integrates with Symphony, and uses what are called service containers. This is the idea that, for example, on a multilingual website, Drupal 8’s service containers handle translation. These service containers can be changed at any time. If you change the language or various menu structures, these get rewritten to disc every time. So when this needs to be rebuilt, it’s one of those locking operations that may have an impact on how the site’s running. Now, it shouldn’t affect performance, but it’s one of the many things our support team has studied and will keep watching as part of Acquia’s review of all the changes in Drupal 8.

Just like a huge and complex puzzle that we got for Christmas and have been assembling for some time now, we know almost everything Drupal 8 can do but we’re still learning little things here and there about how it differs from Drupal 7. The best we can do is continue to understand what these differences are, and know what to look for going ahead.

Blog series: Load Testing Your Drupal WebsiteWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: YesAuthor: Erik Webb
Categories: Planet Drupal

Ask Acquia: Is it worth the effort to hide that my site runs Drupal?

August 14, 2015 at 6:44pm

You might think that hiding the content management system (CMS) your site is running will make it difficult for attackers to identify your site as a target when new security vulnerabilities come to light. But it turns out that hiding Drupal doesn’t actually give you a leg up in the security game.  

One reason: most attacks are executed by bots, which generally do not even check what CMS your site is running before attempting known security vulnerabilities.

“Security through obscurity” won’t even deter human hackers looking to execute targeted attacks against your website. Anyone making the effort should be assumed to have the time and resources to find what you’ve hidden - or at least to bruteforce your site if that fails. Obscuring the vulnerabilities your site might have does nothing to address the vulnerabilities that are there.

What if I want to hide it anyway – are there any downsides?

Hiding that your site runs Drupal can actually make it harder to maintain real security. The changes you’d need to make to completely conceal Drupal are so extensive that they will break core. Security updates and patches will no longer be readily compatible with your altered version of Drupal, requiring time and extensive effort before they can be applied. The longer your website is without the latest security update, the longer known vulnerabilities exist in your code – ready to be exploited by bots searching for late adopters.

I still want to hide Drupal. How can I do so?

The Acquia Help Center provides detailed instructions for Hiding the fact that your site runs Drupal - with the caveat that the outlined steps take a significant amount of effort to implement and maintain, and probably aren’t worth your effort.

Blog series: Ask AcquiaWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Ask Acquia
Categories: Planet Drupal

Timing and Knowledge Matter When Load Testing

August 14, 2015 at 4:58pm

Timing is everything, as they say, and that’s especially true when trying to optimize website performance.

Last time out in this series on load testing, I offered ways to prepare for site visitors of all stripes. This time, let’s flip to the other side: back-end operations. Surely, how you maintain and upgrade your site will have just as much sway on site performance as visitors will.

Consider the saving of content. Does it affect performance? It does for many sites. For example, you’ll have sites for which load tests were run completely from the user’s perspective, but the first time these sites slide, the administrators will try to save a piece of content and hit a contention point. The cache is clear, but something strange has happened that wasn’t anticipated.

At Acquia, we’ve also seen a revision of saved content behave differently than brand new content that was saved. So just remember to test all possible kinds of actions, and do so early in the game, before a site slides.

Deployment can affect traffic, so you need to figure out the best time to schedule updates, changes and the like. Some of our large customers deploy only at night. Some customers deploy a dozen times a day. It depends on the job. Say you’re unveiling a brand new theme; then you’ll need to clear out all caches and should consider doing the job at a low traffic time. But if you’re only fixing a bug, then maybe you can send out a small code change and not worry about it affecting traffic. You just need to carefully consider ahead of time what could happen.

If you schedule load tests for an hour or two, have a developer run a deployment, make some changes, save some content, see what happens, and perform the last background operations, basically anything that’s scheduled.

We have a lot of customers who every hour, on the hour, will see glitches in performance. The biggest cause is all the background jobs they have running to sync content or connect to some external Web service. But because many of them setup monitoring early while running load tests, they’re able to see these spikes early.

Keep that in mind next time you perform a load test: a service that tracks performance will help. New Relic is one I recommend. During a load test, it allows you to see not just how requests perform from the customer’s perspective, but also how the application is performing.

Again, don’t wait. Take these steps before a launch so you can catch any problems.

In the next blog post, we’ll look at how Drupal 8 makes page caching easier and other things you can expect from the new platform when load testing.

Blog series: Load Testing Your Drupal WebsiteWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Erik Webb
Categories: Planet Drupal

Drupal 8 All-in: Acquia is ready for your D8 project now!

August 13, 2015 at 1:11pm
Language Undefined

Part 2 of 2 - In a recent conversation with Tom Erickson, Acquia's CEO, we got to talking about Acquia's "Drupal 8 All-In", what it is and what it means for the Drupal world and those we have yet to convince ... Acquia has drawn a line in the sand, saying, this thing is so great, we are confident we can deliver the kind of help and experience that we've been guaranteeing with Drupal over the last 7 or 8 years. This thing is ready enough. And all of the rough edges that we find along the way; this is the perfect opportunity to sand them off. This thing, Drupal 8, is ready for real business. In short: Acquia is running customer sites on Drupal 8 already, supporting all of Drupal 8, helping get the last rough edges off it for general release, and would love to talk with you about your next project and whether Drupal 8 is a good fit!

Categories: Planet Drupal

Excited about Drupal 8 - Meet Acquia CEO Tom Erickson

August 5, 2015 at 7:31pm
Language Undefined

Categories: Planet Drupal

Remember Everything – Even the Kitchen Sink – When Load Testing

August 4, 2015 at 2:13pm

You wouldn’t believe how many load tests fail to look beyond the home page. Seriously, many load tests focus on the home page and a few popular landing pages, neglecting to check most, or if not all, of the pages.

No, it’s not enough to replicate the visits of several thousand users in an hour for just one page and consider it a successful load test. With this blog – the fourth in a series about load testing – we’ll review the other user paths you should be considering before launching a website.

The most common mistake made in load testing is focusing mostly on the homepage. The thought is, “That’s the page where all visitors start.” And during development, that’s often where everyone is looking during a load test. Make no mistake about it: the homepage is the one that will rarely have a problem.

But you need to also look at all the other ways visitors enter a site. That’s where you see the tricks a site can play if not properly tested. For example, I once saw a retail site crashing on Black Friday, just about the worst time a customer-driven site can go down. The retailer had done a thorough load test of individual pages but hadn’t tested someone going to the public site and then jumping over to the store site. The home page was tested and pages were simply added on top of it. So you not only want to make sure you’re checking all the important pieces, but also that you understand how users are getting to your site and how they’re navigating around it.

Your first step should be identifying what all the user paths are through a site. For example, it’s easy to follow the clicks of an average e-commerce visitor and thus test accordingly: user logs in, visits a product page, adds a product to a shopping cart, reviews cart contents and completes transaction. But consider testing something atypical: logged in as a new user, search for something out of the ordinary and then navigate to a random result.

Logging in as a new user and creating a unique shopping cart is much more likely to catch problems than testing something more common. Try it next time you test. Randomize what’s actually in the load test. Granted, with certain tools that can be a bit difficult, but you need to make sure you’re not hitting the same product page, the same homepage, the same shopping cart – because that’s not what your actual users are going to do.

Oh, and don’t forget to conduct the same searches if you’re using a system separate from your website. For instance, an Acquia search will work at a different pace than a Durpal site will. You want to make sure that anything your site depends on will behave the same way on the separate system. At Acquia, we sell dedicated search instances to ensure that heavy use of search on your site will scale. You can rest easy knowing all the components of your site are behaving as you’d expect.

Next week, we’ll take a closer look at Web traffic. Until then, don’t forget to test as many pages in as many ways possible – and as different users. You’ll be grateful you did.

Blog series: Load Testing Your Drupal WebsiteWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Erik Webb
Categories: Planet Drupal

Load Testing Your Drupal Website: Avoid Points of Contention

July 29, 2015 at 1:29pm

You’d hate to have your business struggle for critical resources such as skilled employees, Internet bandwidth, or even capital. The same is undoubtedly true for website performance.
 
For developers, “contention” refers to when many website visitors are simultaneously competing for your application’s attention. In this blog – the third in a series about load testing – we’ll look at how you can try to catch a contention issue before it becomes a really big issue.
 
The idea of contention is that there’s a single resource that every request depends on. No matter how many servers you add to your application, there will always be a point where it has to pause, complete some work, and then free up all of the requests.
 
Contention is the hardest thing to catch in a load test, and probably the hardest thing to catch in Drupal, but it’s not impossible. With some diligence, you can stay on top of requests and not see your website’s performance suffer.
 
Consider when your caches need to be rebuilt internally. Surely, it’s best to not have a thousand people simultaneously visit your site and all of those cache entries have to be rebuilt all at once. Drupal will rebuild the cache of the first request, while having all the other requests pause. If that operation takes a hundred milliseconds, every single visitor’s request will pause for that exact same amount of time. To counter that, you can add more servers and resources, but it won’t make much of a difference. In fact, you’ll see that like with a lot of website launches, developers will add more resources and sit back and enjoy the show, only to stress when the inevitable contention issue arrives.
 
Now, contention issues are hard to track down, but it can be done. Thorough load testing can actually catch them by noticing arcs in results, indicating that response times suffer then recover in a short amount of time. This time to recover estimates the cost of relieving this contention. It all boils down to understanding the code, recognizing when contention happens and knowing when you can prevent it.
 
But don’t wait for that moment. If you’re conducting a load test, plug away and find the painful points of contention. You’ll thank yourself later.

Blog series: Load Testing Your Drupal WebsiteWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Erik Webb
Categories: Planet Drupal

Set Your Performance Page (And Other Drupal Website Improvements)

July 22, 2015 at 2:21pm

When Drupal comes out of the box, it's configured for ease of development – not performance. But making the application production-ready should indeed start with checking settings on the Performance page. The settings here let you control caching behavior and optimize bandwidth. If anything, it’s a simple reminder to distinguish development settings from test and production settings to get the best results in each environment.

This blog post, the last installment of a series that reviewed ways to improve Drupal website performance, offers some easy steps you can take to better your site.

Controlling Your Caching

An option on the Performance page turns on caching for anonymous users and for blocks. If block caching is turned on, even logged-in users will benefit. Unfortunately, this won’t happen if content access modules limit users to specific content. If the application includes modules that implement node grants, the block caching feature will be disabled.

It’s also useful if you have content cached on a reverse proxy, like what is leveraged within Acquia Cloud. The "Expiration of Cached Content" setting tells the proxy how long to hold onto the page before requesting the latest version.

Optimize your modules

Don't forget the settings for individual modules. Depending on what you've loaded, you may have the ability to configure caching behavior in the admin user interface. Some modules have an admin interface that isn't needed in production, and you may be able to disable it to avoid the overhead.

Reduce your requests

Reducing the number of network requests and the size of files sent improves performance. Most Drupal sites have a few dozen CSS and JavaScript files. If these are compressed and sent in a single download, the number of requests needed and the size of the transmitted data are reduced.

Configure your "Page Not Found" response

You'd think a missing page would get a fast response, but by default, Drupal needs to perform a complete bootstrap to confirm the page doesn't exist. Fast 404 short-circuits this process and sends an error response immediately. Configure it by specifying the paths and file types that should send the error response.

Tune your database

The database is often the hardest place to optimize, but it's usually where the biggest bottlenecks occur. Eliminating unnecessary writes to the DB can lead to big gains in performance. By default, Drupal's Database module logs every event in the database. Instead, limit unnecessary log entries and use the Syslog module, which instead writes to the operating system's logs. Switching the logging destination is simple, doesn't really lose anything, and can significantly improve performance.

Keep an eye on your system

Performance tuning isn't a one-and-done process. As applications and users change over time, the performance they experience also changes. At Acquia, we use tools like Acquia Insight and New Relic to monitor sites and get suggestions for fixes. Acquia also has a Logstreaming tool that lets us see, live, what's going on with the site. Reacting to this information helps us understand how the site is working and recognize where additional changes are needed.

I hope you enjoyed this series. If you have any questions or suggestions, please leave a comment below.

Blog series: Drupal Website PerformanceWorkflow: PendingFeatured: NoTags: acquia drupal planetDrupal 8 related: NoAuthor: Erik Webb
Categories: Planet Drupal

Gender balance in tech sales through data-driven, objective hiring

July 17, 2015 at 4:48pm
Language Undefined

At a recent Acquia all-company meeting, I was glad to hear that half of the current group of Acquia U students and 7 out of 12 of the latest "BDR" hires in Sales were women. Acquia's CEO, Tom Erickson added that this was the result of some "objective, data-driven" hiring practices. I had to know more. I got Acquia Senior Manager of Business Development and Sales, Chris Hemberger on the line to talk about all of this.

Categories: Planet Drupal

Start Developing Sites on Drupal 8 with Acquia

July 14, 2015 at 9:34pm

This is a part of our Drupal 8 Ready series of blog posts. Read what Dries has to say on buytaert.net. This post will be focused on our push toward Drupal 8 for Drupalists.

If you have been wondering when the right time is to start new projects with Drupal 8, you are not alone. We are approached with this question from partners and clients on a daily basis and have built up a many faceted decision tree for determining which way a project should go. Some of the questions we ask include:

  • When will you start UAT?
  • When do you plan to launch?
  • What authoring and integration functionality will you need at launch?
  • What is the category of the site you will be launching: Campaign, Brand, Intranet, Portal, Community etc.
  • Do you have developers proficient in Drupal 8?
  • Do you have Symfony developers on your team?
  • Do you intend to use modules outside of the top 50?

We have been recommending Drupal 7, but are now at the watershed moment where we can with great certainty recommend that users of our platform start building Drupal 8 sites for launch later this year. There are some projects that won’t be a fit, but if you are still trying to make this decision, it just got a whole lot easier.

Why should we use Drupal 8?
Most reading this post will know the major architectural changes baked into Drupal 8, if not I recomend a visit to this page: https://www.drupal.org/drupal-8.0. These on their own should give you some good reasons to make the move, but digging a little deeper, back-end developers should read about:

According to Morten DK, front-end developers should also be getting very excited about Twig.
In general, Drupal developers are loving Drupal 8, just ask core committer Alex Pott or the teams from Phase2, Amazee Labs and Chapter Three.

The Decision
Acquia has made a company-wide commitment to upgrade Acquia Platform and enable our Customer Success team for Drupal 8. This will ensure our customers can reap the benefits of Drupal 8 while having support to overcome its current challenges.

    Starting your Drupal 8 project now means:
  • Working with a reduced and immature module eco-system
  • Upskilling Drupal developers for object oriented programming
  • Retraining site-builders on the new authoring tools
  • Updating your continual integration scripts

We have a four-pronged approach to supporting our customers in this endeavour:

  1. We have long supported Drupal 8 on our development platform, Acquia Free, but as of this week, customers will be able to develop Drupal 8 sites on Acquia Cloud Enterprise and as of Drupal 8 RC1 we will be offering full production support. This will give customers the ability to leverage the full power of our 50 person, expert-laden, follow-the-sun support team in development, launch and maintenance of Drupal 8 sites.
  2. We have an internal tracking system for active module development among our many projects, allowing our PS, TAM and support teams to give you up-to-date information on the likely release schedule for critical modules.
  3. We have Drupal 8 enablement and migration packages available to speed up your new site development and Drupal 6/7 migration.
  4. Shortly we will be launching our updated Drupal certification programme for developers and site builders for Drupal 8.

The future of Drupal 8 on Acquia
At Acquia we are always thinking about how we can better serve developers. We have put years of research effort into developing better tools Drupal 8 and we have some very exciting releases in our short term pipeline:

  • A new Drupal 8 version of our open source drupal distribution, Lightning. This will be released by the end of the year to give developers a standard architecture and starting point for developing amazing Drupal 8 sites for enterprise (Full Disclosure: This is a shameless plug, as I am the new product manager for Lightning).
  • Upgrades of our existing products Acquia Lift and Acquia Cloud Site Factory to Drupal 8.
  • A completely refreshed automated build system in Acquia Cloud Enterprise with both new APIs and a GUI to enhance your development velocity on Drupal 8.

Thanks @EclipseGc and @webchick for inspiration. I hope this post has helped you make your decision.

Acquia is going all in!

Tags:  drupal drupal 8 developer acquia drupal planet
Categories: Planet Drupal

Pages