A frictionless migration to Drupal 8

Some of the goals we laid out for our migration to Drupal 8 were:

• Pairing our efforts with a site redesign so newly redesigned pages were done in Drupal 8. 

• Avoiding content freezes.

• Achieving faster turnarounds for new page layouts through atomic design principles. We decided that Dupal Bricks, which was a module only offered Drupal 8, was a great solution for our use case. 

• To leverage more effective configuration management.

• Achieving overall site speed gains.

• Avoiding scenarios where we are forced to complete an entire migration under duress and time pressure. 

Problem statement: 

• We knew at some point we wanted to migrate to Drupal 8. But how? Our site had about 1,500 pages, over 25 content types, and about 100 panel pages. It was a big task.

• We were in the beginning stages of a web redesign. We didn’t want to redesign the site in Drupal 7 and then migrate it to Drupal 8 aftwards; that would require theming the site twice. 

• We didn’t think we could build and theme the whole site in Drupal 8 in parallel because it would be a monumental task that would require building the Drupal 8 site in the background while continuing to maintain and update the Drupal 7 site. That seemed like adding double work to our already tight resources. And it would take a long time to do it this way.   

• We did not feel comfortable asking for a long content freeze because it would hurt the business. Even if we did, it would take several months and require a lot of help from contractors.   

• We saw lots of risks:

  1. If we put off migration for too much longer, we might find ourselves forced to migrate before we are ready. For example, because of a module that is no longer supported or a security issue. 

  2. Developers don’t want to work on old technologies, so we risk losing staff.

  3. A resource-intensive migration could hurt or slow down the business.

  4. A typical migration would require hiring contractors and consultants. And we would have to execute under tight deadlines. There is a lot that can go wrong to cause a project to go over time and over budget. 

  5. The web team needed to learn Drupal 8. How will we find time to learn with a full list of projects in our pipeline?

Solution in-depth 

We wanted to find a better way of migrating our site. Since this is a problem facing many organizations, we wanted to know if anyone figured out a less painful way of migrating from Drupal X to Drupal 8. Or for that matter, from web technology ‘A’ to web technology ‘B’.

After talking to a lot of experts, colleagues and community members, we heard a few stories about large sites that were migrated iteratively via a CDN origin redirect technique. We decided to investigate if it could be done in Drupal. After some trial and error, we were able to figure it out. Our CDN configuration was the main tool for making it succeed. For those of you who are not familiar with how a CDN works, below is a brief overview.

Origin redirect via CDN

Medium to large sites will typically have a Content Delivery Network or “CDN” sitting between the end user and the web server. The purpose of the CDN is to cache the website, so instead of getting content from the web-server, the user gets a cached version of the page on a server that is closer to the user than the hosted Drupal web server. This results in incredible performance gains compared to the hosted server. Typical CDN providers for Drupal are Akamai, Cloudflare, Fastly and Cloudfront/AWS. 

When a user requests a web page via their browser, the request first goes to the CDN. The CDN hosts the public domain, in this example let’s just call it: ‘https://mysite.com’. In order for the CDN to cache the HTML for ‘https://mysite.com’ it needs to fetch all the web page code from the origin web-server. Usually this is another CNAME, for this example  we will call it “https://myorgin.com”. The public doesn’t see “https://myorgin.com” only the domain associated with the CDN. See the diagram below.

As shown in the diagram above, it is common for a whole site to point to a single origin. But it is easy to create a rule in the CDN for a certain path, “/new-d8-page” in this example, to point to a different origin server. See diagram below.

This other origin “https://Drupal8origin.com” can be another web server with a different technology stack. But in our case it’s a server hosting a Drupal 8 version of our site.

This is easy to set up. Below are screenshots of sample configurations in Akamai.

As you can see from these rules, one can either have full paths to individual pages or leverage wildcards (*) to designate whole sections of a URL.

We found there is a second CDN configuration needed to address the fact that Drupal 7 and Drupal 8 share some of the same paths for images, CSS, JS and other resources. An example would be the “/sites” directory. We take the same URL path in the first rule and add them to this second rule which states that if the resource (CSS, image, JS) is being called by a specific page, look for that resource in the same directory but in the D8 server.

That’s it! When we did this for the first time, our Drupal 8 site consisted of our navigation, our theme and just one page. It was a single news page that we built and migrated to test out our theory. Over time we have migrated most of our site this way. 

How you can get started 

1. Create your theme in Drupal 8 or your front end framework of choice. In Drupal 8, Twig based theming is one of the first things that you need to learn in Drupal 8. This is a good place to start implementing your front end components leveraging the atomic design approach.

2. Find a content type/node on your Drupal 7 site that gets little traffic and that uses the most basic theme as your first test.

3. Re-create that page in Drupal 8.

4. Use the origin redirect method to publish the page.  

5. As you take on more projects, decide on ones that are good candidates to do in Drupal 8 instead of Drupal 7 based on your current skill level and time. 

6. Keep going until you chip away at our Drupal 7 site, learning as you go. 

7. As you get better at Drupal 8 development and theming, decide at some point every new project will be a Drupal 8 project no matter what. So as you do your normal work, you are also migrating your site. 

Things you will need to do to make it work

Change login URL

You will need to communicate to your editors and content contributors where to login to edit content in Drupal 7 or 8. When you apply the split origin technique, there will be three domains you will manage. In this example there is your public domain (https://mysite.com), your Drupal 7 origin domain (https://d7origin.com/) and your Drupal 8 origin domain (https://d8origin.com). 

1. You will need to keep your content contributors apprised of what content lives where. 

   • If they are editing Drupal 7 content they should login to: https://d7origin.com/user so they don’t go to URLs that are directed to Drupal 8; it can be very confusing. 

   • If they are editing Drupal 8 content they should login to: https://d8origin.com/user otherwise they will not find the correct content to edit. 

2. You will need to migrate users from Drupal 7 to Drupal 8

Editing content

1. Use relative paths when adding hyperlinks to content. You never want your origin URL exposed to the public. 

2. You should unpublish nodes in Drupal 7 that are migrated to Drupal 8 so the older Drupal 7 nodes don’t show up in sitemap files.

SEO 

1. You will need to create a sitemap file for the Drupal 8 site and add the URL to that sitemap of your main sitemap so you don’t lose any SEO.

2. You need to be able to edit your robots.txt file to include rules for both sites. 

Navigation

1. We converted our Drupal 8 top navigation and footer to an endpoint and had our D7 site consume it. This helped simplify maintaining the two sites as the content changed. This is not a requirement, but it helped us a lot. Here is documentation how we did it. [coming soon, Maxi Vera is writing up the documentation.]

What are the benefits?

• The main benefit is you can migrate the site on your own time, project by project. 

• It buys you time to learn Drupal 8, and make mistakes without exposing yourself to huge risks associated with a large migration project.

• The CDN rules are easy to create, update and maintain. 

What are the drawbacks?

• You have to have control of your CDN; to create rules and clear caches. You either need direct access or someone who can help you. 

• You need to clear CDN caches to get your changes to show

• Your site needs to be well organized in terms of URL paths, preferably with well organized mapping between different content types and their associated url patterns.

• If you have a lot of logged in community users we have not solved for this. We would love to get community feedback for this use case. 

• If you need to redesign your site all at once this might not be an ideal solution. 

What technical implementation is required to make it work? (in general)

• You will need origin URLs for both your Drupal 7 and 8 instances and be able to login to both sites. 

• You have to have control of your CDN; to create rules and clear caches. 

• You need to be able to edit your sitemap.xml file to include the Drupal 8 sitemap.xml file

• You need to have Drupal 8 configured to write to a different sitemap.xml file name. 

• You need to be able to edit your robots.txt file to include rules for both sites. 

• Converting your header and footer into an endpoint is a helpful way of migrating your look and feel across both sites. 

• What technical implementation is required (in specific) with your infrastructure? 

  1. We are hosted with Acquia Cloud and use Akamai for our CDN.

Questions 

1. What is the first area of the site you started with?

   We started with the most basic content type, something that just had body content to test to see if the origin redirect would even work. Then we focused on rebuilding the header navigation and footer. Eventually we converted our navigation in Drupal 8 into an endpoint that the Drupal 7 site could consume. That way we only had to manage it from one location. 
    

2. How did you undertake this first migration?

   The first content type we built was our partner section. The nodes were not that hard to re-theme. We had some landing pages that were built in Drupal 7 with views to the partner nodes. We discovered that we could leave the views for Drupal 7 in place as long as we had the nodes in Drupal 7 and 8 both published; the URLs from the Drupal 7 view would just resolve to Drupal 8 nodes. This saved us time as we later rebuilt the landing pages in Drupal 8. 

3. Did you change your process at all after this first section?

   Not much. It was so thrilling to get the first page and section done, after that the rest of the migration was pretty easy. 

4. What is the last area of your site to be migrated? Why?

   We are migrating our resources section last because our conversions happen there and we want to make sure we don’t hurt our business. After working in Drupal 8 for the better part of a year, we are not that worried about it.