Subscribe to Propeople Blog feed
Updated: 2 hours 6 min ago

A Content Staging Solution for Drupal 8 (and more)

February 26, 2015 at 6:02pm

Moving content between different environments is a need for many big companies that have a Drupal site and content that should be created, reviewed, edited and published on different environments. Although Drupal 8 is not yet released, a content staging solution is already in the works. Dick Olsson (dixon_ on drupal.org) and I are working on this solution together and we aim to release an alpha version soon.

The content staging solution for Drupal 8 is based on a re-designed version of the Deploy module. This solution consists of some contrib modules and depends on three Drupal core modules. The core dependencies are Entity API, Serialization and Restful Web Services. The contributed module dependencies are Relaxed Web Services, Multiversion, Key-value Extensions and (soon) Deploy.

The Relaxed Web Services module provides a Restful/Relaxed JSON API and endpoints for entities, file attachments, administrative tasks like revisions comparison, starting/stopping replication, etc. It extends the core REST API with better support for handling UUID references, revisions, file attachments, etc. This module is borrowing the API interface from CouchDB and its Replication API. CouchDB is a NoSQL, document-oriented database.

The Multiversion module provides revision support for all content entities. It also tracks update sequences to make dependency management easier and tracks revision trees (similar to Git) in order to support conflict detection. With Multiversion, entities are never deleted, they are just flagged as deleted. This is needed in order to replicate deletions and for conflict handling.

Key-value Extensions provides an extension of the core key-value API with a backend for lists and sorted sets that you can do range queries on. This module is needed because of the way the Multiversion module stores its sequence indexes.

Deploy (will be implemented soon) - provides a simple user interface to manage replication and conflicts.

Replication

At the moment, we are using the CouchDB replicator to test content replication between different systems. The CouchDB Replication protocol is a protocol for synchronizing documents between 2 peers over HTTP. This protocol will be used to implement the Replication Web Service module for Drupal 8.

The Replication Web Service module will provide the possibility to replicate content between different systems and Drupal 8, it will also have a Drush plugin for running the replication. Furthermore, it will be possible to run live replications in order to synchronize applications.

Offline applications

By using a standardized HTTP replication protocol for Drupal, such as the one CouchDB is using, the same solution will be applicable to other very interesting use cases as well.

The Offline First principle is quite new in web development, but it has many benefits for users and their experience. A website designed after these principles will continue to work, even if there is no Internet connection available. Now it’s possiblel to create Offline First applications with Drupal 8! We can build offline applications using the same suite of modules that we introduced earlier. To do this we need a remote database - represented by a Drupal 8 site and a local browser-based database, for example PouchDB.

The content staging suite provides all necessary features, such as synchronization, revisioning and file attachments, to create an offline application. At the moment, it’s working with PouchDB version 3.2.1. I’ve created a video to demonstrate how synchronization between PouchDB and Drupal 8 works

Video of Test pull and push replication using Drupal 8 and PouchDB

To test this I use an application based on ToDoMVC and PouchDB 3.2.1.

Headless Drupal

In Drupal 8, we have integrated the Twig template framework, a very good thing, especially for front-end developers. However, sometimes we want to create an absolutely custom frontend using the power of libraries and frameworks like AngularJS and Hoodie, combined with Drupal 8 on the back-end.

The solution we implement provides a lot more possibilities than the Restful Web Services module from Drupal 8 core. This will make possible to create awesome applications using frameworks like AngularJS, a PouchDB database and Drupal 8.

Other Systems

The content staging suite will have many different use-cases, allow for replication between different systems and database, not just between Drupal sites.

Currently we have test suites for replication between Drupal 8 and CouchDB (using the CouchDB replicator, but later you will be able to use the Replication Web Services module). We also have test suites for replication between Drupal 8 and PouchDB.

In the future, this solution may be used to integrate Drupal 8 with other libraries and frameworks.

For more information, check out:

Tags: Drupal 8content stagingCheck this option to include this post in Planet Drupal aggregator: planetTopics: Tech & Development
Categories: Planet Drupal

Varnish Tips and Tricks

February 18, 2015 at 5:02am

In this article we would like to share some use cases and recipes for configuring Varnish.

Use custom data for building varnish cache hash

By default, Varnish uses URL as a parameter for caching. In the VCL file, it is also possible to add custom data (for example: location, or custom header value) to hashing by using the sub vcl_hash{} configuration part. But there is yet another solution to be used that Varnish comes equipped with out of the box. We can set a Vary header that is respected. So, if we want our cache to be based on header X-MyCustomHeader in Drupal, then we can set the header to

Vary: Cookie,Accept-Encoding,X-MyCustomHeader. This way, different values of our custom header will have different cache records in Varnish.

Limit access to the site by ip address

When we build an intranet website, we can limit access to it on the Varnish level. This can be done in following way:

First we define list of allowed IP addresses:

acl offices {

   "localhost";

   "127.0.0.1";

   "1.2.3.4";`

   "5.6.7.8";

}

Then we restrict access to non matching addresses:

sub vcl_recv {

   if ( req.http.host ~ "(intranet\.example\.com)$" && !(client.ip ~ offices) ) {

        error 403 "Access denied";

   }

}

SSL termination

As Varnish is not handling https traffic, we need to terminate SSL before it hits Varnish. For that we can use nginx. Here is a list of links to articles that dive deeper into this topic:

https://www.digitalocean.com/community/tutorials/how-to-configure-varnish-cache-4-0-with-ssl-termination-on-ubuntu-14-04

http://edoceo.com/howto/nginx-varnish-ssl

http://mikkel.hoegh.org/2012/07/24/varnish-as-reverse-proxy-with-nginx-as-web-server-and-ssl-terminator/

https://wiki.deimos.fr/Nginx_%2B_Varnish_:_Cache_even_in_HTTPS_by_offloading_SSL

ESI

On a recent Propeople project, we had the requirement to include a block with data from an external website without any caching. The tricky part was that the external site was providing XML with the data. The solution we implemented was to use ESI block pointing to the custom php file that was pulling that XML and parsing it on the fly.

Hiding js requests to external domains

If we need to do some CORS (http://en.wikipedia.org/wiki/Cross-origin_resource_sharing) requests instead of our Javascript doing requests directly to external domain, we can do requests to our site, but with a specific URL. Then, on the Varnish level, we can redirect that request to external domain. In this case, Varnish will act like a proxy. This can be achieved with backend options.

backend google {

 .host = "209.85.147.106";

 .port = "80";

}

sub vcl_fetch {

 if (req.url ~ "^/masq") {

   set req.backend = google;

   set req.http.host = "www.google.com";

   set req.url = regsub(req.url, "^/masq", "");

   remove req.http.Cookie;

   return(deliver);

 }

}

This is an example from a brilliant book: https://www.varnish-software.com/static/book/

Multiple backends, load balancing

It is possible to define multiple backends for Varnish and switch between them. Most basic implementation is round robin or random. Here is an example:

backend web01 {

   .host = "example1";

   .port = "80";

   .connect_timeout = 120s;

   .first_byte_timeout = 300s;

   .between_bytes_timeout = 60s;

   .max_connections = 50;

   .probe = {

    .url = "/";

    .timeout = 10s;

    .interval = 20s;

    .window = 5;

    .threshold = 3;

   }

}

 

backend web02 {

   .host = "example2";

   .port = "80";

   .max_connections = 100;

   .connect_timeout = 120s;

   .first_byte_timeout = 300s;

   .between_bytes_timeout = 60s;

   .probe = {

       .url = "/";

       .timeout = 10s;          

       .interval = 20s;       

       .window = 5;

       .threshold = 3;

   }

}

 

director apache round-robin {

 { .backend = web01; }

 { .backend = web02; }

}

 

sub vcl_recv {

 set req.backend = apache;

}

 

It is also possible to set a specific backend for visitors coming from specific IP addresses. This can have a number of helpful uses, such as making sure that the editors team has  adedicated backend server.

if (client.ip ~ offices) {

 set req.backend = web03;

}


I hope you have enjoyed our tips regarding Varnish configuration. Please feel free to share your own thoughts and tips on Varnish in the comments below!

Tags: VarnishService category: TechnologyCheck this option to include this post in Planet Drupal aggregator: planetTopics: Tech & Development
Categories: Planet Drupal

Propeople, Blink Reaction and Bysted: Creating A New Agency for the Digital Age

December 22, 2014 at 2:00pm

Earlier today, Propeople (together with our parent company, Intellecta) made two big announcements that I am really excited about. The first is that Blink Reaction is joining our larger Intellecta family, and will be merging with Propeople - creating the largest professional Drupal services company in the world! The second piece of news is that Propeople is also merging with Bysted, one of Denmark’s foremost creative agencies. Together, these two deals are strategic in starting an exciting new chapter for Propeople.

Intellecta’s purchase of an 80 percent stake in Blink Reaction also means that Blink Reaction will be operating under my direct management. Our companies will initially operate as independent units but will join together as a new company under a unified brand by the deal’s completion on March 31st, 2015. This new agency will have a truly unique international reach - with 350+ employees worldwide, working across 9 countries. The new agency’s global footprint and working capacity will be unmatched in the Drupal space.

It is an honor to lead the start of a whole new chapter for Propeople and Blink Reaction. I hold the company that Nancy Stango (Founder and CEO of Blink Reaction) and her team have built in very high esteem, and can’t wait to see what we’ll be able to achieve together.

Welcoming Blink Reaction into the Intellecta family will greatly expand our technical capacity, specially when it comes to the development of digital solutions built on Drupal. At the same time, bringing Bysted into Propeople will bring our creative, design, and strategy offerings to new levels. Both of these developments will prove to be strategic in charting the future direction in which Propeople is heading.

This future direction is driven by the vision of being a full-service agency for the digital age. Propeople is an agency that has had a truly technical upbringing; this is at the core of our identity, manifests itself through our entire organization’s culture and touches everything that we do.

The majority of the prominent agencies in the larger digital space tend to come from strong creative or communication backgrounds. The fact that Propeople comes from a technology background is a significant distinction in a world where technical matters have become increasingly more important for organizations as they develop their brand’s digital presence. And I’m not just talking about the importance of coding - but the larger way that analytics, data, and integrations with a variety of systems seem to be the common threads running through everything that a company does.

Today marks an important milestone for me, the Propeople team around the world and our customers. I, for one, can’t wait to get started!

Check this option to include this post in Planet Drupal aggregator: planetTopics: Business & Strategy
Categories: Planet Drupal

Drupal UX Improvements: When Node Forms are a Nuisance

December 11, 2014 at 11:16pm

Node forms can be big. With field collections, reference fields, tags, taxonomies, location fields, and others, node forms can actually be really big. This can make an editor’s experience so frustrating that it’s a surprise she doesn’t have a heart attack when she sees how much data she needs to enter in order to save the form.

Sound familiar? Well, in this article we will share some tips and tricks that we use to simplify editors’ lives and to make the Drupal editing experience more user-friendly.

Split a form into tabs

If you have too many fields on one screen, it’s nearly impossible for an editor to remember what he entered in the beginning by the time he’s reached the end of the page. The answer to this quandary is simple: it’s time to split up the form.

It is possible to use different criteria to split forms into more manageable sections, or tabs. The criteria you’ll use will depend on the type of form and its intended purpose. Forms can be split up by field types, required vs. optional fields, priority rank, more vs. less frequently used; the list goes on. To create an even better user experience, ask the editors themselves about how they’d like forms to be organized. If they have been using a particular form for a while, it’s likely they’ll have plenty of valuable feedback regarding the best way to make the forms more intuitive and improve the efficiency of their workflow.

From a technical perspective, there are multiple ways to achieve more manageable forms. The one we use most frequently utilizes vertical tabs. An example of this is shown in the screenshot below.

 

Vertical tabs help editors to concentrate on one thing at a time and improves the navigation experience when working with especially large forms. A good rule of thumb is to divide the form into chunks that won’t require users to scroll through more than 1 to 1.5 pages to complete all the elements on any given section.

In addition to vertical tabs, you have the option to set up horizontal ones. This can easily be achieved with the Field Group module.

Split a form into columns

Some editors set up their workstations to use fairly wide screens. In this case, another practical approach is to have multiple columns. This solution can be used to nicely group fields together so they can be viewed side by side.

Drupal's Panels module make it easy to configure forms that use columns. Simply set up the layout for a node’s edit form and arrange the fields however you like.

Warn editors about unsaved changes

If you use JIRA or Google Docs, you’ve probably seen a warning message like the one shown below.

 

For a busy editor juggling multiple tasks, this friendly little pop-up can be a lifesaver. And of course… there’s a module for that.

Allow us to introduce Node Edit Protection. It helps editors to remember to save their changes before navigating away from a form. This becomes especially handy if you’ve split your forms into multiple tabs, as editors may think that simply switching to another tab automatically saves their changes.

Taxonomy tag widgets

You won’t hear any complaints from us about the standard autocomplete feature and the way it enables editors to quickly select appropriate tags. There are multiple widgets, however, that put the icing on the autocomplete cake. One of these that we particularly like is Chosen.


 

Another good one is Autocomplete Deluxe.

 

Links

Sometimes editors need some help building links in WYSIWYG or when using link fields. This is especially true when editors need to search for and quickly locate the content or URL they want to reference. Enter the Linkit module, which acts as a link-specific autocomplete function.

linkit2013.jpeg

 

With the Linkit Picker module, we can even have a custom view to search for the content we would like to link to. This allows us to configure additional filters to help editors find content more effectively.

48.png

 

References

If you are using reference fields, there is a nifty module called References Dialog. This module allows editors to create a referenced entity while working within a node using a dialog box.

22.png

 

This comes in very handy, especially when your referenced entities include only a handful of fields and can fit comfortably into the dialog pop-up.

May we suggest...

For more great ideas that will have editors singing your praises, check out this presentation from DrupalCon Amsterdam 2014 about building a better backend, as well as our colleague Boyan Borisov’s presentation about improving Drupal's editorial experience.

Interested in learning even more about how Propeople can create a Drupal platform with user-friendly backend functionality for you? Don't hesitate to contact us.

Tags: DrupalUXcontent managementService category: TechnologyCheck this option to include this post in Planet Drupal aggregator: planetTopics: Tech & Development
Categories: Planet Drupal

Prototypes: A Better Approach to Development

December 3, 2014 at 11:21pm

Traditional web design has always involved creating flat, two-dimensional designs in wireframes or high-definition design comps. While this process feels natural and intuitive for designers, it presents significant shortcomings when it comes to the increasing demand for modern, responsive websites.

Building a website in Drupal typically follows this process:

  1. create low-definition conceptual designs, wireframes, and sketches

  2. create high-definition design files in Photoshop

  3. configure the site and build out various functionalities

  4. create a base theme and apply it to complete functionalities

 

Dev Process (Yuriy).png

 

This process, tried-and-true as it may be, leads to a lot of challenges:

Synchronicity of frontend and backend development

Building functionality is tied to internal elements that sometimes are not exposed to UI at all. For example, integrations with external systems or implementing editorial workflows. This sort of work can represent hundreds of hours from a development perspective, but the theming required is marginal. On the other hand, some parts of web design work--like assembling pages with node listings or just the pages themselves--can involve 10 hours of development and 20 hours (or more!) of theming due to custom Javascript and layouts. In this situation it can be challenging for a busy frontend developer to know how to organize his/her time most effectively.

Clients can't test the site until most of the functionality is done

Theming only after a site’s functionality is ready leads to a problematic situation wherein a web design team can't actually show its client polished work until the end of the project. Theoretically, of course, we can show some functionality or provide elaborate descriptions, but the “bells and whistles” still tend to be missing until a project nears completion. This leads to a huge bottleneck effect as both clients and quality analysts turn their attention to testing a new website on multiple devices and platforms. At this stage, most of the bugs reported will be related to responsive behavior or small changes to theming. This can be a nightmare for frontend developers; they receive tons of work all at once, even as the project is about to finish.

Enabling frontend developers

Frontend developers are forced to learn a lot about preprocess functions and how Drupal works. This whittles down the time frontend Drupal developers have to introduce cool new technologies that exist elsewhere in the frontend world. Despite multiple attempts to make Drupal’s HTML output cleaner, Drupal still produces a lot of code that, most of the time, is superfluous. The truth is the frontend world develops at a faster pace than our PHP tools. New Javascript frameworks mature at lightning speed and we are not really all that good at adopting them. What we would really like to see is frontend developers progressing in their field, rather than getting bogged down by more Drupalisms. But how?

Prototypes to the rescue

Lately Propeople has been rewriting the web design process, building prototypes right after designs become available. In fact, the main deliverables of our design work are the prototypes themselves. Technically, the prototypes we build are sets of HTML pages. The idea here is for these prototypes to empower frontend developers to build a site how they see it, instead of how Drupal dictates. Prototypes free up frontend developers to use new technologies and to properly organize code. Another advantage is that prototypes can be built before Drupal functionality is finalized. In other words, clients can start test driving their website early on in the development process and have a clear idea of how the site is going to look and behave. By the time Drupal functionality is ready, all frontend bugs are resolved, specifically the ones related to responsive behavior. There are multiple PHP-based template engines to use, platesphp.com being one. Our developers use assemble.io.

Prototype based Dev Process (Yuriy).png

 

Prototype-based theming

Let’s say you’ve got your prototypes just the way you want them. Now Drupal’s backend HTML should be made to match the prototypes as precisely as possible. The bad news: this is not easy as it may sound. The good news: we’ve compiled some best practices to lead the way.

  1. Panels, panels, panels. We use a stack of panels modules. It is pretty simple to build custom layouts out from the prototypes, replacing static blocks with implemented panes.

  2. Custom panes with custom templates. In order to control the HTML of custom panes, we create a template file for each one. This makes it easy to tweak the HTML of a single pane as needed. We even display custom panes in lieu of views panes, executing views under the hood. We skip rendering views in order to keep all the theming for one block in one template.

  3. Display entities as view modes. For most view modes (different kinds of teasers), we use separate custom templates.

Prototypes make building a website much more exciting and improves the web design process for everyone involved. Backend developers can actually watch the site’s theming implemented as they complete functionality. Frontend developers have more freedom and fewer Drupalisms to remember. Best of all, prototypes increase client satisfaction because prototypes allow clients to feel more involved in the implementation of their projects.

 Tags: PropeoplePrototypesDesignService category: TechnologyCheck this option to include this post in Planet Drupal aggregator: planetTopics: Tech & Development
Categories: Planet Drupal