Why, what for?

As can be seen from the active discussion in Predictions for 2013, there is a huge demand for painless major version migration.

What's the problem?

The problem is that the major versions of Drupal are incompatible with each other (there is no backward-compatibility). This is good in the sense that each new major version of Drupal doesn't have a compatibility burden, and therefore represents the best possible functionality. But it is bad, because it makes difficult to migrate from earlier versions, and therefore prevents the spread of a new major version.

What is proposed?

  • For each Drupal module and the core make a common part of the major releases as a single major-independent library, and turn major-dependent releases of modules into adapters to this common part.
  • The basis of the common part is the latest major version of the module. Thus, the release of the latest major version will address to the common part without any adapter at all.
  • Each adapter of the n-th major version depends on the adapter of the n +1- th release version. Thus, the older major version of a single site, the greater number of adapters to the common part it needs for each module and the core.

What are the expected results?

  • + Each adapter = upgrade path: this is true because to make all tests for older major version passed, you must edit the adapter, that is, to teach an old major version to understand the new structure. How else to make it work, but to write a complete migration path? In addition, a major version migration process is nothing more than the installation of more advanced adapters, first for the core, and then for each of installed contributed modules - and then uninstallation of the old adapters. This process is reversible until the disabled adapters are not removed entirely - just disable the new adapter and turn on old ones back as they were.
    Thus, by simplifying the process of major version migration, we dramatically increase the user base of the latest major version, which in turn increases the speed of development, testing and implementation of new functionality.
  • + Each adapter = API changes documentation: By using such system we could improve change-list management: each adapter is a complete list of all major-version changes, so we could naturally place all documentation into this adapter, which is formatted using PHPDoc syntax - so there is no need in maintaining separate change-lists (for example, like Converting 6.x modules to 7.x).
  • + There is no need to port patches thanks to maximizing code reuse: every bug fixed in the common parts, automatically disappears IN ALL major versions, regardless of how many adapters are used to reach this common part through.
    This reduces the maintainership burden, which has a positive effect on the speed of development too.
  • + It increases the interest of ordinary users to upgrade to the latest major version: because it is a sure way to speed up your site by disabling adapters of the previous major version.
    Thus, the most advanced, bleeding-edge functionality developed, tested and implemented faster again thanks to an increase of the user base.
  • - Additional dependency: minus is that there is an additional dependency. Each module that wants to use the Forwardport doctrine, is forced to become dependent on Libraries. I assume this is a price worth paying for the benefits that this doctrine promises.
  • - The installation procedure differs a lot from the usual: to bring the module that embodies the Forwardport doctrine in working condition, the following steps are required:
    • Download major-dependent release of the module as usual (using drush dl or by unpacking downloaded distribution manually into sites/all/modules)
    • Due to the dependence of the Libraries, download and install the Libraries
    • Using Libraries download and install a common major-independent part of the module, which is plugged as a library, because it is not major-version dependent.
    • Install the module itself, since all the dependencies are satisfied.

    I guess this is not much of a problem, as many of the existing modules (some of which are very popular) are already working on this scheme: PHPMailer, Wysiwyg, Colorbox and so on.

So, what is the plan?

  • First, we need a protective layer (ANTICORRUPTION LAYER in terms of domain-driven development): namely, a buffer that will implement this doctrine gradually without any sudden movements. That is, on the one hand, it works as a proof of concept, and the other - without disrupting the existing infrastructure. Evolution rather than revolution.
    I'm going to use Module Object Oriented Programming API, one of the modules I maintain, as such a fuse. It is useful (among other things) by the fact that it operates each module as a class - which dramatically simplifies development of adapters: simply apply Adapter design pattern. The task is more facilitated by the fact that we have the decorator pattern already implemented in Moopapi: it is not difficult to expand it to the adapter.
  • It is necessary to allocate a certain module as a pilot project: extract its major-independent part and turn it into a library (that is, to make it available to all major-dependent release of this module).
    I'm going to use Botcha as such a pilot project. First, because it is once again one of the modules I maintain - that is, I have full access to its source code. And secondly, because I've already done all the necessary preparatory work - and here's the results: in the attachment you can find the diff-file, which contains all the differencies between major releases of Botcha. In my opinion it looks optimistic and perspectively: it would not take much time to abstract these differences, putting them to the major-dependent adapters, and to turn a common part into the library. In the first iteration, I believe, we can deal without the adapter classes as such. For the demonstration of concept it is enough simply to put the SWITCHes by VERSION in the right places - then all of the code will go into the common part, and the major-dependent releases will be just a simple implementation of Libraries-hook. This design provides a good basis for further refactoring.

Why Forwardport?

Name emphasizes its opposite to doctrine of backporting: instead porting changes from new major version to old one, we take the most recent code as a foundation for all major releases - and teach adapters to work with it in the old way. Thus, we maintain a strength of Drupal (no compatibility burden) - but at the same time gain new positive quality: a painless major migration.

Questions? Suggestions? Wishes?

Any feedback is welcome. I want to receive the comments from experienced Drupal-developers to evaluate how promising my idea before I start actively implement it.
Thank you for reading! Please post a comment.

botchas_differencies.txt14.44 KBPatchRanger


webchick’s picture

Version:8.x-dev» 9.x-dev
Category:task» feature

This looks related to #1052692: New import API for major version upgrades (was migrate module), so cross-linking. Also re-categorizing as a feature and moving to D9, like that other issue, since this would be a major new API.

As you can see from the later responses in that issue, that doesn't necessarily mean "no" for D8, but it does mean at this point we would both a) have to be below thresholds, and b) this would have to be done enough that it doesn't introduce any major/critical follow-ups for it to still be valid for D8. See http://buytaert.net/code-freeze-and-thresholds for more info.

PatchRanger’s picture

I have created minimum viable product to illustrate the idea, as a proof of concept: new release of Botcha (botcha-6.x-4.x) has been splitted into 2 parts: Drupal-major-version-independent part ("library", which is placed into x.x-1.x branch) & Drupal-major-version-dependent part ("adapter", which stays a usual release). "Library" is available for download via drupalcode.org, so I have posted a special issue to whitelist it: #1940330: Add drupalcode.org into packaging whitelist. All Simpletest-tests are green for this new release at localhost, though it could not be tested by drupal.org testbot due to external dependency, see related issue #1923582: Add ability for testbot to run 'composer install' during installation.
Let me invite everybody to review and test new way of code management:

As usual any feedback is welcome.
For now I am going to create an appropriate D7 release too.

PatchRanger’s picture

Issue summary:View changes

Added API changes argument

PatchRanger’s picture

Ok, done: I've created an appropriate D7 release too (botcha-7.x-4.x, http://drupalcode.org/project/botcha.git/tree/refs/heads/7.x-4.x ).
Now I have a fully-functional illustration of how it is supposed to manage the code: any changes done in the common part ("library") touch both versions (D6 & D7), so it gets rid of backporting.

What's next?

But it is not the full strength of Forwardport doctrine. To get even more we should do the same with Drupal core. What do I mean? As you could see looking in Botcha-4.x code, there are a lot of switch-case's, which are necessary because Drupal core is not adapted by this doctrine. We should split Drupal core into 2 parts too. "Library" - is the most recent Drupal core code (let's say D8-dev) and "adapter" contains everything we need to make it to work with some concrete version.

What is the plan?

Here is what I am going to do next:

  • Create a sandbox (or use Moopapi for it), which is intended to develop adapters (D7 & D6) for the most recent D8 code. It will use Composer (or drush make) to download the freshest D8 code as a "library".
  • Use this code against D7 database - and make all Simpletest-tests passed by creating an appropriate adapter. It means also that we will have a fully-functional D7->D8 upgrade path.
  • Use this code against D6 database as well - and make all Simpletest-tests passed by creating an appropriate adapter. It means also that we will have a fully-functional D6->D7 upgrade path.

It looks like an ambitious task - and I am not sure where to start from. It seems obvious to start with Simpletest - but it is tightly binded with other core modules, so it is not so easy.

PS I've added "API changes documentation" argument to the initial message.

PatchRanger’s picture

Issue summary:View changes

Typo fixed

ordermind’s picture

This concept unfortunately means that the version-agnostic part of the module will need to support the lowest PHP version that the adapters support.

catch’s picture

Priority:Major» Normal

Downgrading to normal. A lot of this is covered by the new release cycle and the fact we're using interfaces a lot more. So 9.x should only/mainly include changes that absolutely cannot be done in a backwards compatible way with 8.x