Create stronger composer integration [#2939866]

For discussion: The current simplytest.me implementation does not make use of Composer and has complications with respect to dependencies.

@balsama highlighted an example on Twitter:

One thing that could be improved is that http://Simplytest.me uses the D.O-built tarballs as the source. This prevents you from testing modules like Address that have additional Composer dependencies. (Without awkward workarounds)

This may resolved by generating a composer.json from the user-provided input (e.g. build a composer.json file that specifies the project/patch/distro/core version that a user enters from the homepage). The composer.json would subsequently be the input passed to the STM servers for provisioning. This could be something like:

1. Generate composer.json
2. Pass URL to composer file to one of the load balanced server
3. Generate codebase
4. Provision server from codebase

I know there is stronger support for Drupal 8 but it's worth exploring the impact to Drupal 7 as well.

Comments

Comment #1

26 January 2018 at 18:48

nerdstein created an issue. See original summary.

Comment #2

nerdstein

he/him

English

United States

CreditAttribution: nerdstein as a volunteer and at CivicActions commented 26 January 2018 at 18:54

I am curious if there is any existing composer-based UI projects out there. It would be cool to reuse or extend what is there instead of developing something custom

Comment #3

8thom CreditAttribution: 8thom as a volunteer and commented 27 January 2018 at 00:33

Issue summary:

View changes

@nerdstein I'd recommend using - https://github.com/drupal-composer as the base.

It supports both D7 - https://github.com/drupal-composer/drupal-project/tree/7.x & D8 (default)

Then you're able to just require the project and/or version if specified.

composer require drupal/address
composer require drupal/address:1.3

Composer will work out all other dependencies, and it should work for modules / themes / profiles and even packagist packages.

If something isn't working it should be fixed in packages.drupal.org as it would also be affecting users of composer.
Dependency complications isn't something this project should need to deal with.

Comment #4

8thom CreditAttribution: 8thom as a volunteer and commented 27 January 2018 at 00:46

Also the list of available packages is available @

D7 -- https://packages.drupal.org/7/packages.json
D8 -- https://packages.drupal.org/8/packages.json

You just have to loop through the "provider-includes"

Comment #5

nerdstein

he/him

English

United States

CreditAttribution: nerdstein as a volunteer and at CivicActions commented 31 January 2018 at 01:11

I see some real opportunities to simplify the current architecture with this.

Currently, there is a project import feature of SimplyTest.me that loads a bunch of data from a drupal.org sponsored XML file for the projects, themes, distros and more.

Composer now leverages the "packages.json" files @8thom noted above. I may be wrong, but this could be developed as a ReactJS decoupled app (or pick your JS framework of choice) that would perform the following:

1. Select your major version of Drupal (ideally by just selecting a branch from 7.x or 8.x)
2. The autocomplete widget queries the correct JSON file based on the major version selection
3. A list of projects are returned from the queried JSON results

Should this happen, the results should be near-real time (or as close as the JSON files are updated) and drastically simplify the STM site architecture by removing the data, the import of the data, and likely more. To avoid an issue with the infrastructure, the form submit should mirror the current data passed to the servers.

@mixologic said that as long as we avoid leveraging anything with cgit.drupal.org, it's safe.

Advanced search will have other considerations. I am curious how to get a list of patches or if a user would be expected to paste a URL (Composer would support this).

Comment #6

greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commented 31 January 2018 at 01:53

If you want to try a decoupled app, I'd strongly recommend looking at the Drupal distribution Contenta as the back-end. As a bonus, it comes with example front-ends in many different popular js frameworks, including React.

Comment #7

Jon Pugh

he/him

English

Newburgh, NY

CreditAttribution: Jon Pugh at ThinkDrop commented 31 January 2018 at 14:49

That's a great idea, Adam. It would also allow you to update the system in phases: improve the user experience while keeping the legacy infrastructure for now.

Trying to update everything at once will prove challenging.

Comment #8

nerdstein

he/him

English

United States

CreditAttribution: nerdstein as a volunteer and commented 27 February 2018 at 01:58

A stronger Composer Drupal 8 integration is now pushed to the 7.x-1.x branch. Drupal 7 is a bit tricker.

Capturing convo from Slack for persistence.

So, packages.drupal.org/7 works, and you can composer require any drupal project you want from that list.
If you are considering building the whole site with composer, you may want to do what we do on drupalci,
basically thats this: https://cgit.drupalcode.org/drupalci_testbot/tree/src/DrupalCI/Plugin/Bu...

and...

mixologic:
anhow, theres also a 'parent:run' in there which does ` $cmd = "./bin/composer ${verbose} config repositories.pdo composer " . $this->drupalPackageRepository . " --working-dir " . $source_dir;`
https://cgit.drupalcode.org/drupalci_testbot/tree/src/DrupalCI/Plugin/Bu...
thats the composer command to add packages.drupal.org/7 to the project.

Basically, packagist is not working well with D7 because D7 doesn't have a composer.json in the project root. So, more commands are needed to make this happen.