Efforts to stabilize Composer-based Drupal platforms have been scattershot. There's Hosting Git (already in aegir core), and then there's Hosting Composer. There are a number of issues in the Aegir queues. See links below.
Considering Composer is now being used to construct Drupal itself, and the Drupal Composer site building method is finalized (enough), we should make composer install a built in feature of Aegir, more specifically Provision.
The attached branch runs composer install in the next few lines after we run drush make. It's as Drupal native as it gets.
Errors are handled through drush which means good capture by hostmaster.
Warnings are currently detected through string match for the word "Warning", I do not know if there is another way to do this. Most common warning is an out of date composer.lock file.
This patch is solely to run composer install
on verify. There is no UI. There is no need for a UI. A drush setting can block running composer install on subsequent verify runs if vendor is present. Other tasks can be provided by other modules, preferable Hosting Task Extras.
After discussion on Twitter, it's clear it's worth it to run composer install every time. If vendor and composer.lock are already present, the command only takes 1 second to run.To test this easily:
- Install Aegir.
- Enable Hosting Git.
- Create Platform
- Git URL: https://github.com/thinkdrop/drupal-composer
- Repository docroot: web
- Save
- Watch Verify end successfully having run composer install after cloning the git repo.
Comment | File | Size | Author |
---|---|---|---|
#31 | 2937147-composer-install.patch | 6.96 KB | Jon Pugh |
#29 | 2937147-composer-install.patch | 6.96 KB | Jon Pugh |
#26 | Screenshot from 2018-01-29 14-15-47.png | 22.4 KB | Jon Pugh |
#8 | Screenshot from 2018-01-16 15-31-58.png | 205.34 KB | Jon Pugh |
#6 | 2937147-composer-install.patch | 2.66 KB | Jon Pugh |
Comments
Comment #2
Jon PughComment #4
Jon PughHere's a patch file for easier review...
Comment #6
Jon PughComment #7
Jon PughComment #8
Jon PughHere's what a platform verify logs looks like:
Comment #9
colanI'm reluctant to get this in as-is given that:
But given that
...I won't block this if everyone's on board. However, if it goes in, we
shouldmust reincarnate Aegir Composer as Aegir Deploy (hosting_deploy) so as not to lose the deployment strategies or the clean-up of the Create Platform page. I actually merged all of the well-written documentation for this stuff earlier today. (It would need a bit of tweaking, but not a huge deal.) See #2931158: Add documentation to Aegir Docs for details.Comment #10
ergonlogicIt's worth noting that hosting_composer also provides an option to build platforms with
composer create project
Comment #11
Jon PughThanks for the feedback, but I really need a technical review or test.
I also think there's some fundamental disagreement on what "small core" really means. To me it means the core components are useful without needing a bunch of add-ons.
What is the decision process for what gets into this "small-core" anyway? I'd argue that building the platform codebase is as core as it gets, no?
"Small core" philosophies is why this patch is so small to begin with. It doesn't do anything extra: just enables a Successful verify on the first try when using git and composer, in only 44 lines of code
I'm not sure how much smaller you can get than that.
Comment #12
Jon PughThere's nothing preventing other modules from providing extra tasks. This isn't an alternative, this is a more native solution to the problem.
Comment #13
helmo CreditAttribution: helmo at Initfour websolutions commentedWhat about the '--no-dev' parameter for composer. One of my D8 test platforms (installed with drush dl) has a composer.lock file but was changed by the updated verify task by downloading things like phpunit.
How to handle servers that don't have composer installed? The first error line in a task log might be obvious for us, but it would not be nice to confront our users with that after the next aegir release. Should we start installing it via debian/aegir3-provision.postinst and not it in the release notes?
Comment #14
Jon PughI did add a @TODO about how to alter the composer install command, perhaps we do default to '--no-dev' and make the entire command a drush option so it could be overridden globally or in the platform?
I will add a check for the executable. If you could add it to the Debian package that would be great.
Comment #15
ergonlogicRe. "small-core", here's a good description. The most relevant definition is:
This was the fundamental idea behind the "golden contrib" initiative.
FWIW, I think adding Drush Make directly to core was a mistake; specifically because it is now becoming irrelevant, and we can't clean turn it off.
Comment #16
Jon PughI can appreciate the philosophical discussion of what should or not be included in this project in the future and the "next generation".
However I am more concerned with the present.
Our users right now are suffering from being unable to deploy composer-managed-drupal on Aegir without going to the command line or by using devshop or hosting_composer.
I have had code in DevShop to do this for probably more than 1 year. It's about time we put this in.
Comment #18
Jon PughBased on advice from Helmo I wrapped the entire thing in a
Now users can "turn it off" by setting that in their drushrc.php file.
Comment #19
memtkmcc CreditAttribution: memtkmcc at Omega8.cc commentedThis feature is so badly needed, that I'm eager to support whatever hack in Aegir core is needed to make it happen. *Now*.
Let's leave code elegance and related subtleties for another occasion (and branch) once we will have more time to do these things accordingly to the rules of the programming art.
That is my opinion/vote, at least.
GTD!
Comment #20
Jon PughComment #21
helmo CreditAttribution: helmo at Initfour websolutions commentedLast week I started coding a bit in the Debian postinst scripts to install composer. However now that it's included in both stretch and xenial it seems better to just defer to those packages.
Unfortunately it's not in Jessie, so adding it to 'requires' would break there. ... something we can do in 4.x but as Jessie is supported until 2020 breaking it in 3.x now is not nice.
2937147-composer-install now has a commit I'm testing for this
Comment #22
helmo CreditAttribution: helmo at Initfour websolutions commentedSeems to work ok... I just added a shell_exec('which composer') to be sure we're not failing when an existing installation just lacks the composer binary.
I created #2938992: Enhance composer support in Aegir core to handle the new option in hosting_composer.
Comment #23
Jon PughThanks helmo...
I think we might need to move the shell_exec()... I'm all about preventing existing installs from breaking, but for users expecting composer install, this will cause a silent failure if composer is not executable.
Let's change it to give a warning if composer is not executable, and tell the user how to disable the composer install (and the warning) by setting the drush option?
Comment #25
Jon PughOk, now what happens is:
if drush variable allows it, and shell exec composer fails, we show a warning:
The composer exectuable was not found. Install composer using instructions located at https://getcomposer.org/doc/00-intro.md#globally or, if you wish to disable composer install, set "provision_composer_install_platforms" to FALSE in the ~/.drush/drushrc.php file.
That way users who do want composer to run, will be told why it won't run.
Comment #26
Jon PughHow does it look to you helmo? If you set to RTBC again I can merge right now.
Comment #27
Jon PughComment #29
Jon PughChanges pushed, New patch attached.
Comment #31
Jon PughYAP
Comment #32
helmo CreditAttribution: helmo at Initfour websolutions commentedGreat ... https://gitlab.com/aegir/provision/pipelines/16791163 is running a final test ... lets merge if that passes.
Comment #33
Jon PughGitlab test passed. Running PR on github to test against the behat tests: https://github.com/aegir-project/provision/pull/27
Comment #34
Jon PughGitHub test passed! https://travis-ci.org/aegir-project/provision/builds/334871009
Merging...
Comment #36
Jon PughComment #37
Jon PughThanks everyone!
This is a big feature, it's really powerful when combined with hosting_git.module. I'm going into the docs now to write up how to use composer and git for Drupal on Aegir!
Comment #38
colanFor follow-up work necessary to provide the Deployment Strategies functionality, see #2938992: Enhance composer support in Aegir core.
Comment #39
Jon PughAdded documentation on using Composer in Aegir to the Platforms page:
https://github.com/aegir-project/documentation/blob/3.x/docs/usage/platf...
Comment #40
Jon PughWe should make a big deal out of supporting this setup now on the next release. People really want to build sites this way and haven't been able to do that using Aegir!
Comment #41
ahillio CreditAttribution: ahillio commentedWow, having followed #2736801 and now this issue, I am so delighted to see this! BIG THANKS Jon and everyone else!!!
Comment #42
mengi CreditAttribution: mengi commentedI agree, this is a big deal. Thanks to all who worked on this!