Problem/Motivation

Setting up Drupal for the first time is a hard task, see http://matthewgrasmick.com/compare-php-frameworks.
This is quite a sad state compared to other project like in the node world:

npm require X
npm install
🍷

The goal of this issue is to provide a similar experience for first time Drupal users.

This command might not be for you when you

  • Know how to run docker
  • Have drush or drupal console installed on your machine
  • You know already how to setup Drupal

Proposed resolution

The steps we need to execute are:

  • Install Drupal (setting the file system up, creating a database, install some install profile)
  • Start a webserver

Luckily there is SQLITE and the PHP built in webserver out there.

Detailed plan

How to test

  1. Apply patch
  2. Run $ php core/scripts/drupal quick-start from the command line
  3. Wait till the browser is opened up.

Remaining tasks

None

User interface changes

Videos:

API changes

API additions to install_drupal() and install_run_tasks(). These methods take a new argument of a callable that allows CLI processes to implement a progress bar like the interactive installer.

Data model changes

None

CommentFileSizeAuthor
#296 2911319-2-296.patch42.26 KBalexpott
#296 294-296-interdiff.txt322 bytesalexpott
#294 interdiff-2911319.txt1.43 KBdawehner
#294 2911319-294.patch42.26 KBdawehner
#293 interdiff-2911319.txt1.46 KBdawehner
#293 2911319-293.patch42.29 KBdawehner
#290 2911319-2-289.patch41.65 KBalexpott
#290 288-289-interdiff.txt627 bytesalexpott
#288 283-288-interdiff.txt1.65 KBmradcliffe
#288 2911319-2-288.patch41.5 KBmradcliffe
#283 2911319-2-283.patch41.45 KBmradcliffe
#283 274-283-interdiff.txt624 bytesmradcliffe
#275 quic-start__patch-274.png33.65 KBAnonymous (not verified)
#274 2911319-2-274.patch41.54 KBalexpott
#274 273-274-interdiff.txt1.58 KBalexpott
#273 2911319-2-273.patch41.54 KBalexpott
#273 272-273-interdiff.txt1.62 KBalexpott
#272 2911319-2-272.patch41.43 KBalexpott
#272 270-272-interdiff.txt1.25 KBalexpott
#271 quick-start__run.png17.88 KBAnonymous (not verified)
#271 quick-start__help-info.png14.39 KBAnonymous (not verified)
#270 interdiff-2911319.txt5.12 KBdawehner
#270 2911319-270.patch41.36 KBdawehner
#265 2911319-2-265.patch41.29 KBalexpott
#265 247-265-interdiff.txt1.47 KBalexpott
#257 issue-2911319-retesting-browser-only.png150.4 KBmaxstarkenburg
#257 issue-2911319-retesting.png308.48 KBmaxstarkenburg
#253 issue-2911319-reinstall-issue-after-sqlite-deletion.png361.88 KBmaxstarkenburg
#253 issue-2911319-reinstall-issue.png314.2 KBmaxstarkenburg
#247 2911319-2-247.patch40.83 KBalexpott
#247 245-247-interdiff.txt2.15 KBalexpott
#246 issue-2911319-2-245.patch.png299.49 KBmaxstarkenburg
#245 2911319-2-245.patch40.93 KBpbirk
#245 237-245-interdiff.txt529 bytespbirk
#241 2911319-windows-quick-install.PNG347.7 KBChelsee
#237 2911319-2-237.patch40.82 KBalexpott
#237 236-237-interdiff.txt2.03 KBalexpott
#236 2911319-2-236.patch40.31 KBalexpott
#236 229-236-interdiff.txt4.81 KBalexpott
#230 Windows 10 Patch 229.mp41.09 MBgeerlingguy
#229 2911319-2-229.patch39.65 KBalexpott
#229 227-229-interdiff.txt1.22 KBalexpott
#227 2911319-2-226.patch39.59 KBalexpott
#227 225-226-interdiff.txt5.49 KBalexpott
#225 interdiff-2911319.txt4.04 KBdawehner
#225 2911319-225.patch39.19 KBdawehner
#223 2911319-2-223.patch38.94 KBalexpott
#223 220-223-interdiff.txt591 bytesalexpott
#220 2911319-2-220.patch38.93 KBalexpott
#220 217-220-interdiff.txt2.05 KBalexpott
#217 2911319-2-216.patch38.84 KBalexpott
#217 214-216-interdiff.txt842 bytesalexpott
#214 2911319-2-214.patch38.67 KBalexpott
#214 211-214-interdiff.txt791 bytesalexpott
#211 2911319-2-211.patch38.67 KBalexpott
#211 208-211-interdiff.txt902 bytesalexpott
#208 2911319-2-208.patch38.48 KBalexpott
#208 2911319-2-208-should-fail.patch38.51 KBalexpott
#208 200-208.patch7.49 KBalexpott
#203 2911319-2-200.patch38.49 KBalexpott
#203 191-200-interdiff.txt11.34 KBalexpott
#201 errors-wsl-php.txt4.7 KBgeerlingguy
#201 errors-in-wsl-ubuntu.png43.95 KBgeerlingguy
#191 2911319-2-191.patch37.49 KBalexpott
#188 2911319-2-188.patch38.47 KBalexpott
#188 171-188-interdiff.txt6.89 KBalexpott
#171 interdiff-2911319.txt4.48 KBdawehner
#171 2911319-171.patch36.42 KBdawehner
#169 2911319-2-169.patch36.36 KBalexpott
#169 167-169-interdiff.txt709 bytesalexpott
#167 2911319-2-167.patch36.2 KBalexpott
#167 158-167-interdiff.txt1.16 KBalexpott
#158 2911319-2-158.patch35.22 KBalexpott
#158 157-158-interdiff.txt13.05 KBalexpott
#157 2911319-2-157.patch35.95 KBalexpott
#157 151-157-interdiff.txt7.23 KBalexpott
#152 2911319-2-151.patch33.22 KBalexpott
#152 150-151-interdiff.txt1.92 KBalexpott
#150 2911319-2-150.patch33.24 KBalexpott
#150 148-150-interdiff.txt17.48 KBalexpott
#148 2911319-2-148.patch32 KBalexpott
#148 146-148-interdiff.txt674 bytesalexpott
#146 2911319-2-146.patch31.86 KBalexpott
#146 142-146-interdiff.txt46.87 KBalexpott
#143 2911319-142-no-composer.patch22.76 KBdawehner
#142 interdiff-2911319.txt8.28 KBdawehner
#142 2911319-142.patch23.69 KBdawehner
#135 2911319-2-135.patch23.39 KBalexpott
#135 131-135-interdiff.txt10.87 KBalexpott
#131 2911319-2-131.patch23.1 KBalexpott
#131 129-131-interdiff.txt6.34 KBalexpott
#129 2911319-2-129.patch22.6 KBalexpott
#129 127-129-interdiff.txt1.89 KBalexpott
#128 124-127-interdiff.txt4.05 KBalexpott
#127 2911319-2-127.patch23.37 KBalexpott
#127 124-127-interdiff.txt1.98 KBalexpott
#124 2911319-2-124.patch23.06 KBalexpott
#124 122-124-interdiff.txt944 bytesalexpott
#122 2911319-2-122.patch22.89 KBalexpott
#122 121-122-interdiff.txt11.48 KBalexpott
#121 2911319-121.patch19.02 KBheddn
#121 interdiff_120-121.txt2.51 KBheddn
#120 2911319-2-120.patch17.86 KBalexpott
#120 116-120-interdiff.txt5.34 KBalexpott
#116 interdiff-2911319.txt6.01 KBdawehner
#116 2911319-116.patch16.9 KBdawehner
#67 2911319-67.patch14.04 KBdawehner
#67 interdiff-2911319.txt1.78 KBdawehner
#62 57-62-interdiff.txt983 bytesdanquah
#62 2911319-62.patch13.6 KBdanquah
#58 2911319-57-no-composer.patch12.73 KBalexpott
#57 2911319-57.patch13.55 KBalexpott
#57 56-57-interdiff.txt1.08 KBalexpott
#56 interdiff-2911319.txt5.51 KBdawehner
#56 2911319-56.patch13.6 KBdawehner
#50 2911319-50.patch9.85 KBmglaman
#50 interdiff-2911319-47-50.txt3.74 KBmglaman
#47 2911319-47.patch9.79 KBmglaman
#47 interdiff-2911319-32-47.txt1.19 KBmglaman
#32 interdiff-2911319.txt2.68 KBdawehner
#32 2911319-32.patch9.84 KBdawehner
#28 interdiff-2911319.txt6.83 KBdawehner
#28 2911319-27.patch8.72 KBdawehner
#25 interdiff-2911319.txt10.57 KBdawehner
#25 2911319-25.patch8.38 KBdawehner
#21 2911319-21.patch5.83 KBdawehner
#16 Screen Shot 2017-11-02 at 12.10.49 PM.png336.42 KBwebchick
#10 2911319-10.patch5.27 KBdawehner
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner created an issue. See original summary.

yoroy’s picture

dawehner’s picture

It is a bit indeed. The way how I see it is that #2911319: Provide a single command to install & run Drupal is far less of a radical proposal :)

kenorb’s picture

Btw. One command to install and run Drupal:

drush qd

See: What is the easiest way to install clean Drupal from scratch using drush?

kenorb’s picture

kenorb’s picture

alexpott’s picture

Perhaps also related is #2242947: Integrate Symfony Console component to natively support command line operations as that will make running commands simpler.

Another idea would be to add a command to the root composer.json that can do this. See https://getcomposer.org/doc/articles/scripts.md#writing-custom-commands

dawehner’s picture

Note: #2869825: Leverage JS for JS testing (using nightwatch) has a command to install Drupal for tests, which might be interested to have a look at as well.

alexpott’s picture

Issue summary: View changes

In #1543858: Add a startup configuration for the built-in PHP server that supports clean URLs we discussed the need for documentation in core files. This issue should add it.

dawehner’s picture

Status: Active » Needs review
FileSize
5.27 KB

Here is some start of a naive implementation ...

alexpott’s picture

Issue summary: View changes

Just tested the command - so sweet. Nice work @dawehner.

I think we should add some resilience - for example what happens if sqlite is not available? Also the output could be more instructive as to the url to visit and the admin user password. Ah... see code review @dawehner++

  1. +++ b/core/scripts/dev-start.php
    @@ -0,0 +1,104 @@
    +  // Installs drupal using sqlite.
    +  if (!extension_loaded('pdo_sqlite')) {
    +    $output->writeln('Sqlite required ...');
    +    return;
    +  }
    

    Lol I should have looked at code. So we need a better message but awesome to see this!

  2. +++ b/core/scripts/dev-start.php
    @@ -0,0 +1,104 @@
    +      // @todo Do we want to make this configurable?
    +      'profile' => 'standard',
    +      // @todo A different langcode would be nice as well?
    +      'langcode' => 'en',
    

    Yep this would be super nice. Of course such things need validation :(

  3. +++ b/core/scripts/dev-start.php
    @@ -0,0 +1,104 @@
    +        // form_type_checkboxes_value() requires NULL instead of FALSE values
    +        // for programmatic form submissions to disable a checkbox.
    +        'enable_update_status_module' => NULL,
    +        'enable_update_status_emails' => NULL,
    

    These look like sensible defaults for a dev environment.

  4. +++ b/core/scripts/dev-start.php
    @@ -0,0 +1,104 @@
    +  if (file_exists('sites/default/settings.php')) {
    +    $result = unlink('sites/default/settings.php');
    +    if ($result === FALSE) {
    +      $output->writeln('Removing settings.php failed, please do it manually ...');
    +      return;
    +    }
    +  }
    +  if (file_exists('sites/default/files/.sqlite')) {
    +    $result = unlink('sites/default/files/.sqlite');
    +    if ($result === FALSE) {
    +      $output->writeln('Removing .sqlite failed, please do it manually ...');
    +      return;
    +    }
    +  }
    

    I think before doing something destructive we should involve the user.

  5. +++ b/core/scripts/dev-start.php
    @@ -0,0 +1,104 @@
    +  ->register('dev-start')
    

    It might be worth having two command dev-start and dev-install. dev-start should only do an install if there's no settings.php otherwise it could just start the webserver.

Chi’s picture

Issue tags: +Security

On my localhost when I access to core/scripts/dev-start.php through web it tries to remove settings.php file.
I know this should be handled by proper server configuration but anyway it would be more secure if we take care about it.
For instance in the core/scripts/db-tools.php there is the following condition.

if (PHP_SAPI !== 'cli') {
  return;
}
Chi’s picture

+ $builder = new ProcessBuilder([$binary, '-S', 'localhost:8888' , '.ht.router.php']);
There is a nice implementation for this in Symfony but unfortunately it is tied to the framework.
https://github.com/symfony/symfony/issues/24534

dawehner’s picture

While it would be super cool to have some a component, I don't think its something we should wait for. For me at least its all about having the command available, we can always rewrite the internal completely later at some point.

Wim Leers’s picture

❤️❤️❤️

webchick’s picture

Slightly adjusting the tag, since with this version it says this on the project page. 😂

Security problem.

dawehner’s picture

That's hilarious!

borisson_’s picture

Title: Provide a one command to install and run Drupal command » Provide one command to install and run Drupal command
jibran’s picture

What is the difference between #2926633: Provide a script to install a Drupal testsite and this?

dawehner’s picture

dawehner’s picture

FileSize
5.83 KB

* Moved it into a symfony command file
* Added support for install profile + language

jonathanshaw’s picture

Title: Provide one command to install and run Drupal command » Provide a single command to install & run Drupal
alexpott’s picture

  1. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,168 @@
    +    ;
    +    ¶
    +    parent::configure();
    +  }
    

    Some unintended whitespace.

  2. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,168 @@
    +    $root = dirname(dirname(__DIR__));
    

    Needs to be $root = dirname(dirname(dirname(dirname(dirname(__DIR__)))));

  3. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,168 @@
    +
    +    install_drupal($classLoader, $parameters);
    +    $output->writeln('Drupal successfully installed.');
    

    This process can take a long time. Maybe we can use the process command to fork and run this in a separate process and use Symfony's stuff to have a progress bar? At the very least we should have some output prior to this saying we're installing and this might take some time.

  4. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,168 @@
    +    $output->writeln('Starting webserver ...');
    +    $process->run();
    

    The output here should list how to connect to the site and credentials etc..

  5. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,168 @@
    +  /**
    +   * {@inheritdoc}
    +   */
    +  protected function execute(InputInterface $input, OutputInterface $output) {
    +    $root = $this->boot($output);
    +    $this->install($this->classLoader, $output, $input->getOption('install_profile'), $input->getOption('langcode'));
    +    $this->start($root, $output);
    +  }
    

    Can we move this near the top of the file - after configure I was surprised where this was in the file.

  6. Is it worth separating the install and starting webserver commands so that one can terminate the webserver command and restart it without having to re-install?
alexpott’s picture

Also, should have said typing in a single command to get umami up and running for a demo is really cool! Nice work @dawehner

dawehner’s picture

FileSize
8.38 KB
10.57 KB
Is it worth separating the install and starting webserver commands so that one can terminate the webserver command and restart it without having to re-install?

Well, I thought about it. I'm really not sure at all here. One the one hand sure, on the other hand, if you want this you can already use drush. Well, for it being useful for core development having two commands make sense. I combined them together in a composer script though.

Chi’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +use Symfony\Component\Process\PhpExecutableFinder;
    +use Symfony\Component\Process\ProcessBuilder;
    

    Unused use statements.

  2. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +   * @var object
    

    Can we use more specific type Composer\Autoload\ClassLoader?

  3. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +  public function __construct($classLoader) {
    

    The method parameter needs to be in snake case.

  4. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +   * Constructs a new DevStartCommand command.
    

    - DevStartCommand
    + DevInstallCommand

  5. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +    $root = $this->boot($output);
    

    Unused variable $root.

  6. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,83 @@
    +  /**
    +   * Boots up a Drupal environment.
    +   *
    +   * @return string
    +   *   Returns the path to the Drupal root.
    +   */
    +  protected function boot() {
    +    // Installs drupal using sqlite.
    +    if (!extension_loaded('pdo_sqlite')) {
    +      $output->writeln('Sqlite required ...');
    +      return;
    +    }
    +    $root = dirname(dirname(dirname(dirname(dirname(__DIR__)))));
    +    chdir($root);
    +    return $root;
    +  }
    

    The function name and description do not correspond to what this function really does. Also return type should be string|null.

  7. +    // Installs drupal using sqlite.
    +    if (!extension_loaded('pdo_sqlite')) {
    +      $output->writeln('Sqlite required ...');
    +      return;
    +    }
    

    Same thing here. This code does not install Drupal. Also 'drupal' needs to be capitalized.

  8. +    $root = dirname(dirname(dirname(dirname(dirname(__DIR__)))));
    

    Whi not $root = __DIR__ . '/../../../../..';?

  9. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -36,104 +35,29 @@ public function __construct($classLoader) {
    -      ->addOption('install_profile', NULL, InputOption::VALUE_OPTIONAL, 'Install profile to install the site in. Defaults to testing', 'standard')
    

    Per the option description default value should be 'testing'.

  10. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +   * Installs the standard install profile.
    

    Not quite correct, since this can install any installation profile.

  11. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +        $output->writeln('Removing settings.php failed, please do it manually ...');
    

    This needs to be printed to std error.
    $output->getErrorOutput()->writeln('PDO SQLite extension is required for this opertaion.');
    Also I would make the message more verbose and remove ellipsis.

  12. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,83 @@
    +  ¶
    

    Trailing whitespace

  13. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -4,13 +4,12 @@
     use Symfony\Component\Process\ProcessBuilder;
    

    ProcessBuilder is deprecated.

  14. +++ b/core/scripts/dev-site.php
    @@ -0,0 +1,13 @@
    +$application = new Application('dev-start', '0.1.0');
    

    What does '0.1.0' stand for? Can we use \Drupal::VERSION isntead?

  15. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +        $output->writeln('Removing settings.php failed, please do it manually ...');
    +        return;
    

    This return statement does not stop the application from beeing exectured. Since both boot() and install() methods are rather small I propose placing the entire command code to execute() method. In any case we need to return correct exit code from the application.

  16. +++ b/core/scripts/dev-site.php
    @@ -0,0 +1,13 @@
    +<?php
    +
    +use Drupal\Core\Command\DevInstallCommand;
    +use Drupal\Core\Command\DevStartCommand;
    +use Symfony\Component\Console\Application;
    

    Missing file doc comment.

  17. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,83 @@
    +    $builder = new ProcessBuilder([$binary, '-S', 'localhost:8888' , '.ht.router.php']);
    

    Per the coding standard each element of this array should be broken into its own line. Also space after localhost:8888 is redundand.

  18. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,145 @@
    +      ->addUsage('--install_profile demo_umami --langcode fr')
    +    ;
    

    Another CS issue. The ending semicolon should not be on its own line.

mglaman’s picture

Adding in something from Slack chat. We can support the builtin webserver on a random port. Here's some code I used to pick random ports for PhantomJS and Chrome web drivers

 protected static function getAvailablePort() {
    $address = '0.0.0.0';
    $port = 0;
    $socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
    socket_bind($socket, $address, $port);
    socket_listen($socket, 5);
    socket_getsockname($socket, $address, $port);
    return $port;
  }

Source https://github.com/mglaman/webdrivers/blob/master/src/DriverFactory.php#L63

dawehner’s picture

Status: Needs work » Needs review
FileSize
8.72 KB
6.83 KB

Can we use more specific type Composer\Autoload\ClassLoader?

I don't really want to hardcode an autoloader

The method parameter needs to be in snake case.

Does it, see https://www.drupal.org/docs/develop/standards/coding-standards#naming

Per the option description default value should be 'testing'.

I changed the text.

ProcessBuilder is deprecated.

The process class is way nicer

What does '0.1.0' stand for? Can we use \Drupal::VERSION isntead?

¯\_(ツ)_/¯ I think any version number doesn't mean anything

This return statement does not stop the application from beeing exectured. Since both boot() and install() methods are rather small I propose placing the entire command code to execute() method. In any case we need to return correct exit code from the application.

I tried to be in sync with what #2926633: Provide a script to install a Drupal testsite is doing

dawehner’s picture

@mglaman's snippet worked perfectly!

mglaman’s picture

From the issue summary

Run $ php core/scripts/dev-start.php from the command line

But there isn't a default command (still `list`). So I see

Available commands:
  help     Displays help for a command
  install  Installs a Drupal dev site. This is not meant for production or any custom development. It is a quick and easy way to get Drupal running.
  list     Lists commands
  start    Starts up a webserver for the dev site.

Do we want to default to a command which runs `install` and then `start`? so it can be an all in one command? I guess DevInstallCommand is not re-entrant safe and assumes you want to reinstall if you run it again. And DevStartCommand should only worry about running the built-in server.

Wherever this is documented should mention it is a two-step process for the first run, single command thereafter.

EDIT: Command outputs

Install

[03:43 PM]-[mglaman@Matts-MBP]-[~/Drupal/sites/drupal8/web]-[git 2911319] 
$ php core/scripts/dev-site.php install
Drupal installation started.
Drupal successfully installed.

Run builtin

[03:47 PM]-[mglaman@Matts-MBP]-[~/Drupal/sites/drupal8/web]-[git 2911319] 
$ php core/scripts/dev-site.php start
Starting webserver on http://localhost:65264
[03:48 PM]-[mglaman@Matts-MBP]-[~/Drupal/sites/drupal8/web]-[git 2911319] 
$ php core/scripts/dev-site.php start
Starting webserver on http://localhost:65293

Random ports ftw! Able to access installed site.

mglaman’s picture

+++ b/composer.json
@@ -54,6 +54,7 @@
+        "dev-site": "core/scripts/dev-site.php install && core/scripts/dev-site.php start",

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -0,0 +1,138 @@
+class DevInstallCommand extends Command {

I totally missed this. This is great but has one problem.

As stated in my last comment, DevInstallCommand is not reentrant. So if an unexpected user tried to execute `dev-site` thinking it'd execute like `npm start` they would blow away their existing site.

I think when `install` checks for an existing install it should exit successfully if there is a site. Unless a `--force` option is passed. Kind of like how Drush asks you "Are you sure?" for `site-install`. This way we can provide a `npm start` like command for people to get up and go.

dawehner’s picture

Status: Needs review » Needs work
FileSize
9.84 KB
2.68 KB

Good idea!

Note: This isn't working yet.

andypost’s picture

After reviewing this issue after #2242947: Integrate Symfony Console component to natively support command line operations
I see 2 different ways to boostrap core & checks for ability to do it.
Mostly polished class Application extends GenericApplication (generic is SF one)
So if install command will fail we just get NULL and call to start dev server without checking "bootstrapability" of core
Making both host:port to listen configurable
Maybe parts of .htrouter could be reused
Probably #2289405: [META] Port all shell scripts to console commands is better place to discus bootstrap & error handling

  1. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,165 @@
    +      $output->getErrorOutput()->writeln('You need to have sqlite installed.');
    +      return;
    ...
    +        $output->getErrorOutput()->writeln('Removing settings.php failed, please do it manually ...');
    +        return;
    ...
    +        $output->getErrorOutput()->writeln('Removing .sqlite failed, please do it manually ...');
    +        return;
    
    +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,101 @@
    +      throw new \RuntimeException('Unable to find the PHP binary.');
    

    all this places reports to err and return non-zero (or exception?) that's what raised in 2242947#98

    let's came to common ground for all commands to be POSIX in #2289405-21: [META] Port all shell scripts to console commands

  2. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,101 @@
    +  }
    +  ¶
    

    nit, trailing whitespace

  3. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,101 @@
    +   * Boots up a Drupal environment.
    ...
    +  protected function boot() {
    +    $root = dirname(dirname(dirname(dirname(dirname(__DIR__)))));
    +    chdir($root);
    +    return $root;
    

    chdir() does not boot core, maybe better call it chroot5Dirs()

    Actually this is what #2242947-96: Integrate Symfony Console component to natively support command line operations means that it can catch by tests - so could be a problem if this method will be called twice

  4. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,101 @@
    +      'localhost:' . $port,
    ...
    +    $output->writeln('Starting webserver on http://localhost:' . $port);
    

    Why only port configurable? host should be as well

  5. +++ b/core/scripts/dev-site.php
    @@ -0,0 +1,18 @@
    +$application = new Application('dev-start', '0.1.0');
    +$application->add(new DevInstallCommand($classloader));
    +$application->add(new DevStartCommand($classloader));
    +$application->run();
    

    Please, make separate commands) It will be more useful cos I don't wanna reinstall site each time I wanna run dev server

andypost’s picture

#33.5 Maybe option --listen with host (default to localhost) and port (8080 default) may pipe dev server command

PS: That could a good path to https://github.com/php-pm/php-pm-drupal

andypost’s picture

btw SF using https://github.com/symfony/symfony-installer/blob/master/symfony to check requirements and start server installer(demo)

andypost’s picture

+++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
@@ -0,0 +1,101 @@
+    $address = '0.0.0.0';
+    $port = 0;
+    $socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
+    socket_bind($socket, $address, $port);
+    socket_listen($socket, 5);
+    socket_getsockname($socket, $address, $port);

I bet it fails because you bind/listen the same port (on 0.0.0.0) so webserver can't start cos port already used by other process

dawehner’s picture

Issue summary: View changes

Please, make separate commands) It will be more useful cos I don't wanna reinstall site each time I wanna run dev server

This is what this patch is doing right now. I updated the issue summary.

Maybe parts of .htrouter could be reused

Yes indeed, the start command is using that internally.

Making both host:port to listen configurable

Right now the idea is: Let's generate an open port automatically. Because this command is not designed for any production usecase, knowing the port upfront isn't needed? (Maybe there are good other usecases though, do you mind giving some)

Why only port configurable? host should be as well

Please keep in mind, this command is intended for local development / trying out. It is not meant for anything near production.

Chi’s picture

@dawehner, tt's pretty common case using something different form localhost on local machine, especially with Docker based environment.

dawehner’s picture

@Chi
Mh, well, at least in my thinking, when you are in a docker based setup, you have the level of knowledge to setup a webserver as well? I don't want to argue that we shouldn't add it in the future, I just want to focus on one particular usecase in this issue. I also don't think we should block the work on #2242947: Integrate Symfony Console component to natively support command line operations. Internal refactoring is something we can always do later.

mglaman’s picture

I bet it fails because you bind/listen the same port (on 0.0.0.0) so webserver can't start cos port already used by other process

What fails? This is how you find an available port in PHP to bind to. See #30 where I run the command twice and receive ports that do not collide.

mglaman’s picture

@dawehner, tt's pretty common case using something different form localhost on local machine, especially with Docker based environment.

This boots the builtin server. I think we can punt that to avoid scope bloat. The fac remains this gives new users a one time command to run that mimcs `npm start`

mglaman’s picture

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -49,9 +56,29 @@ protected function configure() {
+    $request = Request::createFromGlobals();
+    DrupalKernel::createFromRequest($request, $this->classLoader, 'prod');
+
+    return ($connection = Database::getConnection()) && $connection->schema()->tableExists('sessions');

What if we just followed install.php

    // Do not install over a configured settings.php.
    if (Database::getConnectionInfo()) {
      throw new AlreadyInstalledException($container->get('string_translation'));
    }

If there is a settings.php and a default connection defined, require forced install.

jmolivas’s picture

@kenorb DrupalConsole provides a similar command named quick:start

Actually, this command is something we called a chain command
A chain command is a custom command that helps you automate multiple command execution, allowing you to define and read an external YAML file containing the definition name, options, and arguments of multiple commands and execute that list based on the sequence defined in the file.

DrupalConsole documentation about chain commands => https://docs.drupalconsole.com/en/chains/what-is-a-chain-command.html

YAML code is here:
https://github.com/hechoendrupal/drupal-console-core/blob/master/config/...

The previous definition will execute several commands, in this case, commands that will download and install Drupal using SQLite, and finally start the PHP's built-in server.

dawehner’s picture

Issue summary: View changes
borisson_’s picture

Issue summary: View changes
mglaman’s picture

Assigned: Unassigned » mglaman

I have an incoming patch to change the default behavior to N otherwise the dev script runs kind of weird. Also to highlight the default option and simplify install check.

$ php core/scripts/dev-site.php  install
Drupal installation started.
Drupal successfully installed.
$ php core/scripts/dev-site.php  install
There is already an existing installation. Confirm whether you want to override it. (Y/n) 
Drupal installation started.
Drupal successfully installed.
$ php core/scripts/dev-site.php  install
There is already an existing installation. Confirm whether you want to override it. (Y/n) n

EDIT: This is why to change default

$ composer run-script dev-site
> core/scripts/dev-site.php install && core/scripts/dev-site.php start
Removing settings.php failed, please do it manually ...
Starting webserver on http://localhost:51928

mglaman’s picture

Assigned: mglaman » Unassigned
Status: Needs work » Needs review
FileSize
1.19 KB
9.79 KB

Here is the patch which changes the default confirmation answer so there isn't a reinstall when executing dev-start.

Executing composer run-script dev-start is like running npm start on a JavaScript project. You get an installed and running environment with minimal work. There should not be any need to configure the host or specify the port. The goal is to one and done get an evaluation up and running.

Chi’s picture

@mglaman can you check #26, some of this notes are not addressed yet. Also #12.

mglaman’s picture

Status: Needs review » Needs work

Sorry, missed those Chi.

+++ b/core/scripts/dev-site.php
@@ -0,0 +1,18 @@
+$application = new Application('dev-start', '0.1.0');
+$application->add(new DevInstallCommand($classloader));
+$application->add(new DevStartCommand($classloader));
+$application->run();

Needs check to ensure CLI.

mglaman’s picture

Status: Needs work » Needs review
FileSize
3.74 KB
9.85 KB

This addresses items in #12 and #26 (latter to best I could find remaining items.)

dawehner’s picture

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -76,7 +76,7 @@ protected function isDrupalInstalled() {
-    return ($connection = Database::getConnection()) && $connection->schema()->tableExists('sessions');
+    return !empty(Database::getConnectionInfo());

Are you sure this is the right way? I'd like to be able to install into an empty database

mglaman’s picture

What are the chances the database would be empty if there data connection string was available? That's the check that core does on its install (at least from what I could see in install_begin_request.) The session table is a fair signal, but this can be a bit safer in case the database is that bad (has sessions table, but not others.)

EDIT: You can still install on an empty database, you just need to force.

Chi’s picture

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -22,19 +21,19 @@ class DevInstallCommand extends Command {
+   * @var \Composer\Autoload\ClassLoader
...
+   * @param object $class_loader

Note that per #28 the object type hint was chosen intentionally. Personally, I cannot think of any use case with different autoloader in this command. So I would prefer using full class name for type hinting. Same type is used in upcoming patch in #2242947: Integrate Symfony Console component to natively support command line operations.

However, in either case we should use only one type hint across this file for the purpose of consistency. Currently both of them are used.

mglaman’s picture

I also found something else while using this for a Drupal Commerce demo. When my example profile had Bootstrap it exploded.

Adding define('MAINTENANCE_MODE', 'install'); fixed it. We're currently not defining the maintenance mode, as expected on install, to install

dawehner’s picture

Assigned: Unassigned » dawehner

Working on tests right now.

dawehner’s picture

This is a start to write some tests.

alexpott’s picture

Just cleaning up the whitespace errors in patch so it applies clean.

alexpott’s picture

Hmmm I need a version of this patch with no root composer changes. Ignore this patch.

alexpott’s picture

+++ b/composer.json
@@ -54,6 +54,7 @@
+        "dev-site": "core/scripts/dev-site.php install && core/scripts/dev-site.php start",

this should be "dev-site": "@php docroot/core/scripts/dev-site.php install && @php docroot/core/scripts/dev-site.php start",

alexpott’s picture

Hmmm... actually this works:

        "dev-site": [
            "@php docroot/core/scripts/dev-site.php install",
            "@php docroot/core/scripts/dev-site.php start"
        ],
danquah’s picture

Two quick questions (and I apologise if I've missed the answer for them in the comments)

1: core/scripts/dev-site.php seems to need to be executable before composer run-script will execute it. Is making it executable something that happens during a build of core prior to distribution or is it something that should be done as a part of this patch?

2: I can't see that the comment regarding setting a username and password in #11 was actually address, is it work that needs doing? (I'd be happy to pitch in)

danquah’s picture

Ok, I should obviously have read the #60 with regards to my first question :)

As penance I've verified that the approach does indeed work - dev-site.php does not need to be executable if we run it using @php

Patch and interdiff attached.

dawehner’s picture

Nice, I had no idea this existed. Thank you @danquah for the patch!

deviantintegral’s picture

Issue summary: View changes
deviantintegral’s picture

Status: Needs review » Needs work
+  protected function getAvailablePort() {
+    $address = '0.0.0.0';

Should this be 127.0.0.1 to match that we bind to localhost below? Otherwise, could we be grabbing an available port on the wrong interface?

Comment #11 is visible for me - if not, it notes that there's no way to actually log in to the installed site. Rather than showing a user name and password, we could print a one-time login with the destination set to <front>?

I also got the following exception after leaving the server open for a few minutes:

[Symfony\Component\Process\Exception\ProcessTimedOutException]
The process "'/usr/local/Cellar/php/7.2.3/bin/php' -d allow_url_fopen='1' -d disable_functions=''
-d memory_limit='1536M' core/scripts/dev-site.php start" exceeded the timeout of 300 seconds.

mglaman’s picture

Should this be 127.0.0.1 to match that we bind to localhost below? Otherwise, could we be grabbing an available port on the wrong interface?

Everything I've read, and when using this, says to bind to 0.0.0.0 which is your host.

if not, it notes that there's no way to actually log in to the installed site.

I tried looking at #11 but don't see where it says there is no way to log in. It spits out the localpost:PORT. However, I do notice see we do not mention the username and password. I feel like generating a one-time login link just adds more scope bloat. I'd rather just print the credentials.

I also got the following exception after leaving the server open for a few minutes:

Ditto. I don't know how we can work around this. You'd have to call Composer with its timeout flag. See the following

$ composer run --help
Usage:
  run-script [options] [--] [<script>] [<args>]...

Arguments:
  script                         Script name to run.
  args                           

Options:
      --timeout=TIMEOUT          Sets script timeout in seconds, or 0 for never.
dawehner’s picture

[Symfony\Component\Process\Exception\ProcessTimedOutException]
The process "'/usr/local/Cellar/php/7.2.3/bin/php' -d allow_url_fopen='1' -d disable_functions=''
-d memory_limit='1536M' core/scripts/dev-site.php start" exceeded the timeout of 300 seconds.

Oh yeah, let's fix that in the composer.json!

This now also prints out a one-time login link.

mglaman’s picture

Nice! I eat my words about login link. That was much simpler than I thought.

alexpott’s picture

+++ b/composer.json
@@ -14,7 +14,8 @@
+        "process-timeout": 0

This looks a bit dodgy :)

webchick’s picture

Project: Drupal core ideas » Drupal core
Version: » 8.6.x-dev
Component: Idea » install system
Priority: Normal » Major

Hm. Not sure why this is in the ideas queue. :)

Also bumping to "major" since it's a blocker for JS testing.

Chi’s picture

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -50,44 +48,35 @@ protected function configure() {
-   * Boots up a Drupal environment.
-   *
-   * @param \Symfony\Component\Console\Output\OutputInterface $output
-   *   The console output.
+   * Changes the directory to the Drupal root.
    *
    * @return string
    *   Returns the path to the Drupal root.
    */
-  protected function boot(OutputInterface $output) {
-    // Installs drupal using sqlite.
-    if (!extension_loaded('pdo_sqlite')) {
-      $output->writeln('Sqlite required ...');
-      return;
-    }
-
+  protected function changeRoot() {

This change needs to be applied to the DevStartCommand as well.

dsnopek’s picture

Should this be 127.0.0.1 to match that we bind to localhost below? Otherwise, could we be grabbing an available port on the wrong interface?

Everything I've read, and when using this, says to bind to 0.0.0.0 which is your host.

I think trying to bind to a port on 0.0.0.0 will fail if any interface on the server is using that port. So, this shouldn't end up in a situation where getAvailablePort() gives a port that's not actually available on 127.0.0.1. If anything, it'll skip ports that are available on 127.0.0.1 if they are used on other interfaces, which shouldn't cause any problems.

andypost’s picture

Hey why IPv4 only? There's IPv6 so this indeed should be compatible with it.
PHP did fix that in https://bugs.php.net/bug.php?id=68420

+++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
@@ -0,0 +1,101 @@
+    $address = '0.0.0.0';
...
+      'localhost:' . $port,

+++ b/core/tests/Drupal/Tests/Core/Command/DevSite.php
@@ -0,0 +1,43 @@
+        $response = $guzzle->get('http://localhost:' . $match[1]);

There's total mess in interfaces & hostnames

dsnopek’s picture

Hey why IPv4 only? There's IPv6 so this indeed should be compatible with it.

I mean, we're talking about something that's ultimately listening on localhost. Do you really need to connect to localhost over IPv6? Sure, I guess this means that the available port checking is not including the IPv6 interfaces, but we really only care about is the available ports on localhost.

mglaman’s picture

This is meant to be a quick start command. We do not need to worry about IPv6 or IPv4.

I think trying to bind to a port on 0.0.0.0 will fail if any interface on the server is using that port.

On the docs: http://php.net/manual/en/function.socket-getsockname.php. Port is handled by reference to be updated to the actual port it ended up binding to. As mentioned in https://www.drupal.org/project/drupal/issues/2911319#comment-12490747 I have used this to run 30-40 concurrent user load tests using ChromeDriver on random ports.

Per #73 what is the problem? localhost maps to 127.0.0.1 which is where the port binds.

This seems to be a concern by many, but not reproduced in error or steps to reproduce. I have been using that line for quite a while, which is why I suggested it.

The scope of the command is to just provide a quick development server and SQLite install for a demo purpose.

dawehner’s picture

@andypost @dsnopek @deviantintegral
Can someone of you make a suggest what we do otherwise? I would love the functionality to find an open port automatically as it improves the user experience.

dsnopek’s picture

RE: @mglaman in #75, I think you're misunderstanding what I was trying to say, and I guess, I probably said it poorly. :-)

I was trying to say that the code in getAvailablePort() would fail to bind on any already in use port on any interface, ie. it would find a port that was free across any interface (well, any IPv4 one ;-)) on the machine. So, just that that code is being extra paranoid, and so wouldn't cause problems with the fact that in the end the server is being run on localhost.

I think we could change getAvailablePort() to just check '127.0.0.1' rather than '0.0.0.0' which would only look for a port that was free specifically on localhost, but what's the point? The current code is more paranoid than necessary, not less, so that discrepancy shouldn't cause any problems. That's the point I was trying to make.

Can someone of you make a suggest what we do otherwise?

As stated above in #72 and #74, I think what the code is currently doing is fine.

EDIT: Actually, to try and clarify even more: when I'm writing above (and in #72) that it will "fail to bind" I'm referring specifically to the socket_bind() call in this code in the patch:

+  protected function getAvailablePort() {
+    $address = '0.0.0.0';
+    $port = 0;
+    $socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
+    socket_bind($socket, $address, $port);
+    socket_listen($socket, 5);
+    socket_getsockname($socket, $address, $port);
+    return $port;
+  }

So, I'm not referring to ultimately running the server failing or anything like that. Just that the socket_bind() above would not succeed if it were trying a port that was in use on any IPv4 interface.

Whew. Writing words is hard. :-) Sorry for not being clear the first time around!

dawehner’s picture

Assigned: dawehner » Unassigned
geerlingguy’s picture

Just wanted to add notes on how to test the current patch quickly in a fresh environment (with no Drupal or anything else), for my (and others?) reference:

  1. Ensure you have PHP 7+, Git, and Composer installed.
  2. Run git clone --branch 8.6.x https://git.drupal.org/project/drupal.git && cd drupal
  3. Apply the latest patch: wget -q -O - https://www.drupal.org/files/issues/2911319-67.patch | git apply -
  4. Run composer install
  5. Run composer dev-site
  6. Grab the URL (e.g. localhost:XXXXX) and append the 'Login using' string to the end.

One minor nit from a UX perspective: What if we make the entire login URL copy-pasteable? So instead of a new user having to realize they copy out the localhost:XXXXX part, then copy the user login path, then merge the two together to get logged in, they can just copy out the full URL and get logged in immediately?

Or provide both, but the login URL prefixed with the http://localhost:XXXXX part, just to make it easier to do the copy/paste. OR, final suggestion would be to do what Drush does and just provide a random admin password, e.g.:

[notice] Starting Drupal installation. This takes a while.
[success] Installation complete.  User name: admin  User password: nFoGTR6xEb
scotty’s picture

At beginners sprint at midcamp, working on issue.

scotty’s picture

Issue tags: +midcamp2018

We found a typo in the wget patch URL in #79. It is missing "/issues" after "files". Resummarizing, the instructions should read:

  1. Ensure you have PHP 7+, Git, and Composer installed.
  2. Run git clone --branch 8.6.x https://git.drupal.org/project/drupal.git && cd drupal
  3. Apply the latest patch: wget -q -O - https://www.drupal.org/files/issues/2911319-67.patch | git apply -
  4. Run composer install
  5. Run composer dev-site
  6. Grab the URL (e.g. localhost:XXXXX) and append the 'Login using' string to the end.
ressa’s picture

@scotty: It looks like "/issues" after "files" wasn't added in the correction :-)

scotty’s picture

Thanks, just edited my comment :)

droffats’s picture

Notes: From Mid-camp.

OSX doesn't come with wget out of the box. Had to install Homebrew and use it to install wget.

Upon running composer install we got a number of OpenSSL failures. For instance:

Installing instaclick/php-webdriver (1.4.5): Downloading (failed)       
Downloading (failed)       
Downloading (failed)    Failed to download instaclick/php-webdriver from dist: The "https://api.github.com/repos/instaclick/php-webdriver/zipball/6fa959452e774dcaed543faad3a9d1a37d803327" file could not be downloaded: SSL operation failed with code 1. OpenSSL Error messages:
error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
Failed to enable crypto
failed to open stream: operation failed
    Now trying to download from source
  - Installing instaclick/php-webdriver (1.4.5): Cloning 6fa959452e from cache
> Drupal\Core\Composer\Composer::vendorTestCodeCleanup

So the files do download from cache, but it looks like it's failing but it does actually download and install... It would be good to re-run the test from a clean install of OSX.

droffats’s picture

Notes: From Mid-camp.

The site does install and run.

stpaultim’s picture

I just updated to PHP 7.1 on my Mac running OSX 10.12.6 and ran through the instructions for testing.

I had some problem with the wget, but once I downloaded and applied the patch, it worked as expected.

It created and installed a version of Drupal 8 and I was able to access the site with the provided URL, login with the supplied temporary login, and edit a node.

I am working with @scotty, @droffats, and @ex DJ at MidCamp2018!

mradcliffe’s picture

@droffats, @scotty, @ex DJ

I think the next steps are

1. Update the issue summary (How to test) to include the steps that @scotty posted in #81 with any additional gotchas in #85.
2. The issue summary has a remaining task of ensuring that the dev-site composer script doesn't "kill" an existing database. So what that means is that you should install Drupal 8 through the usual install.php process on your local environments MySQL and SQLite. Then run composer dev-site and ensure that you are still able to access the original site you installed.
3. Finally the patch in #67 needs instructions for users in INSTALL.txt. So the patch needs to be modified and re-uploaded. Ensure that you create an interdiff of your patch with #67 if you are doing this.

mradcliffe’s picture

Status: Needs review » Needs work

Setting to needs work for Documentation changes needed in INSTALL.txt.

droffats’s picture

Issue summary: View changes
droffats’s picture

I edited the installed drupal site, Then re-ran

composer dev-site

and confirmed that it did not over-write the existing .sqlite db.

ex DJ’s picture

Notes from MidCamp.

It might be helpful to add a comment to the instructions, noting what kind of Drupal user should be using this command; i.e., intended as a quick way to get Drupal running for a basic introduction, not for serious developers.

With @droffats, @scotty, and a BIG assist from @stpaultim and @mradcliffe

mradcliffe’s picture

Issue summary: View changes
ressa’s picture

Thanks to everybody, great work! I went through these steps for Ubuntu 16.04 to be able to install Drupal 8 with the steps outlined in #81:

Install Git and required PHP packages, and check it went well

$ sudo apt-get install git php7.0 php7.0-curl php7.0-dom php7.0-gd php7.0-mbstring php7.0-sqlite3
$ php --version
$ git --version

Download, move and check if Composer is installed correctly

$ curl -sS https://getcomposer.org/installer | php;
$ sudo mv composer.phar /usr/local/bin/composer;
$ composer -V
mglaman’s picture

Setting to needs work for Documentation changes needed in INSTALL.txt.

Why would we need to change INSTALL.txt? This command is for evaluation and not a legitimate installation setup.

mradcliffe’s picture

The issue summary says that the plan for this issue has the following, @mglaman:

> Communicate that as the simplest way to try out Drupal on your system by adding documentation to INSTALL.txt

Original issue as reported by @dawehner had "Communicate that as the simplest way to try out Drupal on your system" in the Detailed plan section. @alexpott added that requirement in #9 saying that we should add this to core documentation and changed the issue summary to add INSTALL.txt.

Is that still relevant?

Edit: for grammar.

karolus’s picture

Know it may be a tall order, but something along the lines of Dev Desktop is a good onboarding method for non-developers to get acquainted with Drupal. It's easy for experienced people to forget how intimidating the command line and associated processes can be for some.

It's part of the reason Drupal has had problems expanding into creative fields, where people there tend to gravitate to WordPress for apparent ease-of-use.

Yes, a packaged launch/install method may not be the best for launching industrial-strength projects, but is a quick way to people to spin up an install to try things out.

geerlingguy’s picture

It's part of the reason Drupal has had problems expanding into creative fields, where people there tend to gravitate to WordPress for apparent ease-of-use.

Look at the node.js community; everyone there uses npm on the command line to at least install/start/stop a project. I don't think it's too much to ask someone to run a few CLI commands in 2018, especially if this way they don't need an extra bunch of software (Apache and MySQL) installed on their computer.

karolus’s picture

@geerlinguy, true, but that's making some assumptions about users--1) their system doesn't have older/conflicting installs, and 2) they're comfortable with the CLI. Having helped out onboard users onto Drupal over the years, I've seen some pretty head-scratching issues, which can require some in-depth problem solving and fixing to get a stack stood up.

Matthew Grasmick had recently posted on this. Some of the casual/non-technical evaluators may be the ones making the business decision of whether to go Drupal or not. It can look pretty simple for someone with technical chops, even if snags are hit--again, this is where past experience helps ease the troubleshooting.

Even for non-code projects, like a data-analytics app I'm working on--ease of use is a critical consideration in a competitive market.

ressa’s picture

It's easy for experienced people to forget how intimidating the command line and associated processes can be for some.

@karolus: I couldn't agree more, and that's why I posted the commands required to get Ubuntu 16.04 ready for Drupal 8 single command installation in #93.

Perhaps someone could check if the commands work for Mac as well, or they need tweaking?

I am now so comfortable with the CLI that I can wipe my machine for everything PHP- and Composer-related, and reinstall the necessary programs, just to see what the bare minimum is. The newbie-me would have taken forever to just get to the point where I could actually run the single commands which installs Drupal, or might never have gotten that far.

Let's help non-developers get acquainted with Drupal, as much as we possibly can.

quietone’s picture

Really nice to see the progress here and thank you to everyone.

However, I don't see an un-install option. Am I missing it or is that not a consideration?

dawehner’s picture

However, I don't see an un-install option. Am I missing it or is that not a consideration?

Do you mind opening up a new issue for that? I think its worth discussing but I don't think we need this option right from the start.

@ressa
You could massively simplify this by using lamp-server and maybe the downloaded package.

ressa’s picture

@dawehner
Ubuntu should be ready for single command install with three lines:

$ sudo apt-get install git php7.0 php7.0-curl php7.0-dom php7.0-gd php7.0-mbstring php7.0-sqlite3
$ curl -sS https://getcomposer.org/installer | php;
$ sudo mv composer.phar /usr/local/bin/composer;

That is a fairly short process, but if your suggested lamp-server solution is faster and easier to set up, we should of course use that, if it reduces the barrier of introduction to Drupal.

Would you care to share the steps involved?

I don't want to pollute this issue with irrelevant discussions, but I feel that getting a new Drupal user ready to actually send off the single command which installs Drupal is as important as the single command solution being built here.

geerlingguy’s picture

Having helped out onboard users onto Drupal over the years, I've seen some pretty head-scratching issues, which can require some in-depth problem solving and fixing to get a stack stood up.

I'm just saying that no matter what option we choose, there will be head-scratching issues. At MidCamp last week, I was trying to help someone on a Windows laptop figure out why a fresh install of ADD2 on a new Windows 10 Pro machine had an error message while adding a Drupal site that a zip file couldn't be expanded. After 10 minutes neither of us could figure it out (and I had to run to a session :( ).

At least if we use PHP (and maybe Composer) we limit the amount of dependencies and permissions required to almost bare minimum :)

My ultimate end goal (not saying this is "Drupal.org's end goal", just mine) is that we have something akin to a "quick start guide" for Drupal, and it's literally:

  1. Download Drupal
  2. Run Drupal with some command
  3. Now you're running Drupal!

Which this issue achieves pretty handily. Again, comparing more to Node.js, Go, Python, Symfony, Laravel, etc. communities, this is super standard. I don't know if we want to compare Drupal to Wordpress anymore—we jumped the shark to using Composer, giant dependency trees, modern PHP etc., and we live with that (or own it).

dawehner’s picture

Status: Needs work » Needs review

Download Drupal
Run Drupal with some command
Now you're running Drupal!

Yeah! I hope we do indeed achieve the same. I guess instead of telling people to run composer we could also point them to a dedicated file running the two commands.

Setting to needs work for Documentation changes needed in INSTALL.txt.

I think we cannot do that right now until we aren't down the the three steps layed out by @geerlingguy. Does this make sense?

mradcliffe’s picture

That makes sense. I noticed that the biggest difficulty for getting this setup wasn't the patch or the composer command, but 1) applying the patch (getting a wget/curl with OpenSSL support) and 2) getting the prerequisites installed (PHP and Composer).

But I also just noticed the test fail in #67 too so it's NW anyway. :*(

heddn’s picture

re #109: I personally prefer to use wget myself, but curl is much more universally available. Curl is a requirement for PHP and Drupal. So perhaps we should write any instructions with that in mind and use curl in future examples.

dawehner’s picture

Getting the tests running is a bit challening. Here is some intermediate progress on it.

yoroy’s picture

@dawehner wondered if this needed product manager feedback. Not sure, but this is an outstanding idea of course :)
I see this pattern of "3 commands in the terminal to running X" everywhere as well and it would be great to provide the same for Drupal. Great in the sense of catching up with the rest of the world but more importantly as something that would enable us to clean up and focus our onboarding materials (guides, documentation etc).

I personally prefer to use wget myself, but curl is much more universally available. Curl is a requirement for PHP and Drupal. So perhaps we should write any instructions with that in mind and use curl in future examples.

This is great. Would be good to make any necessary decisions with this perspecitive in mind.

mradcliffe’s picture

I think it's okay that PHP + composer is a requirement before the 3 commands as that is the same exact situation as Node.js. You need node.js to run npm projects. It's sad that PHP doesn't have a native packaging utility bundled like node.js, but that used to be the case with node.js a few years ago. I don't think that getting PHP + composer should be considered one of the 3 commands in that case.

Also composer install is only necessary when testing this issue because the intended use case is someone downloads a Drupal 8.6.0 release, then they can run composer dev-site directly, which saves one step. :-)

So

A. Get and install PHP.
B. Get and install Composer.

1. Download Drupal 8.6.0
2. Extract drupal-8.6.0
3. Open terminal
4. Change directory to drupal-8.6.0
5. composer dev-site
6. Copy URL.
6. Open web browser and paste URL into address bar.

That's 2 commands (cd and composer) in the terminal.

Edit: the following is probably too much bike shedding. Sorry.

+    $process = new Process([
+      $binary,
+      '-S',
+      'localhost:' . $port,
+      '.ht.router.php',
+    ], $root, NULL, NULL, NULL);
+    $output->writeln('Starting webserver on http://localhost:' . $port);
+    $process->run();

Could we write output after this to mention that the command succeeded? That was also confusing when we were testing. We didn't think it was "done" or ready and that the command had stalled out so we killed the process.

💧 Your site is ready for you at URL

On my linux desktop that droplet isn't going to render (and maybe not on Windows either), but emojis are friendly, right?.

+  protected function install($class_loader, ConsoleOutputInterface $output, $profile = 'standard', $langcode = 'en', $site_path = NULL) {

+    $output->writeln('Drupal successfully installed.');
+
+    $user = User::load(1);
+    $one_time_login = user_pass_reset_url($user);
+    $output->writeln("Login using $one_time_login");
+  }

And maybe this could be done so that the Login using is printed after the web server is started? I know that install/start are distinct, but is there ever a reason not to generate a one-time login URL during start if I'm evaluating or trying to get started quickly? I think I would want to be in the administration UI to start exploring or making content right away.

Also to consider, angular cli, drush and webpack dev servers can start a browser on some devices. I personally find this annoying, but others find it useful and it would eliminate the final step of navigating / copying and pasting the URL into the browser.

Mixologic’s picture

Also to consider, angular cli, drush and webpack dev servers can start a browser on some devices

Composer has this built into it too:
composer browse drupal/examples takes you to the cgit repo and
composer browse -H drupal/examples takes you to the project page in your system configured browser.

Seems relatively simple to do: https://github.com/composer/composer/blob/685ff8699ba378cc815849fb7daa8d...

quietone’s picture

Added followup to create an uninstall command as suggested in #101. #2953810: Provide a single command to un-install after installing with single command

dawehner’s picture

I added support for opening up the one time login URL in the browser automatically.

cashwilliams’s picture

Status: Needs review » Needs work

Testing with #116 -

Browser does open with one time login link.

However, I get WSOD on quite a few pages, such as /, /admin/content, /admin/reports/dblog. This could be 8.6 issue, or an issue with this patch, not sure.

Debugging this isn't easy as no logs are printed to the screen after the server is started. Would it be possible to have logging output the same way `drush run-server` does?

cashwilliams’s picture

Status: Needs work » Needs review

I tested same patch applied code base and it seems to be a problem with 8.6, not this patch. So marking this back for review.

However, just to show my point for consideration -

dev-site output while getting WSOD pages:

> @php core/scripts/dev-site.php install
> @php core/scripts/dev-site.php start
Starting webserver on http://localhost:50057
^C

drush run-server output while getting WSOD pages:

HTTP server listening on 127.0.0.1, port 8889 (see http://127.0.0.1:8889/), serving site default, logged in as admin...
PHP 7.1.14 Development Server started at Sun Mar 18 07:44:39 2018
Listening on http://127.0.0.1:8889
Document root is /Users/cash/Code/Drupal/scratch/d86
Press Ctrl-C to quit.
[Sun Mar 18 07:44:41 2018] Uncaught PHP Exception Drupal\Core\Field\FieldException: "Attempt to create a field user_picture that does not exist on entity type user." at /Users/cash/Code/Drupal/scratch/d86/core/modules/field/src/Entity/FieldConfig.php line 308
[Sun Mar 18 07:44:41 2018] 127.0.0.1:50256 [500]: /
[Sun Mar 18 07:44:43 2018] Uncaught PHP Exception Drupal\Core\Field\FieldException: "Attempt to create a field user_picture that does not exist on entity type user." at /Users/cash/Code/Drupal/scratch/d86/core/modules/field/src/Entity/FieldConfig.php line 308
[Sun Mar 18 07:44:43 2018] 127.0.0.1:50258 [500]: /
dawehner’s picture

@cashwilliams
Do you think adding logs could be added in a follow up? There are so many things we could do :) I prefer though to get something done, and then iterating on top of that.

alexpott’s picture

Patch attached fixes the dev start command so that the webserver is started before the browser opens (and also outputs all the output from php -s as per #118 because I was mucking around with the process object).

It also automatically logs the user in to save a button click.

It also removes the global composer timeout setting in favour of something specific for the dev-site command.

It also does a couple of coding standards cleanups.

heddn’s picture

Here's a hack at the INSTALL.txt docs. It was the minimum instructions that still used Curl. It defaulted to using unzip as that is more readily available on Windows.

I also changed the install message. It was a bit jarring to just see a "wait" message without any expectation what would happen next.

That should mean this can be reviewed for its final time as all the key points in the IS are now covered.

alexpott’s picture

Here's a patch with a test that works. After implementing the test I'm wondering if we can't share a lot of core with #2926633: Provide a script to install a Drupal testsite since that install command seems to be a very special case of what we're implementing here.

alexpott’s picture

Status: Needs work » Needs review
FileSize
944 bytes
23.06 KB

Hmmm that test passes locally. Let's see if we can work out what is going on on the bot.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -163,6 +161,7 @@ protected function install($class_loader, ConsoleOutputInterface $output, $profi
     }
+    copy("sites/default/default.settings.php", "{$site_path}/settings.php");
     if (file_exists("{$site_path}/files/.sqlite")) {

Oh I see :) This makes sense.

  1. +++ b/core/INSTALL.txt
    @@ -10,6 +11,29 @@ CONTENTS OF THIS FILE
    +
    +Prerequisites: Ensure you have PHP 5.5.9 (or greater) and Composer installed.
    +
    

    Can we link to composer, in case someone doesn't know what it is?

  2. +++ b/core/INSTALL.txt
    @@ -10,6 +11,29 @@ CONTENTS OF THIS FILE
    +- unzip drupal-8.x.x.zip
    

    Can we document windows somehow here too?

  3. +++ b/core/INSTALL.txt
    @@ -10,6 +11,29 @@ CONTENTS OF THIS FILE
    +NOTE: This quick start solution uses PHP's built-in web server and is not
    +intended for production use.
    

    Maybe we could drop a little comment like "Read more about running Drupal in the next sections".

  4. +++ b/core/includes/install.core.inc
    @@ -285,8 +285,12 @@ function install_state_defaults() {
    +  * @param $settings
    + *   An optional array of installation settings.
    

    Nitpick, this is an array, isn't it?

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.98 KB
23.37 KB

Fixing test coverage and addressing #126 - apart from #126.2 - I have no idea about Windows unzip documentation.

alexpott’s picture

FileSize
4.05 KB

The last interdiff was wrong it should have been.

alexpott’s picture

We don't need so much change to install_begin_request() - the additional parameter is unnecessary.

alexpott’s picture

  1. +++ b/core/includes/install.core.inc
    @@ -324,7 +326,7 @@ function install_begin_request($class_loader, &$install_state) {
    -  $site_path = DrupalKernel::findSitePath($request, FALSE);
    +  $site_path = empty($install_state['site_path']) ? DrupalKernel::findSitePath($request, FALSE) : $install_state['site_path'];
    
    +++ b/core/lib/Drupal/Core/Installer/InstallerKernel.php
    @@ -66,4 +67,15 @@ public function getInstallProfile() {
    +  /**
    +   * {@inheritdoc}
    +   */
    +  public static function createFromRequest(Request $request, $class_loader, $environment, $allow_dumping = TRUE, $app_root = NULL) {
    +    // This override exists because we don't need to initialize the settings
    +    // again as they already are in install_begin_request().
    +    $kernel = new static($environment, $class_loader, $allow_dumping, $app_root);
    +    static::bootEnvironment($app_root);
    +    return $kernel;
    +  }
    

    For reviewers, these are the changes to the installer to support the new command. They really exist only for testing so that we can install to a path other than sites/default.

  2. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,178 @@
    +      ->addOption('site_path', NULL, InputOption::VALUE_OPTIONAL, 'Forces to use a specific site directory.')
    
    +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,199 @@
    +      ->addOption('site_path', NULL, InputOption::VALUE_OPTIONAL, 'Forces to use a specific site directory.')
    

    In reality these command options are only really for testing the commands so I think we need to update the docs. And they wouldn't actually work. If we want a user to be able to do this then we need to change \Drupal\Core\DrupalKernel::findSitePath() to allow us to determine the site path from an environmental variable.

alexpott’s picture

Patch attached removes the site_path option since it is not going to work for regular users without changes to DrupalKernel::findSitePath(). Instead we use an environment variable DRUPAL_DEV_SITE_PATH to make the commands testable. We can open a followup to use this same variable in DrupalKernel::findSitePath() and then we could add the options back. This would be neat as you could then demo two sites at the same time (perhaps a before and after).

kenorb’s picture

The following line in core/INSTALL.txt is showing some weird characters.

Wait… installation can take a minute or two.

alexpott’s picture

@kenorb that's a unicode ellipsis... what are you seeing in \Drupal\Core\Asset\CssCollectionOptimizer::optimize() there's a comment in there that uses exactly the same character. We use it all over the place.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
@@ -0,0 +1,191 @@
+    if (file_exists("{$site_path}/settings.php")) {
+      $result = unlink("{$site_path}/settings.php");
+      if ($result === FALSE) {
+        $output->getErrorOutput()->writeln('Removing settings.php failed, please do it manually ...');
+        return;
+      }
+    }

This sadly makes it impossible for composer scripts (like post-install-cmd to add bits to the settings.php).

alexpott’s picture

Patch addresses #134. Also I tried to test the --force option and it just didn't work cleanly. In order to actually delete the sqlite db and to properly re-install you have to delete the entire files directory. So rather than do all of this here I think we should just tell people that the site is installed already and be done. For me that's all that is necessary for a viable patch. If people need a way to re-install a site then we can work on that in followups.

Added test coverage of trying install over the top of an existing site.

alexpott’s picture

With #135 when running the composer run-script dev-site I see:

> Drupal\Core\Composer\Composer::removeTimeout
> @php core/scripts/dev-site.php install
Drupal is already installed.
> @php core/scripts/dev-site.php start
PHP 7.1.10 Development Server started at Wed Mar 21 11:39:59 2018
Listening on http://localhost:62293
Document root is /Volumes/devdisk/dev/sites/drupal8alt.dev
Press Ctrl-C to quit.
[Wed Mar 21 11:40:14 2018] ::1:62336 [200]: /core/assets/vendor/normalize-css/normalize.css?p5wkm7
[Wed Mar 21 11:40:14 2018] ::1:62337 [200]: /core/themes/stable/css/system/components/ajax-progress.module.css?p5wkm7
[Wed Mar 21 11:40:14 2018] ::1:62338 [200]: /core/themes/stable/css/system/components/align.module.css?p5wkm7
// ... and all the rest.

I think this looks good and is working as expected. The one thing that would be nice is if the composer command passed the arguments on the install command so you could select the profile. Or maybe if there was no profile argument it asked.

alexpott’s picture

Composer doesn't allow us to interact with the user https://github.com/composer/composer/issues/5856 :(

I'm not sure we should provide a composer command.

dawehner’s picture

Do you think it would be still helpful to run both after each other, without requiring people to know the intermediate steps?

alexpott’s picture

@dawehner totally! One thing I was thinking about is are the two commands actually necessary and whether dev-site was the right name. I think a single quick-start command might be a good idea. This command would install (if necessary) and start up the PHP web server.

dawehner’s picture

@alexpott
Well, I'd like to be able to install modules / do other bits like services.yml changes before starting up Drupal.

phenaproxima’s picture

Status: Needs review » Needs work

I really dig this patch, man. I sense much win within, and couldn't find any serious problems. Only random questions and nits.

  1. +++ b/core/INSTALL.txt
    @@ -10,6 +11,31 @@ CONTENTS OF THIS FILE
    +Wait… installation can take a minute or two.
    

    "Wait" is followed by a smart ellipsis. Can it be three periods instead?

  2. +++ b/core/INSTALL.txt
    @@ -10,6 +11,31 @@ CONTENTS OF THIS FILE
    +- A successful installation will result in logging in to your new site in your
    +  web browser.
    

    Why is this separated in a list item? It seems like it could come right after "...take a minute or two."

  3. +++ b/core/includes/install.core.inc
    @@ -324,7 +326,7 @@ function install_begin_request($class_loader, &$install_state) {
    +  $site_path = empty($install_state['site_path']) ? DrupalKernel::findSitePath($request, FALSE) : $install_state['site_path'];
    

    Nit: I think we could shorten the line by using the ?: construction.

  4. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,171 @@
    +      ->addOption('install_profile', NULL, InputOption::VALUE_OPTIONAL, 'Install profile to install the site in. Defaults to standard', 'standard')
    +      ->addOption('langcode', NULL, InputOption::VALUE_OPTIONAL, 'The language to install the site in. Defaults to en', 'en')
    

    The descriptions should probably end with periods.

  5. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,171 @@
    +      ->addOption('site_name', NULL, InputOption::VALUE_OPTIONAL, 'Set the site name.', 'Drupal')
    

    This should say what the default value will be.

  6. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,171 @@
    +    if (!extension_loaded('pdo_sqlite')) {
    +      $output->getErrorOutput()->writeln('You need to have sqlite installed.');
    +      return;
    +    }
    

    It seems to me that we should allow a user to specify a database URL, in case they don't have SQLite or don't want to use it. But we should continue to default to SQLite. Maybe that's more follow-up material, though.

    Also, the error message could be more helpful. Maybe it could include a URL on php.net explaining how to install the extension?

  7. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,171 @@
    +    if (!file_exists($site_path)) {
    

    is_dir() might be preferable here.

  8. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,171 @@
    +      mkdir($site_path, 0775);
    +    }
    +    if (!file_exists("{$site_path}/settings.php")) {
    +      copy("sites/default/default.settings.php", "{$site_path}/settings.php");
    +    }
    

    We should probably throw exceptions and fail if copy() or mkdir() fail.

  9. +++ b/core/lib/Drupal/Core/Command/DevInstallCommand.php
    @@ -0,0 +1,171 @@
    +    $site_path = getenv('DRUPAL_DEV_SITE_PATH');
    +    return $site_path ?: 'sites/default';
    

    Nit: This could be collapsed to a single line.

  10. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +      ->addOption('suppress_login', 's', InputOption::VALUE_NONE, 'Disables opening a login URL.')
    +      ->addOption('no_tty', NULL, InputOption::VALUE_NONE, 'Disables tty.');
    

    Shouldn't the option names have dashes, not underscores?

  11. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +   *   The drupal kernel.
    

    Nit: 'drupal' should be 'Drupal'.

  12. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +   * @throws \Exception
    +   *   Exception thrown if kernel does not boot.
    

    I"m not sure we need this, because the method itself does not throw any exceptions.

  13. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +    $address = '0.0.0.0';
    +    $port = 0;
    +    $socket = socket_create(AF_INET, SOCK_STREAM, SOL_TCP);
    +    socket_bind($socket, $address, $port);
    +    socket_listen($socket, 5);
    +    socket_getsockname($socket, $address, $port);
    +    return $port;
    

    Is there anything here that might fail? If so, how should we handle those failures?

  14. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +   * opens a url in your system default browser
    

    'opens' should be 'Opens', and 'url' should be 'URL'.

  15. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +   *   The URL to brower to.
    

    s/brower/browse

  16. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +      return $output = $process->run();
    

    Never seen this kind of thing before. What will it return?

  17. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +    $linux = (new Process('which xdg-open'))->run();
    +    $osx = (new Process('which open'))->run();
    

    Nit: Can these variables be prefixed with is_?

  18. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +    if ($this->openBrowser("http://localhost:$port$one_time_login/login", $io) === 1) {
    

    We're assuming the server name is localhost. 127.0.0.1 might be safer here. Even better, it'd be cool if the server name could be passed in as a command option.

  19. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +      $io->error('Error while opening up a one time login URL');
    

    This error message is superfluous because openBrowser() will echo out a useful error message if the browser cannot be opened.

  20. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +    if (($binary = $finder->find()) && $binary === FALSE) {
    

    This logic seems strange. What will find() return if it fails? If it's false, won't $binary automatically fail the check, and cause the exception?

  21. +++ b/core/lib/Drupal/Core/Command/DevStartCommand.php
    @@ -0,0 +1,208 @@
    +  protected function getSitePath() {
    +    $site_path = getenv('DRUPAL_DEV_SITE_PATH');
    +    return $site_path ?: 'sites/default';
    +  }
    

    Nit: Could be collapsed into a single line.

  22. +++ b/core/scripts/dev-site.php
    @@ -0,0 +1,22 @@
    +$application = new Application('dev-start', '8.0.x');
    

    Shouldn't the version be 8.6.x, or \Drupal::VERSION?

dawehner’s picture

Status: Needs work » Needs review
FileSize
23.69 KB
8.28 KB

"Wait" is followed by a smart ellipsis. Can it be three periods instead?

Wait ..… isn't hte smart ellipsis used exactly for this usecase?

Why is this separated in a list item? It seems like it could come right after "...take a minute or two."

Good point

Nit: I think we could shorten the line by using the ?: construction.

Note: This doesn't work when the array key isn't set.

The descriptions should probably end with periods.

Done

This should say what the default value will be.

Done

It seems to me that we should allow a user to specify a database URL, in case they don't have SQLite or don't want to use it. But we should continue to default to SQLite. Maybe that's more follow-up material, though.

I opened up a follow up: https://www.drupal.org/project/drupal/issues/2955344

is_dir() might be preferable here.

Done

We should probably throw exceptions and fail if copy() or mkdir() fail.

Done

Nit: This could be collapsed to a single line.

done

Shouldn't the option names have dashes, not underscores?

Great point!

Nit: 'drupal' should be 'Drupal'.

Done

I"m not sure we need this, because the method itself does not throw any exceptions.

Sure, but it still throws an exception, so it might be worth catching. ¯\_(ツ)_/¯

Is there anything here that might fail? If so, how should we handle those failures?

Sure, let's do that

'opens' should be 'Opens', and 'url' should be 'URL'.

Done

s/brower/browse

Done

Never seen this kind of thing before. What will it return?

There is no need for that :)

Nit: Can these variables be prefixed with is_?

Sure

We're assuming the server name is localhost. 127.0.0.1 might be safer here. Even better, it'd be cool if the server name could be passed in as a command option.

What about allowing this in a follow up?

This error message is superfluous because openBrowser() will echo out a useful error message if the browser cannot be opened.

Are you sure? What happens when running xdg-open/open fails?

This logic seems strange. What will find() return if it fails? If it's false, won't $binary automatically fail the check, and cause the exception?

Well, otherwise, $binary wouldn't be defined ...

Shouldn't the version be 8.6.x, or \Drupal::VERSION?

Sure

dawehner’s picture

Forgot to also upload a no-composer file.

mradcliffe’s picture

+In the instructions below, replace the version 8.x.x with the specific version
+you wish to download. Example: 8.6.0.zip. You can find the latest stable version
+at https://www.drupal.org/project/drupal.
+
+- curl -sS https://ftp.drupal.org/files/projects/drupal-8.x.x.zip --output drupal-8.x.x.zip
+- unzip drupal-8.x.x.zip
+- cd drupal-8.x.x
+- composer dev-site
+

I think the first two commands need additional review. Maybe in a follow-up?

Let's say I'm a new user. I go to drupal.org. I click "Download & Extend". I click "Download .zip". I find the downloaded file and double-click it to extract it (works on Windows and MacOS). I read INSTALL.txt. Do I need to run curl, unzip and cd again?

My next step is to open a command prompt (Terminal.app, PowerShell, Git Bash, Cmd or Terminal/xterm/aterm/iterm/whatever). I think some of the more OS-specific commands are best suited for online documentation guide at https://www.drupal.org/docs/8/install rather than INSTALL.txt or even drupal.com/getdrupal.com.

Would this work?

In the instructions below, replace the version 8.x.x with the specific version
you wish to download. Example: 8.6.0. You can find the latest stable version at
https://www.drupal.org/project/drupal. Run the following commands after you
have downloaded Drupal and extracted the .zip or .tar.gz file. Now open your
:"

1. cd drupal-8.x.x/
2. composer dev-site

I am not sure how to describe change directory since who knows where an user will have downloaded Drupal to. It's probably ~/Downloads or C:\Users\username\Downloads by default.

mglaman’s picture

I am not sure how to describe change directory since who knows where an user will have downloaded Drupal to

The pattern I followed in my book was always cd /path/to/drupal-8.x.x. Helps convey that it isn't _right_ there all the time.

alexpott’s picture

Discussed these changes with @dawehner.

  • The patch moves the command to core/scripts/quick-start.php
  • The command is named quick-start because this is not only for developers and in the future it could be expanded to be the CLI installer for Drupal so let's use quick-start and add a disclaimer
  • It removes the composer dependency - commands run via composer can not ask the user questions
  • It allows the user to select the profile the same way the interactive installer does
  • The user is logged in and redirected to the front page - this way they get a more welcoming start than the user edit form
alexpott’s picture

Issue tags: -Security improvements

The security tag was added when the script didn't do

if (PHP_SAPI !== 'cli') {
 return;
}

it does now... and anyway on a well configured apache I get a 403 cause of .htaccess when I tried to access the script but the double protection is good. Also the script will not install over an existing site so that's good too.

alexpott’s picture

Installing the standard profile via the quick-start command was producing an error because the CommentManager service is required to generate the 1 time login link and some where in it's dependencies it required a request... so adding one to the stack fixes that.

phenaproxima’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Command/QuickStartCommand.php
    @@ -0,0 +1,70 @@
    +      ->addOption('install_profile', NULL, InputOption::VALUE_OPTIONAL, 'Install profile to install the site in.')
    +      ->addOption('langcode', NULL, InputOption::VALUE_OPTIONAL, 'The language to install the site in. Defaults to en.', 'en')
    +      ->addOption('site_name', NULL, InputOption::VALUE_OPTIONAL, 'Set the site name. Defaults to Drupal', 'Drupal')
    +      ->addOption('suppress-login', 's', InputOption::VALUE_NONE, 'Disables opening a login URL.')
    +      ->addOption('show-server-output', NULL, InputOption::VALUE_NONE, 'Shows output from the PHP development server')
    

    Some of the options have underscores, and some have dashes. They should be consistent (I think dashes are preferred, since that's the standard for command-line options.) Also, some of the descriptions mention the default value, and others don't. Some end with periods, and others don't.

  2. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,242 @@
    +
    +  /**
    +   * Constructs a new QuickStartInstallCommand command.
    +   *
    +   * @param object $class_loader
    +   *   The class loader.
    +   */
    +  public function __construct($class_loader) {
    +    parent::__construct('install');
    +    $this->classLoader = $class_loader;
    +  }
    

    Seeing as how the $classLoader member is type hinted, should the $class_loader parameter be as well?

  3. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,242 @@
    +      ->addOption('install_profile', NULL, InputOption::VALUE_OPTIONAL, 'Install profile to install the site in.')
    +      ->addOption('langcode', NULL, InputOption::VALUE_OPTIONAL, 'The language to install the site in. Defaults to en.', 'en')
    +      ->addOption('site_name', NULL, InputOption::VALUE_OPTIONAL, 'Set the site name. Defaults to Drupal', 'Drupal')
    

    Underscores or dashes? All the commands in this patch should be consistent. Also, the descriptions should mention default values, if any, and end with periods.

  4. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,242 @@
    +    if (!extension_loaded('pdo_sqlite')) {
    +      $output->getErrorOutput()->writeln('You need to have sqlite installed.');
    +      return;
    +    }
    

    I think we should rephrase this error message to be more helpful. Maybe "You must have the SQLite PHP extension installed. See https://secure.php.net/manual/en/sqlite.installation.php for instructions."

  5. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,242 @@
    +   * @param string $profile
    +   *   (optional) The installation profile to use.
    +   * @param string $langcode
    +   *   (optional) The language to install the site in.
    +   * @param string $site_path
    +   *   (optional) The path to install the site to, like 'sites/default'.
    

    These should mention what the default values are.

  6. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,242 @@
    +    if (!is_dir($site_path)) {
    +      mkdir($site_path, 0775);
    +    }
    

    mkdir() may return FALSE, we should throw an exception if it does.

  7. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,242 @@
    +   * @see \_install_select_profile()
    

    Nit: Does this need a leading backslash?

  8. +++ b/core/lib/Drupal/Core/Command/QuickStartServerCommand.php
    @@ -0,0 +1,222 @@
    +    if (($binary = $finder->find()) && $binary === FALSE) {
    

    This logic is a little convoluted and makes me uncomfortable. Can it be refactored into a more "traditional" form that doesn't feel as implicit/magical?

alexpott’s picture

Status: Needs work » Needs review
FileSize
17.48 KB
33.24 KB

Thanks for the review.

  1. Fixed - had just standardised on -
  2. We can not know the classloader type. So object is the honest typehint
  3. Fixed
  4. Fixed
  5. Fixed by removing - these defaults are unnecessary - we're getting them all from somewhere else.
  6. Fixed
  7. Fixed
  8. Fixed

Also

  • implemented a test for running the server command when there is no install and made the error that occurs when you do that more helpful.
  • Always generate a one time login url - this helps to test standard and ensure we don't have #146 happen. Also if you suppress the automatic login the one time link is displayed to the user.
alexpott’s picture

So how should the profile selector work?

machine name | proper name

 Select an installation profile [Standard]:
  [standard  ] Standard
  [minimal   ] Minimal
  [demo_umami] Demo: Umami Food Magazine (Experimental)

machine name | description

 Select an installation profile [Install with commonly used features pre-configured.]:
  [standard  ] Install with commonly used features pre-configured.
  [minimal   ] Build a custom site without pre-configured functionality. Suitable for advanced users.
  [demo_umami] Install an example site that shows off some of Drupal’s capabilities.

proper name | description

 Select an installation profile [Install with commonly used features pre-configured.]:
  [Standard                                ] Install with commonly used features pre-configured.
  [Minimal                                 ] Build a custom site without pre-configured functionality. Suitable for advanced users.
  [Demo: Umami Food Magazine (Experimental)] Install an example site that shows off some of Drupal’s capabilities.

Not in all cases Console helps you auto-complete so that's really good. Personally I really like having the description because it tells you why you are selecting an installation profile. I also really like having the machine name because it is what you'd pass in if you want to avoid the prompt. So I've coded up option 2.

The other thing I'm pondering is if we should make the install profile an argument rather than an option - to be more like drush site-install?

alexpott’s picture

And here's a patch :)

phenaproxima’s picture

The other thing I'm pondering is if we should make the install profile an argument rather than an option - to be more like drush site-install?

IMHO, it should be an option. Drupal's default behavior is to install Standard unless told otherwise, so the console commands should be in keeping with that. Although this is targeted at developers, I still think we can follow the "don't make me think" UX rule.

cashwilliams’s picture

On argument vs option -

I vote argument, it would just be an optional that defaults to standard.

`drush si` and `drush si standard` do the same thing.

heddn’s picture

Machine name + Description and default to [standard]

Chi’s picture

  1. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,247 @@
    +      $output->getErrorOutput()->writeln('You must have the SQLite PHP extension installed. See https://secure.php.net/manual/en/sqlite.installation.php for instructions.');
    +      return;
    
  2. +++ b/core/lib/Drupal/Core/Command/QuickStartInstallCommand.php
    @@ -0,0 +1,247 @@
    +      $output->writeln('Drupal is already installed.');
    +      return;
    
  3. +++ b/core/lib/Drupal/Core/Command/QuickStartServerCommand.php
    @@ -0,0 +1,224 @@
    +      $io->getErrorStyle()->error('No installation found. Use the \'install\' command.');
    +      return;
    

Three different ways to write error message. I propose we stick to one of them for the purpose of consistency. Also return 1 is needed here to set the appropriate exit code.

alexpott’s picture

Re #156 Error writing standardised and return an error code. However re #156.2 as it says in the docs that is on purpose not an error so that is not changed.

Unfortunately changing install profile to an argument is not that simple because of the way we have a default command and an install command so it can't be an argument for the quick-start command. So I vote leaving as optional. The other way to go would be to change the application to be called drupal. So then we'd have:

php core/scripts/drupal.php quick-start
php core/scripts/drupal.php install
php core/scripts/drupal.php server

I think I'd prefer

php core/scripts/drupal quick-start
php core/scripts/drupal install
php core/scripts/drupal server

But then we'll have the all php files must end in .php brigade up in arms. There's also a pre-existing core/scripts/drupal.sh to get confused with.

The default is the same as the install profile select form - unfortunately this is not a simple thing to document in a command. The default if present is Standard, If that is not present then Minimal and if that is not then whatever is next. However if there is one profile that is a distribution then that overrides everything and is selected automagically. Tried to document this.

The patch attached also implements a progress bar. I couldn't resist - it was too simple a change to install_run_tasks() and install_drupal() to support this.

Also whilst testing I've noticed that installing in other languages doesn't quite work as well as one would hope. Dunno why yet. But I suspect the translation downloader was not written to be run by a non-interactive installer.

alexpott’s picture

I think we should bring our work into line with #2242947: Integrate Symfony Console component to natively support command line operations and standardise on core/scripts/drupal. Here's a patch that does that and also makes the command easier to test locally - the last patch broke that because of the way the progress bar works.

Also now I've made that move I've changed install profile to an argument.

Chi’s picture

@alexpott, should we add this command to composer scripts?

cilefen’s picture

Or core/bin/drupal? FWIW that would be more Composer standard.

phenaproxima’s picture

Or core/bin/drupal? FWIW that would be more Composer standard.

+1 for this.

Chi’s picture

Or core/bin/drupal?

We could specify bin key in composer.json file. See #2242947: Integrate Symfony Console component to natively support command line operations(#89 and #91) for details.

pingwin4eg’s picture

If we name it drupal and put it in the bin, wouldn't that clash with the drupal/console project?

alexpott’s picture

@pingwin4eg I think not. I think that console typically goes in vendor/bin but yes there would be a clash if we tell composer to treat core/bin/drupal the same and when installing a Drupal project put core/bin/drupal in vendor/bin. But there is certainly a naming clash :) I think it is ironic that #2242947: Integrate Symfony Console component to natively support command line operations is using the console name and drupal/console is using drupal name. Anyhows.

I really don't know what to do with core/scripts/drupal vs core/bin/drupal. I feel that drupal is the right name for the command especially given we want to expand the scope of what a core provided cli can do. Part of me thinks core/scripts is simpler because that's where all the scripts are now and another part thinks core/bin is obviously more in-tune with everything else. Though thinking about my recent experience with craft they just have it in the doc root which has a certain amount of allure - think php drupal quick-start</core> or even <code>./drupal quick-start if drupal is executable. However going for the root directory will be difficult for sites created with composer create-project - at least another level of complexity. So.... my gut feeling says let's punt on this in this issue. We can always move it from core/scripts/drupal later in #2242947: Integrate Symfony Console component to natively support command line operations and just leave behind a deprecated version that just includes the new version.

With respect to putting the command into composer.json so it can be run with composer run-script that does not work. Composer is not a very good script runner at the moment because it does not allow user input. See #137 for more details. Another reason not to do that is that it introduces a completely unnecessary dependency and the less of them the better.

alexpott’s picture

Issue summary: View changes

Updated issue summary with the current state.

alexpott’s picture

So multi-lingual install works via CLI if \Drupal::VERSION is equal to 8.6.0 but not when it is equal to 8.6.0-dev :)

And picking through the interactions of

  • _install_prepare_import()
  • locale_translation_batch_fetch_download()
  • locale_translation_batch_fetch_import()

Is not at all simple!

alexpott’s picture

Patch attached fixes multi-lingual install. It also fixes multi-lingual install for Drush on development versions of Drupal too :)

I think we probably should carve these changes of into a blocking issue for more focused discussion on them. They are however very very small :)

Mile23’s picture

If we name it drupal and put it in the bin, wouldn't that clash with the drupal/console project?

If we put it in core/scripts or core/bin or whatever, then we can tell people to run core/scripts/drupal in documentation, and it will work alongside drupal/console by accident. It will be confusing, because it will be similar to documentation for drupal/console, but it will be workable.

If we tell composer.json that core/scripts/drupal is a vendor binary, then there will be a couple problems. https://getcomposer.org/doc/articles/vendor-binaries.md

1) It will conflict with the drupal console project, since they both have the same filename.

2) The script might need to account for being run from an arbitrary place in the filesystem (since the user can require drupal/core globally, and since they can also say config:bin-dir:literally/any/where).

So, Option A: If we think we might at some point turn this limited-scope script into a full console tool, we should either ask drupal/console to change their name (not very nice), or use a different name for this script. To this end, if we're planning to expand from here, I'd endorse adding this script to drupal/core/composer.json:bin, so that we address this problem now instead of not.

What we actually call it doesn't matter that much as long as it doesn't conflict. #2242947: Integrate Symfony Console component to natively support command line operations calls it console, which is hopelessly generic and might conflict some other way eventually.

Option B: We could not hinge this on future expansion. We could just call it drupal-installer because that's what it does. Any future console integration could happen separately and simply involve adding the happily-factored command objects from this script to the console app for the other one. Naming it drupal-installer avoids the documentation confusion, and allows us to leave that script in place even after we add whatever other general-purpose app we make.

alexpott’s picture

Hey look at what is possible now!

$ php core/scripts/drupal install minimal --langcode=de

Drupal installation started. This could take a minute.
 17/17 [▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓] 100%

Glückwunsch, Drupal wurde erfolgreich installiert.

I guess now I need to work asking the select profile question once that is translatable. As we now have the callback in the install system that should be possible.

@Mile23 well the aim of #2242947: Integrate Symfony Console component to natively support command line operations is to build a generic Drupal CLI provided by core so in my mind it is that issues job to sort out the clash with drupal/console and ponder the composer bin questions. All we should do here is worry about what we are telling people to do here and unfortunately drupal-installer quick-start does not make sense. Also yes it is not nice but Drupal is this projects name and console calling their command drupal is actually the clashing thing - there already was core/scripts/drupal.sh and has been for years. There's no way around it that the obvious name for a drupal cli tool provided by core is drupal - it's what anyone would expect.

We have 3 commands here: quick-start, install and server. They are all doing things with drupal hence core/scripts/drupal. I'd vote for leaving it at that and solving the bigger issues in #2242947: Integrate Symfony Console component to natively support command line operations.

dawehner’s picture

Let's remove drupal.sh, honestly, I doubt anyone is using it: #2955805: Remove drupal.sh

Here are a couple of nitpick/small changes.

If we tell composer.json that core/scripts/drupal is a vendor binary, then there will be a couple problems. https://getcomposer.org/doc/articles/vendor-binaries.md

I'm curious whether console could rename itself on the longrun to console? On the other hand II actually believe that they have similar commands already, so maybe the right solution is for them to reimplement our new commands with the same interface.

ressa’s picture

This is fantastic, great work so far everyone. Here are updated instructions for testing the current patch quickly in a fresh environment:

Ensure you have PHP 5.5.9 (or greater) installed, and run these commands:

  1. curl -sS https://ftp-origin.drupal.org/files/projects/drupal-8.6.x-dev.zip --output drupal-8.6.x-dev.zip
  2. unzip drupal-8.6.x-dev.zip && rm drupal-8.6.x-dev.zip
  3. cd drupal-8.6.x-dev
  4. wget -q -O - https://www.drupal.org/files/issues/2018-03-24/2911319-2-169.patch | git apply -
  5. php core/scripts/drupal quick-start standard

A fresh Drupal installation will now have opened up in your browser, logged in as administrator.

Mile23’s picture

--langcode=de

Woot!

We also have #1540390: Deprecate the drupal.sh script and #2289405-21: [META] Port all shell scripts to console commands where @andypost argues that porting existing scripts is more purposeful than simply doing an integration framework.

I think we're at a sort of critical mass where this stuff should happen in a structured way.

alexpott’s picture

dawehner’s picture

Issue summary: View changes

I added some steps contributors could do and would be really valueable.

kim.pepper’s picture

Awesome! So happy to see this issue moving along.

We need to handle invalid profile names. My first attempt was:

php core/scripts/drupal quick-start umami
Drupal installation started. This could take a minute.
Error: Call to a member function getPath() on null in /Users/kim/dev/sandbox/drupal-8.6.x-dev/core/includes/install.core.inc on line 819
geerlingguy’s picture

I'm going to definitely do the Try it out on windows task. I knew lugging my ThinkPad to DrupalCon would be helpful in some way or another :)

Not sure if PHP or Windows 10 Pro itself comes with SQLite available, but I'm going to follow my own guide for installing native PHP/Composer on Windows, and then run through testing this patch and see what happens. Fingers crossed.

alexpott’s picture

@kim.pepper can we get some info on your system? How did you get the code, what OS, PHP version etc...

Mile23’s picture

Re #180: The profile is called demo_umami, not umami. So we have to figure out if the profile exists or not before acting on it.

MrMason’s picture

I am going to give the video of this working a try. I'm not sure if it's at a point where we can do this but I have a mac and all the software to record this. I'm going to run this fresh and report back if I am going to be able to get this flowing properly to record it.

kim.pepper’s picture

@alexpott Yep sure:

  • MacOS 10.13.4
  • PHP 7.1.12 installed via homebrew
  • Installed following steps in #172
alexpott’s picture

@kim.pepper did #181 fix it for you?

larowlan’s picture

From #178

We need to handle invalid profile names

i.e we should recover gracefully if the profile does not exist.

dawehner’s picture

Assigned: Unassigned » dawehner

Thank you @geerlingguy and @MrManson
Feel free to find me in slack/drupalcon, when you need help/want to chat.

i.e we should recover gracefully if the profile does not exist.

Sure, I'm having a look at that!

kim.pepper’s picture

@alexpott Like @larowlan said, I was merely pointing out there is a stacktrace if you pass it an invalid profile name. We should just return a message like "invalid profile name".

alexpott’s picture

Status: Needs work » Needs review
FileSize
6.89 KB
38.47 KB

Here's a fix for the invalid profile name issue. Also provides a suggestion if you get it slightly wrong.

@kim.pepper - i totally missed

We need to handle invalid profile names.

. Sorry. Nice find.

alexpott’s picture

Sorry @dawehner I didn't see you had assigned yourself.

kim.pepper’s picture

Awesome work @alexpott with the suggestions. Today I learnt a new php function: 'levenshtein'

alexpott’s picture

The blocker has been merged. Rerolled. Multi-lingual install will work!

geerlingguy’s picture

Windows steps:

  1. Install PHP and Composer (guide I wrote for native Windows / PowerShell usage).
  2. Install Git for Windows (I chose the PowerShell integration during setup).
  3. Open PowerShell, go to a folder (e.g. username\Sites) where you want to clone Drupal.
  4. Clone Drupal: git clone --branch 8.6.x https://git.drupal.org/project/drupal.git drupal8
  5. cd into the Drupal folder: cd drupal8 (I love how PowerShell aliases the posix-y commands nowadays :).
  6. Download the latest patch from this issue: Invoke-webrequest -URI "https://www.drupal.org/files/issues/2018-04-12/2911319-2-216.patch" -OutFile 2911319-2-216.patch
  7. Apply the patch: git apply -v 2911319-2-216.patch
  8. Install dependencies: composer install
  9. Run the environment: php core/scripts/drupal quick-start demo_umami

(If you get an error in the patch download, run [Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12 to make sure PowerShell can interact with Drupal.org's HTTPS).

Almost everything worked, but it looks like socket_create() was the one fatal flaw... output from quick-start command: Call to undefined function Drupal\Core\Command\socket_create() It looks like I was missing the php_sockets.dll extension in my php.ini file. Uncommented that line and now it's working!

I'll post a video of the entire process in a few.

geerlingguy’s picture

Hmm, on second thought—the localhost server is started, it dumps the URL to the console, then it opens a random Windows Explorer window and quits the server and dumps me back to my current path in PowerShell. Video will show the details once I get it uploaded.

geerlingguy’s picture

Issue tags: +Nashville2018

Adding tag.

geerlingguy’s picture

Ah, getting the error (masked because of the suppressed output):

PS C:\Users\jgeerling\Sites\drupal8> php core/scripts/drupal quick-start demo_umami --suppress-login --show-server-output
Drupal is already installed.
Starting webserver on http://localhost:53336
Press Ctrl-C to quit.
One time login url: http://localhost:53336/user/reset/1/1523426637/J337M7KchmBTonQCW11BRMNNR15Ufvq7ZwWngjheXUI/login
<br />
<b>Warning</b>:  Unknown: php_network_getaddresses: getaddrinfo failed: No such host is known.  in <b>Unknown</b> on line <b>0</b><br />
[Wed Apr 11 01:03:58 2018] PHP Warning:  Unknown: php_network_getaddresses: getaddrinfo failed: No such host is known.  in Unknown on line 0
[Wed Apr 11 01:03:58 2018] Failed to listen on localhost:53336 (reason: php_network_getaddresses: getaddrinfo failed: No such host is known. )
geerlingguy’s picture

Changing localhost to 127.0.0.1 gives the following different error:

[Wed Apr 11 01:07:12 2018] Failed to listen on 127.0.0.1:53386 (reason: ?)
geerlingguy’s picture

Here's the video promised earlier, of testing on Windows 10 Pro of the patch in #191: https://drive.google.com/file/d/17uRiPG6iTcFIxUV_uggajm0HBM1C9I46/view?u...

ressa’s picture

The latest patch works perfectly, failing gracefully and showing this if you try php core/scripts/drupal quick-start umami:

[ERROR] 'umami' is not a valid install profile. Did you mean 'demo_umami'?

Linux steps

Install Git and required PHP packages, if needed:
sudo apt-get install git php7.0 php7.0-curl php7.0-dom php7.0-gd php7.0-mbstring php7.0-sqlite3

Install Drupal Umami demo:

  1. curl -sS https://ftp-origin.drupal.org/files/projects/drupal-8.6.x-dev.zip --output drupal-8.6.x-dev.zip
  2. unzip drupal-8.6.x-dev.zip && rm drupal-8.6.x-dev.zip
  3. cd drupal-8.6.x-dev
  4. wget -q -O - https://www.drupal.org/files/issues/2018-04-11/2911319-2-191.patch | git apply -
  5. php core/scripts/drupal quick-start demo_umami

When you are done testing, close the PHP server with CTRL + Z and delete the drupal-8.6.x-dev folder.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Command/InstallCommand.php
@@ -208,16 +214,71 @@ protected function getSitePath() {
+      $alternatives = [];
+      foreach (array_keys($profiles) as $profile_name) {
+        $lev = levenshtein($install_profile, $profile_name);
+        if ($lev <= strlen($profile_name) / 4 || FALSE !== strpos($profile_name, $install_profile)) {

Nice!!!

  1. +++ b/core/lib/Drupal/Core/Command/InstallCommand.php
    @@ -208,16 +214,71 @@ protected function getSitePath() {
    +      $alternatives = [];
    +      foreach (array_keys($profiles) as $profile_name) {
    +        $lev = levenshtein($install_profile, $profile_name);
    +        if ($lev <= strlen($profile_name) / 4 || FALSE !== strpos($profile_name, $install_profile)) {
    +          $alternatives[] = $profile_name;
    +        }
    +      }
    +      if (!empty($alternatives)) {
    

    Why do we not just compute the levenstein difference and sort the result, and limit the list to two?

  2. +++ b/core/tests/Drupal/Tests/Core/Command/QuickStartTest.php
    @@ -190,6 +190,19 @@ public function testQuickStartInstallAndServerCommands() {
    +    // Install a site using the standard profile to ensure the one time login
    +    // link generation works.
    

    This comment seems wrong.

geerlingguy’s picture

Re-tested everything in Windows 10 under the WSL in Ubuntu, and am getting different errors; the PHP webserver is started, but trying to load a page results in:

The website encountered an unexpected error. Please try again later.

I'm attaching a screenshot of the terminal after running quick-start with no arguments (in WSL, it had an error opening the one-time login URL, presumably because the environment in WSL doesn't easily allow the script to open a Windows GUI browser), and then again with --suppress-login --show-server-output.

I'm also attaching a text file with the full error message when trying to load a page under WSL/PHP.

dawehner’s picture

@geerlingguy Thank you for your great testing! I'm curious, can you install via sqlite normally?

alexpott’s picture

I'm a bit confused as to how #191 was green - running core/tests/Drupal/Tests/Core/Command/QuickStartTest.php locally showed that it was very broken. Ah I see it's the marking tests as skipped let's remove that - see what happens and then implement something better.

Patch attached fixes the tests and also changes how ports are discovered and what hostname is used by default. It also allows both to be customised. It works fine OSX. I wonder about Windows @geerlingguy?

geerlingguy’s picture

@dawehner - It seems like the Drupal install completes successfully (using SQLite), it's just that something fails with the hostname allocation :/

@alexpott - I'll try the new patch tonight. For my own reference, the new options are --host and --port.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Command/InstallCommand.php
@@ -271,10 +271,13 @@
+   * @param bool $include_hidden
+   *   (optional) Whether to include hidden profiles. Defaults to FALSE.
+   *
    * @return string[]
    *   An array of profile descriptions keyed by the profile machine name.
    */
-  protected function getProfiles() {
+  protected function getProfiles($include_hidden = FALSE) {
     // Build a list of all available profiles.
     $listing = new ExtensionDiscovery(getcwd(), FALSE);
     $listing->setProfileDirectories([]);

Can't we change the setting dynamically to adapt that?

geerlingguy’s picture

More debugging on Windows 10:

  • If I use --host localhost or --host 127.0.0.1, I get the same errors as seen earlier.
  • If I use --host [IP from ipconfig] (using either the wired or wireless physical device IP), I get the same error as with 127.0.0.1, namely Failed to listen on 170.20.20.2:8888 (reason: ?).
  • If I use --host 0.0.0.0, I get the same error.
  • If I run a server using PHP directly, with php -S localhost:8888 (after having used quick-start to install Drupal), the server stays up and I can browse the Drupal site in a browser.
  • Running PowerShell as administrator (or not) made no difference in the result.

It looks like reason: ? is only returned when the error string is empty (see this line in the php src). Further digging and Googling is getting me nowhere. Any other ideas?

Can someone else with Windows 10 test this process out and see if maybe it's just my machine? It's a pretty plain-vanilla Win 10 Pro install, no VPNs, no special software, nothing special on it... but who knows, maybe some other testing with VirtualBox or Docker could've messed up the networking internally?

It's weird though that the PHP built-in server works fine, but the Symfony component doesn't seem to do so. Maybe we're missing an option that's required to get it working? I'm going to see if I can also test with the Laravel artisan quickstart environment and see if their Symfony-based quickstart environment works here.

geerlingguy’s picture

Interestingly, Laravel's laravel new [appname] command required PHP 7.1.x or later to work, so I had to upgrade my Windows native system PHP from 7.0.x to 7.1.x. (But neither 7.0 nor 7.1 made a difference when using quick-start with this patch.)

After doing that, I created a new Laravel app, then ran: php artisan serve, and got the following:

PS C:\Users\jgeerling\Sites\blog> php artisan serve
Laravel development server started: <http://127.0.0.1:8000>

Content was served and the server kept running fine. So I wonder if it's something in our implementation?

alexpott’s picture

So Laravel is doing passthru($this->serverCommand(), $status);. Symfony uses proc_open() which is what Drush is using too. But Drush is running the server command interactively. Looking at Drush it seems there is a way to keep the auto-logging in. So copying them but using the passthru approach of Laravel.

Also patch attempts to fix the test skipping issue. We can't run this test if sites/simpletest does not exist and that is not a usual requirement for running Drupal core unit tests.

jonathanshaw’s picture

On Windows 10 using powershell (admin) and #208 I get

Drupal development server started: <http://127.0.0.1:8888>
One time login url: <http://127.0.0.1:8888/user/reset/1/1523527919/jT6i4d7LMM_GzfYn1TXsZRXWIHMabJf9GjPBAkh5s8g/login>
Press Ctrl-C to quit.

But no browser opens, no error appears.
Sockets is loaded according to php -m.

Copying and pasting the link into a browser and it works fine. Yay!

alexpott’s picture

Added some verbose output so we can try to work out why the browser does not open. @jonathanshaw can you add a -v to your command and then try the browser command from your shell. Thanks!

jonathanshaw’s picture

Drupal is already installed.
Drupal development server started: <http://127.0.0.1:8888>
One time login url: <http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwcaM-uRloYnkI_HESe4/login>
Press Ctrl-C to quit.
Browser command: sleep 2 && start "web" explorer ""http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwcaM-uRloYnkI_HESe4/login?destination= 2F""
Server command: "C:\Program Files (manually installed)\php7\php.exe" -S 127.0.0.1:8888 .ht.router.php
alexpott’s picture

@jonathanshaw so what happens if in another shell you do
sleep 2 && start "web" explorer ""http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwcaM-uRloYnkI_HESe4/login?destination= 2F"" whilst the server is running?

alexpott’s picture

I think we might have too many quotes for Windows. How about this?

jonathanshaw’s picture

In powershell (admin):

#214:

Drupal is already installed.
Drupal development server started: <http://127.0.0.1:8888>
One time login url: <http://127.0.0.1:8888/user/reset/1/1523539460/vxyxe2mR34DWSPnNXHabVtWYwmUz_J8_fDPvjKynOjs/login>
Press Ctrl-C to quit.
Browser command: sleep 2 && start "web" explorer "http://127.0.0.1:8888/user/reset/1/1523539460/vxyxe2mR34DWSPnNXHabVtWYwmUz_J8_fDPvjKynOjs/login?destination=%2F"
Server command: "C:\Program Files (manually installed)\php7\php.exe" -S 127.0.0.1:8888 .ht.router.php

#213:
with or without double quotes:

At line:1 char:9
+ sleep 2 && start "web" explorer "http://127.0.0.1:8888/user/reset/1/1 ...
+         ~~
The token '&&' is not a valid statement separator in this version.
    + CategoryInfo          : ParserError: (:) [], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : InvalidEndOfLine

Sleep 2 is fine. As is sleep 2; sleep 2 but not sleep 2 && sleep 2.

Sometimes destination=%2F becomes destination= 2F but this doesn't seem to affect what I'm reporting here.

Trying start on its own:

start "web" explorer "http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwcaM
-uRloYnkI_HESe4/login?destination= 2F"

gives

Start-Process : A positional parameter cannot be found that accepts argument
'http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwcaM-uRloYnkI_HESe4/login?destination= 2F'.
start "web" explorer ""http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwca
M-uRloYnkI_HESe4/login?destination= 2F""

gives
Start-Process : A positional parameter cannot be found that accepts argument ''.

So double quotes is clearly not working, but it's not the only problem.

What works:

sleep 2; start "http://127.0.0.1:8888/user/reset/1/1523531894/7t7kP_toHPwJ2ExCx-aXr0nMwcaM-uRloY
nkI_HESe4/login?destination=%2F"
dawehner’s picture

Assigned: dawehner » Unassigned
alexpott’s picture

Thanks for all the info @jonathanshaw. Patch attached should fix that. It also removes usage of ProcessUtils::escapeArgument() because that is deprecated. That's not being picked up by the tests because the command PHP is being run totally outside of Drupal. Tricky to know how to solve that. Should have a follow-up for that.

geerlingguy’s picture

@alexpott - latest patch (216) outputs:

Drupal development server started: <http://127.0.0.1:8888>
One time login url: <http://127.0.0.1:8888/user/reset/1/1523543971/eAL-jgJmqtHw8FJK9dh0-WfU6K1573FaZFO1fp3DKUo/login>
Press Ctrl-C to quit.
Browser command: sleep 2; start "web" explorer "http://127.0.0.1:8888/user/reset/1/1523543971/eAL-jgJmqtHw8FJK9dh0-WfU6K1573FaZFO1fp3DKUo/login?destination= 2F"
Server command: C:\PHP7\php.exe -S 127.0.0.1:8888 .ht.router.php

But browser still doesn't open automatically. If I run the Browser command separately, I get:

PS C:\Users\jgeerling> sleep 2; start "web" explorer "http://127.0.0.1:8888/user/reset/1/1523543971/eAL-jgJmqtHw8FJK9dh0
-WfU6K1573FaZFO1fp3DKUo/login?destination= 2F"
Start-Process : A positional parameter cannot be found that accepts argument
'http://127.0.0.1:8888/user/reset/1/1523543971/eAL-jgJmqtHw8FJK9dh0-WfU6K1573FaZFO1fp3DKUo/login?destination= 2F'.
At line:1 char:10
+ sleep 2; start "web" explorer "http://127.0.0.1:8888/user/reset/1/152 ...
+          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [Start-Process], ParameterBindingException
    + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.StartProcessCommand
jonathanshaw’s picture

Yeah, the command needs to be start "http... not start "web" explorer "http....

alexpott’s picture

How about this? Moved the sleeping to PHP-land so we should be abler to work on any shell in Windows.

jonathanshaw’s picture

Uggh. start "http://... works from powershell fine, but when run form inside the script it does nothing but open a new command prompt.

I will play around with syntax inside the script and see if I get any joy.

jonathanshaw’s picture

And we have a winner ...
$cmd = 'start "" ' . $url;

Start requires a title (can be zero length string) as its first argument when called from inside a script.

alexpott’s picture

Here goes... I choose to have a title vs an empty title - because empty quotes just looks weird. https://ss64.com/nt/start.html says:

Always include a TITLE this can be a simple string like "My Script" or just a pair of empty quotes ""
According to the Microsoft documentation, the title is optional, but depending on the other options chosen you can have problems if it is omitted.

jonathanshaw’s picture

Yay! #223 works for me on windows 10 pro, powershell (admin), php 7.1

dawehner’s picture

I started to generate an admin password automatically. On top of that I simplified some things a bit.

geerlingguy’s picture

@alexpott - And now I've tested it on macOS High Sierra, and also under Travis CI / Linux / Ubuntu 14.04 (see automated build, and everything seems to work well!

I'm tempted to hit RTBC, but do we want to leave this issue in review so some DrupalCon sprinters tomorrow can re-test, gather screenshots, etc. for an RTBC tomorrow (and heck, maybe on-stage commit!).

alexpott’s picture

+++ b/core/lib/Drupal/Core/Command/InstallCommand.php
@@ -183,8 +185,10 @@ protected function install($class_loader, SymfonyStyle $io, $profile, $langcode,
-    $success_message = t('Congratulations, you installed @drupal!', [
+    $success_message = t('Congratulations, you installed @drupal! User name: @name  User password: @pass', [
       '@drupal' => drupal_install_profile_distribution_name(),
+      '@name' => 'admin',
+      '@pass' => $password,
     ], ['langcode' => $langcode]);

This change means we don't get the message in the user's chosen language. Cause Congratulations, you installed @drupal is text already in core. Left alone for now. Maybe we should drop the translatability and use the info tag to make the output readable.

Patch attached continues the tidy-ups - fixing the borken distribution handling, and remove the defaults from the description - console adds this for us.

ressa’s picture

With the latest patch, the browser opens up on the /user/1/edit page, and not front as it did previously.

NOW: http://127.0.0.1:8889/user/1/edit?pass-reset-token=DSdMFZoQlgmUwGxdHBusNqgulAM1vIo06oljzHx-ZaOp-M_Iv4p-rTI93WhpKzMRROgEcC4G1w

BEFORE: http://localhost:39048/

I think opening front is the best user experience.

alexpott’s picture

Okay let's add that back but we need confirmation this doesn't break windows. There were problems with this earlier. As we're not escaping user input I'm avoiding using escapeshellarg() on Windows since we know this is a generated URL.

geerlingguy’s picture

FileSize
1.09 MB

@alexpott - #229 still works great on Windows 10, running PowerShell as administrator or not; see attached file.

ressa’s picture

Thanks for fixing it @alexpott, #229 works perfectly, opening the front page in the browser on Ubuntu 16.04.

larowlan’s picture

  1. +++ b/core/INSTALL.txt
    @@ -10,6 +11,30 @@ CONTENTS OF THIS FILE
    +- curl -sS https://ftp.drupal.org/files/projects/drupal-8.x.x.zip --output drupal-8.x.x.zip
    +- unzip drupal-8.x.x.zip
    +- cd /path/to/drupal-8.x.x
    +// Installs Drupal and open it up
    +- php core/scripts/drupal quick-start
    

    Is the long term plan to move to a phar that does these steps for you?

    e.g. the issue summary uses an npm example

    We still have these steps.

  2. +++ b/core/lib/Drupal/Core/Command/InstallCommand.php
    @@ -0,0 +1,317 @@
    +    }
    +    elseif (!$install_profile) {
    

    can use a normal if here, previous if returned

  3. +++ b/core/lib/Drupal/Core/Command/InstallCommand.php
    @@ -0,0 +1,317 @@
    +      if (!mkdir($site_path, 0775)) {
    

    Should we add some output here when running in verbose?

  4. +++ b/core/lib/Drupal/Core/Command/InstallCommand.php
    @@ -0,0 +1,317 @@
    +      $io->progressAdvance();
    

    it's a pity SymfonyStyle doesn't expose ProgressBar::setMessage - we could provide some useful feedback here to break up the 'this could take a minute'

    Upstream issue?

  5. +++ b/core/lib/Drupal/Core/Command/QuickStartCommand.php
    @@ -0,0 +1,71 @@
    +      ->addUsage('demo_umami --langcode fr');
    

    Worth adding some more usage examples now we have host/post/suppress?

  6. +++ b/core/lib/Drupal/Core/Command/ServerCommand.php
    @@ -0,0 +1,260 @@
    +  protected function openBrowser($url, SymfonyStyle $io) {
    +    // Time in seconds to wait for before opening the browser.
    +    $wait = 2;
    +    $is_windows = defined('PHP_WINDOWS_VERSION_BUILD');
    +    if ($is_windows) {
    +      // Handle escaping ourselves.
    +      $cmd = 'start "web" "' . $url . '""';
    +    }
    +    else {
    +      $url = escapeshellarg($url);
    +    }
    +
    +    $is_linux = (new Process('which xdg-open'))->run();
    +    $is_osx = (new Process('which open'))->run();
    +    if ($is_linux === 0) {
    +      $cmd = 'xdg-open ' . $url;
    +    }
    +    elseif ($is_osx === 0) {
    +      $cmd = 'open ' . $url;
    +    }
    +
    +    if (empty($cmd)) {
    +      $io->getErrorStyle()
    +        ->error('No suitable browser opening command found, open yourself: ' . $url);
    +      return;
    +    }
    +
    +    if ($io->isVerbose()) {
    +      $io->writeln("<info>Browser command:</info> $cmd");
    +    }
    +
    +    // Need to escape double quotes in the command so the PHP will work.
    +    $cmd = str_replace('"', '\"', $cmd);
    +    // Sleep in PHP so that Windows powershell users also get a browser opened
    +    // for them.
    +    $php = "<?php sleep($wait); passthru(\"$cmd\"); ?>";
    +    $process = new PhpProcess($php);
    +    $process->start();
    

    a lot of this feels like it has use outside of this class - i.e as a generic helper class, not disimilar to things like the table, progress and question helper - should we split out? We have similar stuff in run-tests.sh that we'd eventually want to be a console command, so there is at least one other chance for re-use

  7. +++ b/core/tests/Drupal/Tests/Core/Command/QuickStartTest.php
    @@ -0,0 +1,256 @@
    +  protected function fileUnmanagedDeleteRecursive($path, $callback = NULL) {
    

    similarly here - should this be a trait - there is use for it outside this test?

borisson_’s picture

#232, I don't think we should extract things from this class to helper classes / traits for now. We can always do that in the future. Especially if we annotate the Commands with @internal, and I think we should do that.

  1. +++ b/core/lib/Drupal/Core/Command/ServerCommand.php
    @@ -0,0 +1,260 @@
    +    // Time in seconds to wait for before opening the browser.
    +    $wait = 2;
    ...
    +    $php = "<?php sleep($wait); passthru(\"$cmd\"); ?>";
    

    This variable is used only in one place as far as I can see. So we should probably remove the variable and use 2 in the place where it is used.

benjifisher’s picture

Issue tags: +Novice

I am adding the Novice tag for further testing, including screenshots/screencasts. There are also non-novice parts left for this issue.

pbirk’s picture

I'm working on this today at DrupalCon Nashville in the mentored sprint room. Come find me if you are, too.

alexpott’s picture

Thanks for the reviews @larowlan and @borisson_

I've fixed #232.2,3,5

I'll go and look for upstream issues re #232.4

I agree with @borisson_ we should make these @internal for now and look to make things generic as and when the need arises.

alexpott’s picture

Re: #232.4 I looked upstream and found that this was already possible.

pbirk’s picture

I tested against both #229 and #237 in a DrupalVM environment and it worked both times. My environment required that I update the --host option. I noticed that option when I reviewed the patch. I think a quick note in the INSTALL.txt file would make this more clear and will submit a patch for that shortly.

alexpott’s picture

@pbirk thanks for testing! What did you need to update it to?

heddn’s picture

+++ b/core/lib/Drupal/Core/Command/InstallCommand.php
@@ -0,0 +1,336 @@
+    $io->writeln('Drupal installation started. This could take a minute.');

Is this needed now?

Chelsee’s picture

FileSize
347.7 KB

I tested this on my Windows 10 Pro Microsoft Surface in power shell. Everything seemed to work, it opened up my browser and looked good.

alexpott’s picture

+++ b/core/lib/Drupal/Core/Command/InstallCommand.php
@@ -0,0 +1,336 @@
+        $progress_bar->setMessage(t('Installing @drupal', ['@drupal' => drupal_install_profile_distribution_name()]));

On an English install this message doesn't really appear. It's really just to cover for downloading translations which doesn't have a 'display_name'. But I think now we have the progress bar feedback is much better and problem @heddn is correctly. That message is redundant. We added before we had the progress bar. Anyone else have any thoughts about that?

pbirk’s picture

@alexpott I needed to use --host=192.168.88.88 since I'm accessing the site from the browser on the host computer, not via the VM (where I don't have a GUI).

pbirk’s picture

This is just a quick patch that adds a note to highlight the quick-start options available in the INSTALL.txt file.

maxstarkenburg’s picture

FileSize
299.49 KB

With Jeff Geerling's help (thanks!), I applied patch 2911319-2-245.patch on a Windows 7 Enterprise local install of Drupal 8.6.x (standard profile) using PowerShell and it opened in the browser and loaded fine. DrupalCon Nashville 2018. First issue, woo!

alexpott’s picture

Status: Needs work » Needs review
FileSize
2.15 KB
40.83 KB

Fixing the tests...

alexpott’s picture

@geerlingguy do you have any idea how we could determine 192.168.88.88 or whatever it needs to be for DrupalVM automatically? Re #243

mlsamuelson’s picture

I tested this on OS X with a local MAMP environment, PHP 7.1.8, and installed the demo_umami install profile successfully. Experience was smooth.

geerlingguy’s picture

@alexpott - Hmm, inside the VM itself, it would need to know what interface is being used and grab the IP from there... seems like it would be complicated to do automatically (and prone to breaking outside of a VM environment). I'd say documentation might be the best approach here.

David Radcliffe’s picture

This worked well for me on Mac OS. Here's my screencast. https://vimeo.com/264667336

pbirk’s picture

@alexpott: I think determining the correct host to use should be the user's responsibility if 127.0.0.1 isn't going to work. I only ran into the issue because I was using DrupalVM when I choose to work on this issue at DrupalCon. A normal DrupalVM user gets a working environment after installing DrupalVM, so they're unlikely to attempt the quick-start command, anyways.

I think highlighting the available options is still a good idea, since we don't know what environment is in use. The first time I installed Drupal, it was directly on the same development server I hosted our custom PHP website. The same issue exists in that situation, but there's no automated way to determine the IP or port to use then, either.

maxstarkenburg’s picture

Status: Needs review » Needs work
FileSize
314.2 KB
361.88 KB

After Ctrl-C'ing and bothering with some local git adding/committing stuff, I tried running php core/scripts/drupal quick-start standard again, but the admin password (which I wanted to reset) was too far up in the terminal to be accessible anymore. It was recommended I delete sites/default/settings.php, which I did and got the error message in the first screenshot. It was then suggested I delete sites/default/files/.sqlite, after which running php core/scripts/drupal quick-start standard also produced different errors as seen in the second screenshot (that a composer install didn't fix). Windows 7 Enterprise.

gerzenstl’s picture

Status: Needs work » Needs review

I tested the patch on a Vagrant machine. From the VM I needed to use a different domain on the "One time login url" link provided by the installer.

I think that a mention about --host parameter in the INSTALL.txt would be very helpful in these cases.

Other than that, the install experience was smooth.

gerzenstl’s picture

Status: Needs review » Needs work
pbirk’s picture

@maxstarkenburg and I are sitting together at DrupalCon now talking through the issue he points out in #253. We started from the beginning using his computer.

The crux of the issue is that it's very easy to miss or even lose the password and/or one-time login URL that appear in the quick-start commands output.

Here's what we experienced. This is after completely removing the Drupal directory and restarting with a fresh 8.6.x zip archive.

  • Ran quick-start in a small PowerShell window
  • The quick-start command output the username, password and one-time access URL
  • A browser automatically opened, led to the one-time login URL, then redirected to the home page
  • The home page displays the message, "You have just used your one-time login link. It is no longer necessary to use this link to log in. Please change your password."
  • The message doesn't include a link leading to a form to reset the password and there is no form to do so on the landing page.
  • Request log info gets output to PowerShell as soon as the website opens and for any new page requests. This can easily push the username, password and one-time access URL off the screen if the PowerShell window is small enough.

Now the following issues exist:

  • If the user navigates to the edit page for the Admin account, they need the current admin password to change the password.
  • If the user finds the one-time access URL and copy/pastes that in the browser, they get an access denied page

I think removing the request log output would help a lot. I'm also not sure why the first attempt to open the one-time access link redirected. That's not the behavior I saw on my system.

In addition to this description, Max is going to upload some screenshots.

maxstarkenburg’s picture

Here are the screenshots as promised in #256. This seems like a pretty serious UX issue to me. I barely noticed the password output in PowerShell the first time and overlooked the messaging suggesting I change it in the browser.

dawehner’s picture

Thank you for all the manual testing! So highly appreciate.
I'm curious whether adding a couple of new lines before and after would help?

dawehner’s picture

Issue summary: View changes

Added the video to the issue summary

pbirk’s picture

@dawehner: Do you mean whitespace above and below the password, etc... output to PowerShell? That might make it more noticeable, but wouldn't solve the potential problem of request log info pushing the info off the screen.

heddn’s picture

If we go the extra work of opening a browser, the attention now shifts to the browser. Few will ever look at the console again. I suggest that if we open the browser, we send folks to the user edit page where they can set the password. This is contrary to what was discussed in #228 and #146. We want folks to set the password. Otherwise they will forget to do that.

Most new users of Drupal I observe who use drush to generate a one-time long will almost always change the password on the user edit page. That is the desired outcome. Still print the password, yes. But also encourage the desired behaviour as well.

jonathanshaw’s picture

One option would be to set username to "admin" and password to "password" and in addition to outputting that message in the console, also use drupal_set_message to tell people this in the browser too.

Another would be to put a one time login in drupal_set_message.

Mile23’s picture

Issue summary: View changes

This setup is not so people can experience the thrill of changing their password. :-) It's so they can type a command and magically demo Drupal.

The initial password after installing is a random string, so it's unlikely to be easy to hack. We could make it a UUID or maybe longer than it is currently, if there's some concern. We shouldn't display the password in cleartext, and only give the one-time login URL.

Opening the browser should be the front page because that's what we're trying to show off. Ideally, we'd have a demo site they could explore that has some tour type stuff, but demo_umami is pretty good, too.

If you hit control-c and stop the server and then restart it, it should (and does) open the same site, and will give you a different one-time login link.

Let's add a message before the install process (so there's time to read it before it scrolls away) saying: "Don't do this on a production server."

Mile23’s picture

Issue summary: View changes

Oops, accidentally changed IS.

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.47 KB
41.29 KB

But this is supposed to be easy to use. This works for me on OSX. Can someone test on Windows. If you want the server request output just add -v to the command.

Note we had problems with Windows and Symfony's process component so I've done this without using this.

dawehner’s picture

But this is supposed to be easy to use. This works for me on OSX. Can someone test on Windows. If you want the server request output just add -v to the command.

That is a good point!

ressa’s picture

This setup is not so people can experience the thrill of changing their password. :-) It's so they can type a command and magically demo Drupal.

I couldn't agree more.

ressa’s picture

Double post.

dawehner’s picture

Issue summary: View changes

I updated the issue summary a bit.

dawehner’s picture

Just some tiny improvements.

One thing we could do in a follow up: We could look into the batch process of the actual installation process.

Anonymous’s picture

Fantastic work!

I checked the #270 patch for Windows 10. Everything works fine: the site is successfully installed with different profiles (and languages) and automatically opens in the browser!

Only a couple of minor wishes:

quick-start__help-info.png

First usage line bit confused:
quick-start [options] [--] [<install-profile>]
Why [<install-profile>] goes last, if in the examples below it goes first (I understand that there is no difference, but it's confusing).

And what is [--]?

Maybe better:
quick-start [<install-profile>] [--options]
Or even just delete this line?


Example usage, which would help me:
quick-start --host my-site.com --port 80
Because developers (I'm) can use specialized utilities to work with sites that allow access only by folder-name with default port. Not by localhost + special port.
quick-start__run.png

First line contains placeholder %message%:

 0/15 [>---------------------------]
%message%

Announced 15 steps (if you choose a langcode). But
15/15 [============================]
Завершение переводов

16/16 [============================]
Завершение переводов

17/17 [============================]
Завершение переводов

17/17 [============================]

If the language is not selected, then 14/14 steps take place without problems.

alexpott’s picture

@vaplas the help output is the default output from Symfony's console. If you think upstream improvements are in order then we should do them upstream.

The patch attached should fix the %message% and step counting for translations. Also wow importing translations is slow.

alexpott’s picture

Add usage suggested by @vaplas.

alexpott’s picture

Anonymous’s picture

FileSize
33.65 KB

@vaplas the help output is the default output from Symfony's console.

Hah, I'm lol :)

@alexpott, many thanks for clarifying this point and the realization of everything else! #274 looks perfect! RTBC++

heddn’s picture

Wrt #263 (which I whole heartedly support), should we default to 'demo_umami'? Or standard? For someone who doesn't know anything about drupal, defaulting to which of the profiles should be selected could give the new user a helping hand. If they just select enter, then they would launch the demo?/standard?/minimal profile by default.

If we go this route, I really like the suggestion in #263 to default to a demo install profile.

heddn’s picture

Status: Needs review » Needs work

The following is what I currently get if I don't enter anything in response to the profile. While not bad, we could probably be a little more failsafe and check for an empty profile name.

$ php core/scripts/drupal quick-start 

 Select an installation profile [Install with commonly used features pre-configured.]:
  [standard  ] Install with commonly used features pre-configured.
  [minimal   ] Build a custom site without pre-configured functionality. Suitable for advanced users.
  [demo_umami] Install an example site that shows off some of Drupal’s capabilities.
 > 

 6/18 [▓▓▓▓▓▓▓▓▓░░░░░░░░░░░░░░░░░░░]
Set up database

In install.core.inc line 1159:
                                                                                                                                    
  <ul>                                                                                                                              
  <li>To start over, you must empty your existing database and copy <em>default.settings.php</em> over <em>settings.php</em>.</li>  
  <li>To upgrade an existing installation, proceed to the <a href="core/scripts/update.php">update script</a>.</li>                 
  <li>View your <a href="http://:core/scripts">existing site</a>.</li>                                                              
  </ul>                                                                                                                             
                                                                                                                                    

quick-start [--langcode [LANGCODE]] [--site-name [SITE-NAME]] [--host [HOST]] [--port [PORT]] [-s|--suppress-login] [--] [<install-profile>]

mradcliffe’s picture

That's odd. What operating system and shell, @heddn?

I tried with windows 10 pro. and PowerShell with the patch in #274.

1. Downloaded PHP 7.1.16 binaries and extracted to C:\Program Files\PHP\php-7.1.16.
2. Installed Composer via Composer Setup.exe, and chose to use the PHP 7.1.16 php.exe.
3. Cloned drupal 8.6.x branch into c:\dev\drupal (using TortoiseGit).
4. Downloaded the ptach file in #274.
5. Opened PowerShell terminal (through VS Code).
6. Changed directory to C:\dev\drupal
7. Ran git apply 2911319-2-274.patch
4. Ran composer install. Realized I needed to enable gd, curl and pdo_sqlite in C:\Program Files\PHP\php-7.1.16\php.ini.
5. Ran composer install
6. Ran php core/scripts/drupal quickstart
7. Hit enter
8. Logged into an admin in my browser at http://127.0.0.1:8888/ and I got standard profile by default.

PS C:\dev\drupal> php .\core\scripts\drupal quick-start

 Select an installation profile [Install with commonly used features pre-configured.]:
  [standard  ] Install with commonly used features pre-configured.
  [minimal   ] Build a custom site without pre-configured functionality. Suitable for advanced users.
  [demo_umami] Install an example site that shows off some of Drupal’s capabilities.
 >

 0/18 [>---------------------------]
Installing Drupal
18/18 [============================]
Congratulations, you installed Drupal!
Username: admin
Password: zhx9YKpbwMa3BQQ-
Drupal development server started: <http://127.0.0.1:8888>
This server is not meant for production use.
One time login url: <http://127.0.0.1:8888/user/reset/1/1523902268/mB4QkGanwB86cn_nmaHnPhuk2RstDuLf2fJdrnYzMkM/login>
Press Ctrl-C to quit.

Edit: forgot that I also saw I needed to uncomment pdo_sqlite.

heddn’s picture

$ more /etc/issue
Linux Mint 18.3 Sylvia

Standard shell.

mradcliffe’s picture

Thanks, heddn.

I tried on my custom Slack box at home, but wasn't able to reproduce when hitting enter at the prompt. It successfully installed to the standard profile. I also tried putting a space, which had the same result (probably because spaces equate to false?).

I was only able to reproduce the issue when I starting over, I cleared out settings.php, but _not_ all the content of the files directory, which gave me that error because .sqlite file was still there.

heddn’s picture

re #280: that's it, I forgot to clear out my files directory. I deleted settings.php. I didn't think about hunting for a sqlite file. I figured its name was semi-random at time of install so I wouldn't need to worry about it. Still NW or RTBC?

mradcliffe’s picture

Doing another review I noticed INSTALL.txt:

+Download and extract the Drupal package

Needs trailing colon?

+- cd /path/to/drupal-8.x.x
+// Installs Drupal and open it up
+- php core/scripts/drupal quick-start

"Installs Drupal and open it up". Huh? Do we mean "Now run the following command that installs Drupal"? Do we even need that? Maybe we should nuke that line?

+
+Prerequisites:
+- PHP 5.5.9 (or greater) (https://php.net).

I'm not sure if anyone's manually tested in PHP 5.5.9 although the test bot run should suffice. It's not added so I added a re-test of that.

mradcliffe’s picture

Here's a patch to fix my nits and back to Needs review.

heddn’s picture

+++ b/core/INSTALL.txt
@@ -10,6 +11,32 @@ CONTENTS OF THIS FILE
+- curl -sS https://ftp.drupal.org/files/projects/drupal-8.x.x.zip --output drupal-8.x.x.zip

That last patch reminds me of a gripe I had when first writing the quickstart readme docs. It rather painful for first time users to know the latest stable version. Not to mention its hard to script in a predictable manner.

Could we have some niceness like github and others have? Where latest is an alias to the most recent stable release? That would clean-up this section of the docs a lot. And make it possible for a predictable and scriptable download of drupal.

https://ftp.drupal.org/files/projects/drupal-8-latest.zip
https://ftp.drupal.org/files/projects/drupal-7-latest.zip

phenaproxima’s picture

Could we have some niceness like github and others have? Where latest is an alias to the most recent stable release? That would clean-up this section of the docs a lot. And make it possible for a predictable and scriptable download of drupal.

+1000 for this idea, but it should not block committing this patch. Let's defer that to a follow-up issue.

heddn’s picture

Status: Needs review » Reviewed & tested by the community

I've opened #2961550: Project download URL for latest stable release to address #284.

We're super close on this.

  1. +++ b/core/INSTALL.txt
    @@ -10,6 +11,32 @@ CONTENTS OF THIS FILE
    +- curl -sS https://ftp.drupal.org/files/projects/drupal-8.x.x.zip --output drupal-8.x.x.zip
    +- unzip drupal-8.x.x.zip
    +- cd /path/to/drupal-8.x.x
    

    8.y.z is the more typical method to list minor and patch version number.

  2. +++ b/core/INSTALL.txt
    @@ -10,6 +11,32 @@ CONTENTS OF THIS FILE
    +result in logging in to your new site in your web browser.
    

    Lots of repetition of the word 'in'. Can we just say:

    ...will result in opening the new site in your web browser.

  3. +++ b/core/INSTALL.txt
    @@ -10,6 +11,32 @@ CONTENTS OF THIS FILE
    +You may need to configure server settings such as the host address or port. Run
    +php core/scripts/drupal quick-start --help for a list of available options.
    

    NOTE: should prefix this too. And consider moving it before the previous note.

heddn’s picture

Status: Reviewed & tested by the community » Needs work

Hmm, a little premature on RTBC. Let's see if we can fix these docs comments. And somehow figure out how to word that sqlite is a file and should be deleted in the install exception from #277

mradcliffe’s picture

How are these changes to INSTALL.txt?

1. I changed it to drupal-x.y.z for consistency as it is referenced already in the INSTALLATION section.
2. I moved the help section above the NOTE and reworded so that the command is on its own line with a bullet/hyphen indentation.
3. I referenced the REINSTALL section for how to start over.

I think the only major instruction is going to be to install PHP and to run php core/scripts/drupal quick-start because a user has to download Drupal and extract it to see core/INSTALL.txt (and that already has composer dependencies). I think those instructions are mainly for a Quick Start section on https://www.drupal.org/docs/8/install (or wherever else it is suitable in the work flow for evaluators).

dawehner’s picture

Issue tags: +First-Time Experience

Introducing a new tag for making install.txt discoverable: #2961649: Add a /INSTALL.txt pointing to /core/INSTALL.txt

alexpott’s picture

I've ummed and ahhed about adding something to the Drupal is already installed. message. I think it is important that this doesn't look like an error because you can use the quickstart command multiple times and it'll show you the message. Therefore I've wrapped the Drupal is already installed. in info tags which makes it green. And add a message which says If you want to reinstall, remove sites/default/files and sites/default/settings.php. to underline that this is choice.

heddn’s picture

Status: Needs review » Needs work

Something in the, "the site is already installed" logic isn't catching that the sqlite file still exists when the settings.php file is deleted. I still get the exception from install.core.inc, instead of the nice info error message in the latest interdiff. I'd paste the error, but it is the same as in #277. Nothing changed.

+++ b/core/INSTALL.txt
-- cd /path/to/drupal-8.x.x
+Follow the instructions in the REINSTALL section below to start over.

This refers to REINSTALL. But then REINSTALL refers users back to the non quick-start section.

We can fix that now or in a follow-up. Ideally we'd just tell folks to re-follow the install instructions without being specific about the step or type of install.

> 4. Follow the Installation Instructions above starting from Step 3 (Run the install script).

Mixologic’s picture

I fired this up last night to use it for some browser test troubleshooting, and had a couple trivial UX issues:

1.

[ERROR] You must have the SQLite PHP extension installed.

Apparently there are *two* different sqlite php extensions, sqlite3, and pdo_sqlite -> maybe add PDO in that message somewhere to disambiguate?

2. If the assumption is that this is going to be targeted at somebody with no drupal knowledge, then I think that we should not ask the user to make any choices, and that the bare php core/scripts/drupal quick-start command should probably default to either standard or umami profile. (probably umami), with instructions/documentation on how to get a standard/minimal profile for more advanced users by passing the name in on the command line. At the same time, this kind of UX bikeshed can totally be ignored, or put into a followup.

dawehner’s picture

Status: Needs work » Needs review
FileSize
42.29 KB
1.46 KB

Apparently there are *two* different sqlite php extensions, sqlite3, and pdo_sqlite -> maybe add PDO in that message somewhere to disambiguate?

I adapted the message and the link pointing to our own documentation.

2. If the assumption is that this is going to be targeted at somebody with no drupal knowledge, then I think that we should not ask the user to make any choices, and that the bare php core/scripts/drupal quick-start command should probably default to either standard or umami profile. (probably umami), with instructions/documentation on how to get a standard/minimal profile for more advanced users by passing the name in on the command line. At the same time, this kind of UX bikeshed can totally be ignored, or put into a followup.

I think it is quite valueable to show this power of Drupal quite early. I can totally see your argument, but yeah, I'm not sure.

dawehner’s picture

Sorry, I forgot to change the pointer to your internal documentation

ressa’s picture

After applying the patch (which seems to work fine) I get this warning:

$ wget -q -O - https://www.drupal.org/files/issues/2018-04-24/2911319-294.patch | git apply -
<stdin>:16: trailing whitespace.
----- 
warning: 1 line adds whitespace errors.
alexpott’s picture

Fixing whitespace in patch.

ressa’s picture

Thanks @alexpott, the warning is now gone.

Anonymous’s picture

Status: Needs review » Reviewed & tested by the community

It seems all the feedbacks were addressed, back to RTBC!

Perhaps some suggestions about the default settings, output info and documentation can be discussed in a follow-up issue, if necessary. In order not to combine the "provide command to install" and "prediction of the future" in one issue.

mradcliffe’s picture

@heddn's comment in #291 wasn't addressed about the error message not firing (correctly?).

I'm not sure if that blocks this issue or not.

larowlan’s picture

Only nits, none should block commit.

Couple of follow-ups worth exploring.

  1. +++ b/core/lib/Drupal/Core/Command/InstallCommand.php
    @@ -0,0 +1,337 @@
    +      static $started = FALSE;
    ...
    +        $started = TRUE;
    

    Inside this callable, we do have access to $this.

    Do you think we should set a class property instead of using a static?

  2. +++ b/core/lib/Drupal/Core/Command/InstallCommand.php
    @@ -0,0 +1,337 @@
    +      foreach (array_keys($profiles) as $profile_name) {
    +        $lev = levenshtein($install_profile, $profile_name);
    +        if ($lev <= strlen($profile_name) / 4 || FALSE !== strpos($profile_name, $install_profile)) {
    +          $alternatives[] = $profile_name;
    +        }
    +      }
    +      if (!empty($alternatives)) {
    +        $error_msg .= sprintf(" Did you mean '%s'?", implode("' or '", $alternatives));
    +      }
    

    this feels like a utility that would have merit elsewhere - do you think it warrant a follow up to create a standalone class?

  3. +++ b/core/lib/Drupal/Core/Command/ServerCommand.php
    @@ -0,0 +1,278 @@
    +    // Some services require a request to work. For example, CommentManager.
    +    // This is needed as generating the URL fires up entity load hooks.
    

    Worth a follow-up to try and undo this in CommentManager?

  4. +++ b/core/lib/Drupal/Core/Command/ServerCommand.php
    @@ -0,0 +1,278 @@
    +  protected function findAvailablePort($host) {
    ...
    +  protected function openBrowser($url, SymfonyStyle $io) {
    

    Worth a follow up to explore making these standalone utilities?

larowlan’s picture

Updating review credits for those who

- provided reviews that changed the shape of the patch
- provided significant issue summary updates
- provided testing reports that shaped the patch
- provided documentation of the process on their machine

There are a lot of comments, if you think I've missed anyone, let me know.

Mile23’s picture

run-tests.sh also does an open-browser type thing: https://cgit.drupalcode.org/drupal/tree/core/scripts/run-tests.sh#n1526

dawehner’s picture

Here are a couple of follow ups:

larowlan’s picture

Issue tags: +Needs change record

We need a change notice here.

I discussed with @kim.pepper who is willing to write one today.

kim.pepper’s picture

Added a draft change notice. https://www.drupal.org/node/2969396

  • larowlan committed 573702f on 8.6.x
    Issue #2911319 by alexpott, dawehner, mglaman, mradcliffe, heddn, pbirk...
larowlan’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -Needs change record +8.6.0 highlights

Committed 573702f and pushed to 8.6.x.

Published the change record.

To everyone who was involved here, thank you so much, this is a significant outcome for Drupal

Mile23’s picture

dawehner’s picture

I'm wondering whether this new feature / task could be something which gets added to 8.5? The documentation on drupal.org could profit from that for example.

kay_v’s picture

Nice writeup in the change record, @kim.pepper ([#2969396]). FYI, I've taken it pretty much as is and created a proposed user guide page. Comments/improvements of course welcomed!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.