Problem/Motivation

This has been discussed for several years, but usually deep in the weeds of installer refactoring issues.

The current installer relies heavily on the batch system and a wizard (database credentials, language, install profile, installing all the modules). This means you have to fill out several forms and watch a batch progress bar before you can do anything.

With linux distros, you often get a bootable disk that gives you a semi-working operating system without storage, which you can then install to disk. This allows for trying the distro without having to do as much initial work, it's a much more immediate experience, and it lets you browse the internet while the actual installer is running too.

I think we can get a lot closer to this with Drupal.

Proposed resolution

- ensure a new site can work without a database connection - will mean defaulting to file and memory-based implementations of some things, and adding the capability to read configuration from an export.

- the only choice at the start is to pick which install profile to run default configuration/content from and maybe language? Somehow the files directory needs to be optional too. This could be a single page form, with defaults so it's possible to simply click 'next'.

- We then give you a 'working' site immediately, with a reminder to configure the database to make it persistent - at which point you can either start from an install profile again or a config export of your current state. You then have to set up site name and other required things, but that could even be a tour + requirements warning rather than blocking? I try to use the UI installer at least sometimes so I can remember what it's like, and adding admin@example.com all the time (including when developing real sites until I know what that e-mail should be) gets annoying to be honest.

- @alexpott has suggested making the installer work like a config import instead of the one-off implementation we have today, that would be consistent with changing to this model.

Remaining tasks

Decide if this is something we want to do, and develop some actual requirements/wireframes of how it might look like.

User interface changes

API changes

Data model changes

Comments

catch created an issue. See original summary.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented

This might also be a way to resolve some of the issues around the 'demo install profile'. Since it'd provide a route from that demo install profile to a different install profile without having to wipe the site manually and re-install.

yoroy’s picture

yoroy CreditAttribution: yoroy at Roy Scholten commented

I only install using the GUI and it's quite a slog indeed. If we can get people to use Drupal without the current clicking and waiting that would be quite a win.

But only if we take care not to create (much) more frustration around the transition from file/memory based to database(d). That might be tricky since I'm reading that would mean people have to export/import their configuration manually. Ideally that would "just happen", but that might be quite an effort, I don't know :). There may be some room for letting people do some work because they've become invested (hopefully) and thus willing to do some work to keep what they have.

On the other hand, not always do you need to keep initial trials and experiments around. What if the system could somehow track and at some point go "You've created quite some content and configuration now. Looks like you're serious about this. If so and you want to keep what you have, best to make some things more definitive first then".

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Chapter Three commented

I think the general idea would be fabulous. This would be a true killer feature, maybe not for core, but I believe for distributions, this could be amazing!

- ensure a new site can work without a database connection - will mean defaulting to file and memory-based implementations of some things, and adding the capability to read configuration from an export.

I was always wondering whether we could default our DB connection to some sqlite database, maybe in the temp directory, and then create even tables lazily (which is what we do now), which would allow us to not even require those in memory databases.

- the only choice at the start is to pick which install profile to run default configuration/content from and maybe language? Somehow the files directory needs to be optional too. This could be a single page form, with defaults so it's possible to simply click 'next'.

For distributions it could be optional. To be honest, you could even have a language selector on the site, when the distribution decides to ship with multilingual enabled by default. Somehow the language switcher could then start a batch to download the required data?
The installer can then still allow people to decide whether its multilingual or not.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
CreditAttribution: alexpott at Chapter Three commented

One thing we need for this is #2788777: Allow a site-specific profile to be installed from existing config because that'll give us full config exports for install profiles to work off.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented

With sqlite there was #1226380: Generate minimal mocked or SQLite environment and use to install Drupal in full environment. The big thing that held that up was whether we can require sqlite - I don't think we can/should, so we'd need a fallback even if the fallback is a page you enter your db credentials on.

For distributions it could be optional. To be honest, you could even have a language selector on the site, when the distribution decides to ship with multilingual enabled by default. Somehow the language switcher could then start a batch to download the required data?

Isn't this more a usability/interface translation issue i.e. if you don't read English you can get to a translation as quickly as possible?

dawehner’s picture

dawehner
Primary language German
CreditAttribution: dawehner as a volunteer and at Chapter Three commented

Isn't this more a usability/interface translation issue i.e. if you don't read English you can get to a translation as quickly as possible?

That is a strong point. Actually when you use a linux live CD you get asked as well. Nevermind :)

yoroy’s picture

yoroy CreditAttribution: yoroy at Roy Scholten commented

#2911319: Provide a single command to install & run Drupal

kenorb’s picture

kenorb CreditAttribution: kenorb commented

Using Drush command, you can install and run Drupal (without a database connection, due to SQLite db file):

drush qd

See: What is the easiest way to install clean Drupal from scratch using drush

Does that meet the criteria?

kenorb’s picture

kenorb CreditAttribution: kenorb commented
Related issues: +#2910136: Experiment: package PHP libraries in a single Phar file
nod_’s picture

nod_
Primary language French
Location Lille
CreditAttribution: nod_ at Acquia commented
Issue tags: +Needs product manager review

Tagging to reassess the need of the issue based on the last few year's improvements (config installer etc.) and whether this is needed in core.

Before talking about technical issues, is that something we want for Drupal the product, does it still makes sense in 2021?

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
CreditAttribution: alexpott commented

I think given the amount of work this would take plus the existence of the quick start command means we could consider shelving this idea for now.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented
Related issues: +#2818085: Promote and allow downloading of vetted install profiles from the installer

Some of the low-level prerequisites to this are happening in their own right (like doing more lazy table creation and install-from-config).

We could speed up the experience of the installer by having the module installation run in a non-blocking batch while you're filling out the form with admin e-mail addresses etc. (issues like #1189464: Add an 'instant' queue runner).

I'm not sure the other changes proposed here gain us much over that tbh.

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented
Related issues: +#3417835: Convert the Standard install profile into a set of recipes, +#1170362: Install profile is disabled for lots of different reasons and core doesn't allow for that

I think now that

… means that this issue is becoming relevant (and desirable) again. Imagine being able to try various complete recipes without having to set up DB/filesystem etc.

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented
Category: Feature request » Plan

For background reading on implementation, there are these two very old issues, however since the installer hasn't changed much in the meantime, a lot of the considerations are still relevant:

#1226380: Generate minimal mocked or SQLite environment and use to install Drupal in full environment
#1530756: [meta] Use a proper kernel for the installer

I think the main focus of this issue should be - would it be desirable if the installer was a single form, ideally with as many defaults filled in as possible, which then takes you to a core recipe selection and/or project browser. If it is, then we can continue towards making it possible in the various issues that are trying to reduce the up-front installer work, but with getting to that point in mind a bit more.

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented

would it be desirable if the installer was a single form, ideally with as many defaults filled in as possible, which then takes you to a core recipe selection and/or project browser

This sounds not just desirable, but AMAZING! 😄

I'm not familiar with the two issues you linked. Is #1530756: [meta] Use a proper kernel for the installer a blocker for this?

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented

I'm not familiar with the two issues you linked. Is #1530756: [meta] Use a proper kernel for the installer a blocker for this?

Trying not to get into the depths of implementation too much here, but I guess we're only at #17 comments after seven years so maybe it won't hurt :P

The biggest issue we still have, which I also brought up back in 2011 in #679112-13: Time for system.module and most of includes to commit seppuku is that the installer depends on system module.

It does so less than it did in 2011, but we still rely on the batch API, which relies on the sessions table, which is defined in system_schema(), which is installed by install_begin_request().

We would need a combination of #3221051: [meta] Replace hook_schema() implementations with ::ensureTableExists() and/or an apcu or file-based sessions storage implementation so that we can either have a session without a database, or have a session as soon as an initial form is submitted (which could be sqlite in the public file system by default).

This might not be exclusive - if we still want to create user 1 and log the user in, we also need to do the {users_data} table which is user_schema(), although that least is only currently a dependency of the end of the installer instead of the beginning of it currently. Either way the general principle is the same - we need the installer to function as an app, which doesn't have circular dependencies on arbitrary modules, so that it can install project browser before user and system module if possible.

The other major part of this for me would be non-blocking batch UI, so that when you install a module from project browser or do other things in the installer, we can show an AJAX progress bar instead of an interstitial page that stops you looking at anything else. This would also especially help with installing translations, which can take minutes with the current installer (especially the config translations sync at the end).

edited to add:

Another major part which is what Wim's alluding to, is doing all of the initial module installs in a 'sandbox' mode - whether that's filesystem storage or sqlite or apcu. But then we'd need a way to export the current configuration, then actually install Drupal to a permanent database and import that config that's just been exported. Let's assume we don't create user/1 until that step and also warn the user that any content created is in sandbox mode.

andypost’s picture

andypost
he/him
Primary language Russian
CreditAttribution: andypost as a volunteer and at Skilld commented

There was issue to ship minimal sqlite db for installer but I can't find the issue

catch’s picture

catch
he/him
Primary language English
CreditAttribution: catch at Third and Grove commented

@andypost I think that was #1226380: Generate minimal mocked or SQLite environment and use to install Drupal in full environment.

To add onto #15/#17 there is maybe a more high level view of this:

1. Make the installer more minimal and self-contained - just requirements check and minimum configuration with defaults, selecting a language etc, but not site name, e-mails, user/1 etc. to enable Drupal to run in sandbox mode.

2. We can then go to the actual sandboxed Drupal install, but with project browser as the first thing you see, allowing you to browse through recipes and modules (local and/or remote) and try them out.

3. At some point we have to take the site from sandbox to a permanent site on a real database, but make that an optional process that you can do after experimenting.

4. Keep drush and maybe UI support for skipping sandbox mode, this might be 'install from existing config'.

andypost’s picture

andypost
he/him
Primary language Russian
CreditAttribution: andypost as a volunteer and at Skilld commented

Btw core already provides site-install (even from config) command and even #3089277: Provide core CLI commands for the most common features of drush

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers as a volunteer commented

but I guess we're only at #17 comments after seven years so maybe it won't hurt :P

hahhaha indeed 😆

The high-level view in #19 paints a more complete picture of the UX that this issue is aiming for. And #17 dives into some (very much appreciated!) detail of what it'd take at a technical level to get there.

It seems it all boils down to the Batch system and the infrastructure that needs. This makes me wonder … and I can't believe I'm about to write this but … do we truly need server-side sessions? Those are AFAICT necessary for A) determining how far a long-running operation got, B) security reasons, C) client closed connection and we want to continue the long running process.
However, in this ephemeral demo-only context with no sensitive data and no traffic being served other than this one user, can't we get away with not using (server-side) session? And instead track the necessary metadata in https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage? That requires the server to trust the client, but in this context … that doesn't matter anyway? 🤔

I probably am overlooking a crucial fact or five. Trying to think out of the box 🤓