Docker Configuration

 Please note that this configuration is intended for local development only.  It has not been secured or optimized for production use.

1) Start with a fresh Composer install.  We're going to update the basic composer.json so do the very minimal setup step.  In a terminal window in your site parent directory (say ~/sites or ~/gitrepo) run this command.

$ composer create-project drupal/recommended-project mysite

2) We're going to work with a number of files in the "mysite" directory created by this command so cd mysite

3) Create a docker-compose.yml file using your favorite editor (it's based in large part on DDEV Solr which is for a three manager/three shard setup.)

NOTE: if you used a directory name besides "mysite" in the composer create-project command, you'll have to replace that string in all the examples where it appears with your directory name for the rest of the tutorial.  docker-compose uses the parent directory name of the project to name containers etc so it is kind of everywhere.

Your docker-compose.yml file should look like this:

version: '3'
services:
  drupal:
    image: drupal:php8.3-apache
    labels:
      - 'traefik.backend=nginx'
      - 'traefik.port=80'
      - 'traefik.frontend.rule=Host:drupal.mysite.localhost'
    expose:
      - 80
    volumes:
      - ./:/opt/drupal
    restart: always

  solr1:
    image: solr:8
    container_name: mysite-solr1
    restart: always
    expose:
      - 8983
    environment:
      SOLR_OPTS: -Djute.maxbuffer=50000000
      ZK_HOST: mysite-zoo1:2181
      SOLR_HEAP: 1g
    labels:
      - 'traefik.backend=solr'
      - 'traefik.port=8983'
      - 'traefik.frontend.rule=Host:solr.mysite.localhost'
    depends_on:
      - zoo1
    volumes:
      - ./solr-cloud/security.json:/var/security.json
      - .:/mnt/config
      - solr1:/var/solr
    command: bash -c "docker-entrypoint.sh solr zk cp file:/var/security.json zk:/security.json && exec solr-foreground"

  zoo1:
    image: zookeeper:3.6
    container_name: mysite-zoo1
    hostname: mysite-zoo1
    restart: always
    expose:
      - 2181
      - 7000
    environment:
      # The pre-trained OpenNLP models require a much bigger buffer.
      JVMFLAGS: -Djute.maxbuffer=50000000
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=mysite-zoo1:2888:3888;2181
      ZOO_4LW_COMMANDS_WHITELIST: mntr, conf, ruok
      ZOO_CFG_EXTRA: "metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider metricsProvider.httpPort=7000 metricsProvider.exportJvmInfo=true"
    volumes:
      - .:/mnt/config
volumes:
  solr1:

The first container is a basic drupal 10 container with a mount for the root filesystem to the /opt/docker directory.  The site is actually served from the /var/www/html directory which is a symlink to the /opt/docker/web directory.  This is suggested to add a layer of security since we can restrict the web services to the /var/www/html tree without exposing the rest of the file system.  We are using the 8 series instead of the 9-apache series because some of the packages we're using for this tutorial are not yet compatible with php 8.x last I checked.

The rest of this file is fairly complex but sets up a small solr cloud instance with three data shards and three management nodes.  We'll only be interacting with one, but in production you could load balance across all three to improve performance.

4. We will also make a docker-compose.override.yml file that includes some services that we would not necessarily want to run in a server environment, including the database container.  Often times there will be an existing database system in place we will point to in production.  Also included are Traefik (a DNS/SSL proxy container), MailHog (for intercepting and reading emails from the docker container), and portainer (a nice gui for looking at containers).  Besides the database these are all optional and you can move the database to the main docker-compose.yml file if you prefer.  By default services in both files will be loaded by "docker-compose up" and friends. 

version: '3'
services:
  db:
    image: mariadb:10.6.19
    environment:
      MYSQL_ROOT_PASSWORD: MyGreatPassword
      MYSQL_DATABASE: drupal
      MYSQL_USER: drupal
      MYSQL_PASSWORD: drupal
    volumes:
      - database:/var/lib/mysql
    labels:
      - 'traefik.backend=mariadb'
      - 'traefik.port=3306'
      - 'traefik.frontend.rule=Host:mariadb.mysite.localhost'
  mailhog:
    image: mailhog/mailhog
    ports:
      - "1025:1025"
      - "8025:8025"
    labels:
      - 'traefik.backend=mailhog'
      - 'traefik.port=8025'
      - 'traefik.frontend.rule=Host:mailhog.mysite.localhost'

  traefik:
    image: traefik:v1.7.16
    command: -c /dev/null --web --docker --logLevel=INFO --defaultEntryPoints='https' --entryPoints="Name:https Address::443 TLS:/certs/server.cert,/certs/server.key" --entryPoints="Name:http Address::80 Redirect.EntryPoint:https"
    ports:
      - 80:80
      - 443:443
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./certs:/certs/

volumes:
    database:

5. As per this tutorial (https://www.grzegorowski.com/traefik-proxy-development-server-self-signed-ssl):

You will need a .key and .crt files. They can be generated for self-signed certificates using the openssl. If you don't have it already, please check on Google (brew install openssl for Mac OS X with Homebrew packages manager). With the openssl ready to use execute (Note: There are only three commands, the last is one long line):

$ mkdir certs
$ cd certs
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout server.key -out server.cert

The only important question while running the openssl command is Common Name (e.g. server FQDN or YOUR name):. It should match the domain name of your server [in our case drupal.mysite.localhost]

 (or any other that you can point to your server), wildcards (*) are allowed.

After preparing the certificate files, you must change their access rights.

$ chmod 644 server.cert
$ chmod 600 server.key
$ cd ..

We also need to configure the Solr cloud basic authentication

$ mkdir solr-cloud
$ cd solr-cloud
$ wget https://raw.githubusercontent.com/drud/ddev-contrib/master/docker-compose-services/solr/solr/security.json
$ cd ..

6. Start containers with

$ docker-compose up -d

When you run docker-compose ps or docker ps you should see something like this

$ docker-compose ps
       Name                     Command               State                                          Ports                                       
-------------------------------------------------------------------------------------------------------------------------------------------------
mysite-solr1         docker-entrypoint.sh bash  ...   Up       8983/tcp                                                                          
mysite-zoo1          /docker-entrypoint.sh zkSe ...   Up       2181/tcp, 2888/tcp, 3888/tcp, 7000/tcp, 8080/tcp                                  
mysite_db_1          /docker-entrypoint.sh mysqld     Up       3306/tcp                                                                          
mysite_drupal_1      docker-php-entrypoint apac ...   Up       80/tcp                                                                            
mysite_mailhog_1     MailHog                          Up       0.0.0.0:1025->1025/tcp,:::1025->1025/tcp, 0.0.0.0:8025->8025/tcp,:::8025->8025/tcp
mysite_portainer_1   /portainer --no-auth -H un ...   Exit 1                                                                                     
mysite_traefik_1     /traefik -c /dev/null --we ...   Up       0.0.0.0:443->443/tcp,:::443->443/tcp, 0.0.0.0:80->80/tcp,:::80->80/tcp

If you run into any containers which crash after starting, the most common error is a port overlap.  Make sure you aren't running containers from other projects and shut down any web servers you're running locally and try again. 

Lando, DDEV, or other Docker-based tools could be the cause, and the easiest method is to stop all containers, and run the previous command again. From Docker: How to Stop and Remove All Containers at Once:

$ docker ps -aq | xargs docker stop

Otherwise you can use 

$ docker logs <container id>
$ docker logs mysite_drupal_1

This will dump the current logs for that container to your terminal window and can help debug the problem.

7. Add in package requirements and install drupal locally for easy customization.  I'm running this inside the docker container, but if you have composer set up on your local it should work outside the container also.  I left in a common error for clarity:

$ docker-compose exec drupal bash
root@49385401d384:/opt/drupal# composer require drupal/search_api_solr drupal/geofield drupal/geocoder drupal/address 'drupal/search_api_location:^1.0@alpha'

If you want to use the Drush command line tool for managing drupal you can also require that.  It does add a bunch of packages and make the image fairly large which is why some people like to leave it out for container based installs.

root@0caa7d89d3cc:/opt/drupal# composer require drush/drush

If you get an error message about memory limits, telling PHP to ignore its internal memory limit for the run will usually allow you to continue.  Runtime requirements are much lower than those needed by composer's dependency checker.

Fatal error: Allowed memory size of 1610612736 bytes exhausted (tried to allocate 4096 bytes) in phar:///usr/local/bin/composer/src/Composer/DependencyResolver/Solver.php on line 223

Check https://getcomposer.org/doc/articles/troubleshooting.md#memory-limit-errors for more info on how to handle out of memory errors.

root@31aed1a81c99:/opt/drupal# php -d memory_limit=-1 /usr/local/bin/composer require drupal/search_api_solr:^4 drupal/geofield drupal/geocoder drupal/address 'drupal/search_api_location:^1.0@alpha' drush/drush

On a recent check of this tutorial I did run into an instance where the mysql client wasn't installed so some drush commands didn't work as expected.  This isn't entirely unexpected since logging into containers and mucking around is generally not standard practice and it makes the container much larger which is also not desirable in many cases.  It is however, much easier to interact with Drupal this way so I often do it anyway.

root@5fed3490f298:/opt/drupal# drush sql-drop
 [warning] The shell command 'mysql' is required but cannot be found. Please install it and retry.
root@5fed3490f298:/opt/drupal# apt update
Get:1 http://deb.debian.org/debian buster InRelease [122 kB]                                          
Get:2 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://deb.debian.org/debian buster/main amd64 Packages [7906 kB]
Get:5 http://security.debian.org/debian-security buster/updates/main amd64 Packages [308 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [15.2 kB]
Fetched 8469 kB in 5s (1774 kB/s)                         
Reading package lists... Done
Building dependency tree       
Reading state information... Done
1 package can be upgraded. Run 'apt list --upgradable' to see it.
root@5fed3490f298:/opt/drupal# apt-get install mariadb-client
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libaio1 libconfig-inifiles-perl libdbd-mysql-perl libdbi-perl libmariadb3 libreadline5 libsnappy1v5
  libterm-readkey-perl mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common mysql-common
Suggested packages:
  libclone-perl libmldbm-perl libnet-daemon-perl libsql-statement-perl
The following NEW packages will be installed:
  libaio1 libconfig-inifiles-perl libdbd-mysql-perl libdbi-perl libmariadb3 libreadline5 libsnappy1v5
  libterm-readkey-perl mariadb-client mariadb-client-10.3 mariadb-client-core-10.3 mariadb-common mysql-common
0 upgraded, 13 newly installed, 0 to remove and 1 not upgraded.
Need to get 8265 kB of archives.
After this operation, 55.4 MB of additional disk space will be used.
Do you want to continue? [Y/n] y

8. You should be able to reach the site and finish the installer at the traefik DNS alias defined in the first docker-compose.yml file

https://drupal.mysite.localhost

If you want to access the solr console there is also a traefik alias for that

https://solr.mysite.localhost/solr/#/

However, for the solr config you will need to refer to it by it's container name and port because of some assumptions made by the admin UI.

The other services in docker-compose.override.yml have their own DNS aliases but their usage is really outside the scope of this tutorial.

https://mailhog.mysite.localhost/

https://portainer.mysite.localhost/ ("404 page not found" ...)

PLEASE NOTE: For geocoding to work, HTTPS referrers are required by most services at this time (at least Google Maps).  Google chrome at this time does not accept the certificate generated by docker as valid, so while these screen shots are of Chrome, I would suggest using Firefox or another browser if you have difficulties.  You will have to accept the self signed certificate to proceed in any event.

If you want to have some content to use for exploration of search features you may want to choose the experimental "Umami Food Magazine" install option, but you can also start with a Standard install and add content as you go.  For simplicity, screen shots in this tutorial will show a Standard install.

To connect to the database container, use the following settings, including setting the hostname to db (the base name of the database container as we defined it in our docker-compose file) under the "Advanced" menu.

Use the GUI installer as shown in the image below, or install with Drush:

root@5fe:/opt/drupal# drush site:install --db-url=mysql://drupal:drupal@db/drupal -y

After the installation is complete, you may face a certificate warning and files mismatch in your browser:

Firefox

Warning: Potential Security Risk Ahead
drupal.mysite.localhost uses an invalid security certificate.
The certificate is not trusted because it is self-signed.
Error code: MOZILLA_PKIX_ERROR_SELF_SIGNED_CERT

Chrome

Your connection is not private
Attackers might be trying to steal your information from drupal.mysite.localhost (for example, passwords, messages, or credit cards). Learn more
NET::ERR_CERT_AUTHORITY_INVALID

In both cases, click "Advanced" and then "Accept the risk and continue" or "Proceed to drupal.mysite.localhost (unsafe)"

You may be missing most images and CSS files, and seeing these errors in the console:

The resource from "https://drupal.mysite.localhost/" was blocked due to MIME type ("text/html") mismatch (X-Content-Type-Options: nosniff).

drupal.mysite.localhost Loading failed for the <script> with source "https://drupal.mysite.localhost/".

The solution is to make the sites/default/files folder writable and clearing the caches:

root@cb83ccb6aaee:/opt/drupal# chmod 777 -R web/sites/default/files/
root@cb83ccb6aaee:/opt/drupal# drush cache:rebuild
 [success] Cache rebuild complete.

You should now be able to generate a one-time log in link with Drush, and log in as administrator:

root@cb83ccb6aaee:/opt/drupal# drush user:login -l https://drupal.mysite.localhost/
https://drupal.mysite.localhost/user/reset/1/1684093521/fzH2a7e_tPZ2tpYL4p-pJfCxh1qHqAadgaCXuCk7ZOg/login

Initial Module Setup

1. Enable the modules we need for the site.

You can also accomplish this from within the docker container using drush en search_api_solr search_api_solr_admin geocoder geocoder_address geocoder_field geocoder_geofield geofield

2. It is also recommended you remove the core search module as it causes performance issues when using solr_api_search and related modules.

This can also be accomplished from within the drupal docker container with drush pm:uninstall search