Problem/Motivation

Currently PHP images has too many layers and unpacked source code (which is useless after intall)

Steps to reproduce

see https://hub.docker.com/r/drupalci/php-8.2-apache/tags

docker history drupalci/php-8.2-apache:dev
IMAGE          CREATED         CREATED BY                                      SIZE      COMMENT
c5a294ce2e7d   2 days ago      /bin/sh -c #(nop)  CMD ["apache2-foreground"]   0B        
<missing>      2 days ago      /bin/sh -c #(nop)  EXPOSE 80                    0B        
<missing>      2 days ago      /bin/sh -c #(nop) WORKDIR /var/www/html         0B        
<missing>      2 days ago      /bin/sh -c #(nop)  ENTRYPOINT ["docker-php-e…   0B        
<missing>      2 days ago      /bin/sh -c curl -sS https://deb.nodesource.c…   193MB     
<missing>      2 days ago      /bin/sh -c apt-get update     && apt-get ins…   1.94MB    
<missing>      2 days ago      /bin/sh -c #(nop) COPY file:a72004d60bd8c098…   374B      
<missing>      2 days ago      /bin/sh -c curl -SL "https://bitbucket.org/a…   79.6MB    
<missing>      2 days ago      /bin/sh -c curl -o /tmp/composer-setup.php h…   7.91MB    
<missing>      2 days ago      /bin/sh -c #(nop) COPY file:db0647a203810f32…   4.32kB    
<missing>      2 days ago      /bin/sh -c #(nop) COPY file:8dfa7a7c5950d30d…   4.32kB    
<missing>      2 days ago      /bin/sh -c docker-php-ext-pecl-install APCu-…   91.5MB    
<missing>      2 days ago      /bin/sh -c #(nop) COPY multi:f3eb57a30eeba02…   6.17kB    
<missing>      2 days ago      /bin/sh -c set -xe  && buildDeps="         a…   657MB     
<missing>      2 days ago      /bin/sh -c #(nop) COPY file:8a5ac966d577d5c8…   601B      
<missing>      2 days ago      /bin/sh -c set -xe;   fetchDeps='   wget  ';…   13.4MB    
<missing>      2 days ago      /bin/sh -c #(nop)  ENV PHP_URL=https://php.n…   0B        
<missing>      2 days ago      /bin/sh -c #(nop)  ENV PHP_VERSION=8.2.10       0B        
<missing>      2 days ago      /bin/sh -c #(nop)  ENV GPG_KEYS=1729F83938DA…   0B        
<missing>      2 days ago      /bin/sh -c #(nop)  ENV PHP_LDFLAGS=             0B        
<missing>      2 days ago      /bin/sh -c #(nop)  ENV PHP_CPPFLAGS=-O2 -g      0B        
<missing>      2 days ago      /bin/sh -c #(nop)  ENV PHP_CFLAGS=-O2 -g        0B        
<missing>      2 days ago      /bin/sh -c mkdir -p /usr/local/etc/php/conf.d   0B        
<missing>      2 days ago      /bin/sh -c apt-get update && apt-get install…   312MB     
<missing>      11 months ago   /bin/sh -c echo "deb http://deb.debian.org/d…   57B       
<missing>      14 months ago   /bin/sh -c a2enmod rewrite     && a2dissite …   60B       
<missing>      14 months ago   /bin/sh -c #(nop) COPY file:bb8f26ae96cdb2cb…   676B      
<missing>      14 months ago   /bin/sh -c #(nop) COPY file:06249e0e634b354e…   440B      
<missing>      14 months ago   /bin/sh -c {   echo '<FilesMatch \.php$>';  …   237B      
<missing>      14 months ago   /bin/sh -c a2dismod mpm_event && a2enmod mpm…   68B       
<missing>      14 months ago   /bin/sh -c set -ex   && sed -ri 's/^export (…   9.92kB    
<missing>      14 months ago   /bin/sh -c apt-get update && apt-get install…   98.7MB    
<missing>      14 months ago   /bin/sh -c #(nop)  ENV TERM=xterm               0B        
<missing>      14 months ago   /bin/sh -c #(nop)  ENV DRUPALCI=TRUE            0B        
<missing>      14 months ago   /bin/sh -c #(nop)  CMD ["bash"]                 0B        
<missing>      14 months ago   /bin/sh -c #(nop) ADD file:3451708ab45bc1bcf…   124MB

Proposed resolution

Optimize build by
- removing unzipped source
- merging layers where possible to has less changes from https://github.com/docker-library/php/blob/master/8.2/bullseye/apache/Do...

Remaining tasks

review/commit

User interface changes

API changes

Data model changes

Issue fork drupalci_environments-3387737

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

andypost created an issue. See original summary.

andypost opened merge request !23

andypost’s picture

andypost
he/him
Primary language Russian
commented

Also upgraded APCu to https://pecl.php.net/package-changelog.php?package=APCu&release=5.1.22 as support for 8.2 added here

andypost’s picture

andypost
he/him
Primary language Russian
commented

Current amount of layers is 37, after fix 31

also removal of source code cleans up >200MB

/bin/sh -c set -xe && buildDeps=" a… 657MB
after
RUN /bin/sh -c set -xe && buildDeps=" … 439MB

Also instead of build dependencies for production it could use fixed set of libraries (instead of -dev) so this layer could become smaller

andypost’s picture

andypost
he/him
Primary language Russian
commented
Status: Active » Needs review

Pushing to dev to get new image and test core

Build log is https://git.drupalcode.org/project/drupalci_environments/-/jobs/93474 no warnings about deprecated apt-key

andypost’s picture

andypost
he/him
Primary language Russian
commented

The pipeline using new image https://git.drupalcode.org/issue/drupal-3315265/-/pipelines/20801

andypost’s picture

andypost
he/him
Primary language Russian
commented

https://hub.docker.com/r/drupalci/php-8.2-apache/tags in zipped format CI will download 465MB instead of 504M and unpacking will eat less CPU

EDIT also slightly less IO as php-src is no longer unpacked on every container start

  • andypost committed b268efff on dev 
    Add node and yarn to php-82-cli image #3387737
andypost’s picture

andypost
he/him
Primary language Russian
commented

Current layers unpacked (via docker history)

- PHP binaries - 152MB (could be minimized further by stripping debug symbols from binaries)
- libraries to run PHP - 175MB
- Nodejs with yarn - 187MB

Download sizes are

https://hub.docker.com/r/drupalci/php-8.2-apache/tags
- production - 504.88 MB
- dev image using cleanu-up - 465.43 MB after https://git.drupalcode.org/project/drupalci_environments/-/commit/bb749b...

https://hub.docker.com/r/drupalci/php-8.2-cli/tags
- no yarn&node - 135 MB
- php + node/yarn - 189.93 MB

PS: removed building php-cgi and php-dbg as unused

  • andypost committed 695ee9a9 on dev 
    php/8.2-cli: clean-up outdated options #3387737

- mysql, mcrypt and...
andypost’s picture

andypost
he/him
Primary language Russian
commented
Title: Minimize PHP image size » Split PHP image into php(cli/apache) and yarn(node/nightwatch)
Parent issue: » #3386474: [omnibus] Speed up gitlab ci runs
Related issues: +#3371963: Update Nightwatch to 3.x, +#3109556: Update from Yarn 1 to Yarn 4, +#3388646: Add option to enable JIT for CI PHP images

In a light of #3371963: Update Nightwatch to 3.x

andypost opened merge request !27

walkingdexter’s picture

walkingdexter commented

Most likely, the above changes are the cause of the problem described below.

Installation of additional PHP extensions via mlocati/docker-php-extension-installer (easy installation of extensions with their deps) fails with an error when using the image drupalci/php-8.2-apache:production.

tar (child): /usr/src/php.tar.xz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

The error doesn't occur when using images drupalci/php-8.0-apache:production, drupalci/php-8.1-apache:production and drupalci/php-8.2-apache:production-old.

I'm not sure if this is the appropriate place for this problem. However, someone may encounter the same error when setting up a GitLab CI configuration for their contrib project, which requires the installation of additional PHP extensions.

Thanks!

andypost’s picture

andypost
he/him
Primary language Russian
commented

Yes, that's right place to report it. Meantime the error points out that the script attempted to extract php source code vut that's wrong action - instead of rebuilding PHP itself it should use installed headers instead.
Moreover the most probably you'll need extra headers for pecl extension to build so it's not enough to just call a common script.

Contrib willing to install something must care about how to build this extras as the debian base image for different php versions vary

walkingdexter’s picture

walkingdexter commented

Meantime the error points out that the script attempted to extract php source code vut that's wrong action - instead of rebuilding PHP itself it should use installed headers instead.

As I can see in the install-php-extensions script the error comes from docker-php-source extract (standard utility in official PHP images). PHP sources are used only to get the list of bundled extensions.

Contrib willing to install something must care about how to build this extras as the debian base image for different php versions vary

The install-php-extensions script already supports a variety of Debian-based and Alpine-based images, as well as various PHP versions. Until recently it was a good solution to install PHP extensions (without additional overhead) in drupalci images too.

Is it possible to revert the change that deletes the /usr/src/php.tar.xz file? Or it's a major change and it's not for discussion? Some other useful scripts compatible with PHP-based images may rely on docker-php-source extract, and the above change may break them too.

andypost’s picture

andypost
he/him
Primary language Russian
commented

This scripts are supposed to build php from very limited base php one
but in CI we ship preconfigured for core/contrib needs
there's https://git.drupalcode.org/project/drupalci_environments/-/blob/dev/php/...

It installs compiler and using pecl to build

andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#3404082: Adopt (semver) versioning for DrupalCi images
walkingdexter’s picture

walkingdexter commented

Maybe this information will be useful.

The recommended approach to installing PHP extensions from GitLab CI guide doesn't work with drupalci/php-8.2-apache:production.

MongoDB example:

$ pecl install mongodb
WARNING: channel "pecl.php.net" has updated its protocols, use "pecl channel-update pecl.php.net" to update
downloading mongodb-1.17.2.tgz ...
Starting to download mongodb-1.17.2.tgz (2,064,433 bytes)
......................................................................................................................................................................................................................................................................................................................................................................................................................done: 2,064,433 bytes
846 source files, building
running: phpize
Configuring for:
PHP Api Version:         20220829
Zend Module Api No:      20220829
Zend Extension Api No:   420220829
Cannot find autoconf. Please check your autoconf installation and the
$PHP_AUTOCONF environment variable. Then, rerun this script.
ERROR: `phpize' failed

I also don't see any option to install a bundled PHP extension. For example, imap is available in pecl only since PHP 8.3.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Just checked scripts - they are broken because using apk (Alpinelinux based package manager) instead of apt so it just needs to fix scrips

ATM you can add following to install build deps apt-get install $PHPIZE_DEPS

andypost’s picture

andypost
he/him
Primary language Russian
commented

Sadly the same issue with official images https://github.com/docker-library/php/blob/master/8.3/bullseye/apache/do...

andypost’s picture

andypost
he/him
Primary language Russian
commented

As script to enable extensions is fixed (in 8.3 images ATM) #3404084: Add PCOV extension to images

It's recommended to use following commands to install extra PECL extensions

    - apt-get update && apt-get install -qy --no-install-recommends $PHPIZE_DEPS
    - pecl install mongodb && docker-php-ext-enable mongodb
andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#3156947: Create a new buildcontainer

andypost’s picture

andypost
he/him
Primary language Russian
commented

Polished PHP 8.1 image, now testing

drupalci/php-8.1-apache   dev               fdf0ce19de4e   21 minutes ago      681MB
drupalci/php-8.1-apache   <none>            eb25b116360c   6 hours ago         1.36GB

  • andypost committed 57bc0228 on dev 
    Modernize 8.2-8.3 images #3387737 and #3108643
andypost’s picture

andypost
he/him
Primary language Russian
commented

Created change record and gonna cherry-pick commits to production

  • andypost committed 5c0009b5 on production 
    Modernize 8.2-8.3 images #3387737 and #3108643


(cherry picked from...

  • andypost committed 82851eb6 on production 
    Minimize php:8.1-apache image #3387737


(cherry picked from commit...

  • andypost committed 9eb10fb6 on production 
    Modernize 8.2-8.3 images #3387737 and #3108643


(cherry picked from...

  • andypost committed c8d4da92 on production 
    Minimize php:8.1-apache image #3387737


(cherry picked from commit...
ptmkenny’s picture

ptmkenny commented

Coming from Slack, I'm adding a quick note here that it would be great to find an easier way to use bundled PHP extensions like sodium. For example, lcobucci/jwt requires sodium, and several Drupal modules depend on this package (e.g., jwt, firebase).

Hopefully we can find a way to test them on an official PHP image.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Thanks! That's another reason to have a base build of php image to produce from it shared extensions for sodium and others