Problem/Motivation

Once PHP 8.2 is available (with an appropriate release) we should add an environment for testing PHP 8.2 in DrupalCI

Choosing pre-release versions of PHP 8.2 that are worth testing

There is a documented policy for pre-release versions of PHP containers, which we should follow (could use updating):
https://www.drupal.org/drupalorg/docs/drupal-ci/drupalci-php-support-pol...

But in general:

  • Updating a pre-release php container is a little tricky because the pre-releases can be hosted in non-standard ways (in individual developer namespaces) and because they may not have up-to-date extensions that we need.
  • Ideally, we deploy a pre-release container only at the first beta phase
  • In any case we want to try to avoid updating the container for every patch release, and select new betas to deploy strategically to save infra time.

This issue will be linked as a related issue to #3283358: [META] Make Drupal 9/10 compatible with PHP 8.2

This issue will stay in 'postponed' until the appropriate release is available.

Proposed solution

To limit how much we have to chase head with this container- update it:

  • Once for alpha
  • Once for beta
  • Once for RC
  • Once for final

With an exception for an update if there's a known bug affecting us that has been fixed.

Remaining tasks

Make the new environment when this issue is 'un-postponed'

Issue fork drupalci_environments-3283449

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

hestenet created an issue. See original summary.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Parent issue: » #3283358: [META] Make Drupal 9/10 compatible with PHP 8.2
Related issues: -#3283358: [META] Make Drupal 9/10 compatible with PHP 8.2
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Issue summary: View changes
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Issue summary: View changes
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Issue summary: View changes
gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

As per the last update from @hestenet, on the meeting between Drupal Association and committers today, myself and Alex Bronstein from the core committer team was present and we agreed with the slack discussion prior with @xjm that having one PHP build per alphas, one per betas and one for RC and one for final stable sounds fine, unless the one build we have is itself buggy. @hestenet already updated the issue summary with this.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Created Alpinelinux packages with all required extensions (apcu, uploadprogress, xhprof, xdebug)

https://pkgs.alpinelinux.org/packages?name=php82-pecl*&branch=edge&arch=...

I was able to make 10.0.x core work after setting display_errors=0 to prevent deprecation messages,
otherwise session failed to start

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented
Status: Postponed » Active

Second alpha will be this week. Is that a good one to deploy to DrupalCI (with the next one being a beta that will be rolled out)? @andypost what do you think?

andypost’s picture

andypost
he/him
Primary language Russian
commented

As I see https://wiki.php.net/rfc/rng_extension and related very probably will be accepted before feature freeze

I fond the current alpha1 mostly good enough to start with for CI, I bet it will be easy to update CI image as next releases will be out

gábor hojtsy’s picture

gábor hojtsy
he/him
Primary language Hungarian
Location Hungary
commented

@andypost: the current agreement is that one instance of alphas, one instance of betas, etc. will be rolled out on DrupalCI to not get into a chasing pattern. So is alpha2 good enough for that? That would be good to verify.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Hi @andypost - I didn't quite understand - are you saying go ahead with a container for alpha2 or wait for alpha3?

andypost’s picture

andypost
he/him
Primary language Russian
commented

I mean to wait alpha2 as it should be tagged tonight

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Understood! Thank you for clarifying

andypost’s picture

andypost
he/him
Primary language Russian
commented

Alpha2 packaged and will be announced in 2 days https://downloads.php.net/~pierrick/

andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#3202016: Let GDToolkit support AVIF image format

This image should build with Avif image support, also would be great to add it to 8.1 one too

That's required to test #3202016: Let GDToolkit support AVIF image format

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Thanks @andypost - I was just about to ask if you could find one with AVIF

andypost’s picture

andypost
he/him
Primary language Russian
commented

@hestenet Here's how it works https://gitlab.alpinelinux.org/alpine/aports/-/blob/master/testing/php82...

- PHP needs to use internal GD library
- the dependency is libavif https://packages.debian.org/source/sid/libavif
- (heif is not yet supported) libavif >= 0.8.2 https://github.com/php/php-src/blob/master/ext/gd/config.m4#L85

andypost’s picture

andypost
he/him
Primary language Russian
commented
Issue tags: +PHP 8.2
andypost’s picture

andypost
he/him
Primary language Russian
commented
Status: Active » Needs review

Queued annotation patch https://www.drupal.org/pift-ci-job/2420836

@hestenet is avif image-type support added?

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Just updating per ongoing Slack convos (and @mixologic knows more than I)

But yes, the new environment should have libavif - however I know we're troubleshooting whether there are other dependencies that are not working properly.

andypost’s picture

andypost
he/him
Primary language Russian
commented

More details on libavif from slack - CI using Debian "buster" which has very outdated version of libavif (minimally required for reading images)

$ docker run --rm -it drupalci/php-8.1-apache:production bash
...
root@ecc332326535:/var/www/html# apt list --installed |grep avif
...
libavif-dev/buster-backports,now 0.8.2-2~bpo10+1 amd64 [installed]
libavif7/buster-backports,now 0.8.2-2~bpo10+1 amd64 [installed,automatic]

As I see it missing libaom in dependencies https://packages.debian.org/buster-backports/libavif7 which is required to create AVIF images (encoder) https://packages.debian.org/bullseye-backports/libaom3

The suggestion to base CI images on "bullseye" which has updated libavif when using "bullseye-backports" https://packages.debian.org/search?searchon=sourcenames&keywords=libavif

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

A new testing environment called: "9.3.x-dev test with PHP 8.1 & MySQL 5.7 w/Bullseye Upgrades" has been added, which may be useful to get the bullseye upgrades working correctly before we apply them to our other testing environments.

Also, per @andypost - beta of php 8.2 is now available: https://www.php.net/archive/2022.php#2022-08-04-2

andypost’s picture

andypost
he/him
Primary language Russian
commented

RC1 announced https://www.php.net/archive/2022.php#2022-09-01-4

PS: beta3-rc1 changes broke xhprof but 2.3.6/2.3.7 release has fixes

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

PHP 8.2 container has been updated to RC1.

hestenet credited Mixologic.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

PHP 8.2 container has been updated to RC2

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Updated environment to PHP 8.2 RC3 - queuing a one-off test: https://www.drupal.org/pift-ci-job/2486742

andypost’s picture

andypost
he/him
Primary language Russian
commented

Somehow container images are not been deployed, so the run from #27 has RC2 https://dispatcher.drupalci.org/job/drupal8_core_regression_tests/67255/...

Also queued https://www.drupal.org/pift-ci-job/2486870 it should be green on 8.2.0RC3

andypost’s picture

andypost
he/him
Primary language Russian
commented 
$ docker run --rm drupalci/php-8.2-apache:production php -v
PHP 8.2.0RC2 (cli) (built: Sep 19 2022 17:14:37) (NTS)
Copyright (c) The PHP Group
Zend Engine v4.2.0RC2, Copyright (c) Zend Technologies
    with Zend OPcache v8.2.0RC2, Copyright (c), by Zend Technologies
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

We caught up on #28 and #29 in Slack, but I forgot to close the loop here, the rc3 container is properly deployed now.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Not sure it's related but javascript tests too often has random fails https://www.drupal.org/pift-ci-job/2488388 like that

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Per: @andypost's request:

PHP 8.2 and Sqlite 3.27 env added - and an initial test queued: https://www.drupal.org/pift-ci-job/2490356

effulgentsia’s picture

effulgentsia commented

Are we sure it's SQLite 3.27? If the DrupalCI container for PHP 8.2 is on Debian Bullseye rather than Buster, then by default SQLite would be 3.34. Are we explicitly downgrading SQLite to 3.27? For what it's worth, I think it's good if the PHP 8.2 environment is on SQLite 3.34 (as long as the label is adjusted to reflect that): it's nice to have the higher version available for testing, and the 8.1 environment is still on 3.27 (for as long as it remains on Buster) for when we need confirmation that things still work on the lower version.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Checked container and it's SQLite Library => 3.34.1

https://dispatcher.drupalci.org/job/drupal8_core_regression_tests/67899/...

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Ah! I didn't even think to check - I've updated the label. Thanks for the catch.
https://www.drupal.org/pift-ci-job/2491600

andypost opened merge request !4

andypost’s picture

andypost
he/him
Primary language Russian
commented

According to https://backports.debian.org/Instructions/

- the repo should be deb http://deb.debian.org/debian bullseye-backports main
- as backports repo has less priority package should add suffix [package name]/bullseye-backports
- moreover libaom intermediate dependency should not be in the requirements but installed as dependency of libavif-dev

Filed MR to fix it https://git.drupalcode.org/project/drupalci_environments/-/merge_requests/4

If you're using current container (drupalci/php-8.2-apache:production or drupalci/php-8.1-apache:production) just use following line to upgrade package apt update && apt install libavif-dev/bullseye-backports

heddn’s picture

heddn
Primary language English
Location Illinois
commented
Status: Needs review » Reviewed & tested by the community

This looks great. LGTM.

hestenet closed merge request !4

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Status: Reviewed & tested by the community » Needs work

Reformatted @andypost's MR for our M4 template generator: https://git.drupalcode.org/project/drupalci_environments/-/commit/efd7a3...

However, when building the containers we get: 

13:45:23 E: Release 'bullseye-backports' for 'libavif-dev' was not found
13:45:24 The command '/bin/sh -c set -xe 	&& buildDeps="         apache2-dev         libcurl4-openssl-dev         libedit-dev         libfreetype6-dev         libicu-dev         libjpeg62-turbo-dev         libonig-dev         libkrb5-dev         libpng-dev         zlib1g-dev         libmcrypt-dev         libpng-dev         libwebp-dev         libavif-dev/bullseye-backports         libaom-dev         libpq-dev         libsqlite3-dev         libssl-dev         libtidy-dev         libxml2-dev         libxslt1-dev         libyaml-dev         libzip-dev         ncurses-dev     " 	&& apt-get update && apt-get install -y $buildDeps --no-install-recommends && rm -rf /var/lib/apt/lists/* 		&& docker-php-source extract 	&& cd /usr/src/php 	&& export CFLAGS="-O2 -g" 		CPPFLAGS="-O2 -g" 		LDFLAGS="" 	&& ./configure             --with-config-file-path=/usr/local/etc/php             --with-config-file-scan-dir=/usr/local/etc/php/conf.d             --enable-ftp             --enable-mbstring             --enable-mysqlnd             --with-curl             --with-libedit             --with-zlib             --with-kerberos             --with-openssl             --with-mysql=mysqlnd             --with-mysqli=mysqlnd             --with-pdo-mysql=mysqlnd             --with-pdo-sqlite             --with-pdo-pgsql             --enable-phpdbg             --with-readline             --with-freetype             --with-zlib-dir             --with-jpeg             --with-mcrypt             --with-xsl             --with-tidy             --with-xmlrpc             --with-gettext=shared             --enable-gd             --with-webp             --with-avif             --with-pear             --enable-sockets             --enable-exif             --with-zip             --enable-soap             --enable-sysvsem             --enable-cgi             --enable-sysvshm             --enable-shmop             --enable-pcntl             --enable-bcmath             --enable-xmlreader             --enable-intl             --enable-opcache             --with-apxs2     && make -j "$(nproc)"     && make install     && make clean' returned a non-zero code: 100
13:45:24 Build step 'Execute shell' marked build as failure
13:45:25 Finished: FAILURE
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Original MR still had 'buster-backports' in one place - going to try that fix.

Also going to remove the explicit libaom-dev declaration, since supposedly libavif-dev/bullseye-backports should build with it

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

New dependency issues: 

13:57:11 Reading package lists...
13:57:11 + apt-get install -y apache2-dev libcurl4-openssl-dev libedit-dev libfreetype6-dev libicu-dev libjpeg62-turbo-dev libonig-dev libkrb5-dev libpng-dev zlib1g-dev libmcrypt-dev libpng-dev libwebp-dev libavif-dev/bullseye-backports libpq-dev libsqlite3-dev libssl-dev libtidy-dev libxml2-dev libxslt1-dev libyaml-dev libzip-dev ncurses-dev --no-install-recommends
13:57:12 Reading package lists...
13:57:12 Building dependency tree...
13:57:12 Reading state information...
13:57:12 Some packages could not be installed. This may mean that you have
13:57:12 requested an impossible situation or if you are using the unstable
13:57:12 distribution that some required packages have not yet been created
13:57:12 or been moved out of Incoming.
13:57:12 The following information may help to resolve the situation:
13:57:12 
13:57:12 The following packages have unmet dependencies:
13:57:12  libavif-dev : Depends: libavif13 (= 0.9.3-2~bpo11+1) but it is not going to be installed
13:57:12 E: Unable to correct problems, you have held broken packages.
13:57:13 The command '/bin/sh -c set -xe 	&& buildDeps="         apache2-dev         libcurl4-openssl-dev         libedit-dev         libfreetype6-dev         libicu-dev         libjpeg62-turbo-dev         libonig-dev         libkrb5-dev         libpng-dev         zlib1g-dev         libmcrypt-dev         libpng-dev         libwebp-dev         libavif-dev/bullseye-backports         libpq-dev         libsqlite3-dev         libssl-dev         libtidy-dev         libxml2-dev         libxslt1-dev         libyaml-dev         libzip-dev         ncurses-dev     " 	&& apt-get update && apt-get install -y $buildDeps --no-install-recommends && rm -rf /var/lib/apt/lists/* 		&& docker-php-source extract 	&& cd /usr/src/php 	&& export CFLAGS="-O2 -g" 		CPPFLAGS="-O2 -g" 		LDFLAGS="" 	&& ./configure             --with-config-file-path=/usr/local/etc/php             --with-config-file-scan-dir=/usr/local/etc/php/conf.d             --enable-ftp             --enable-mbstring             --enable-mysqlnd             --with-curl             --with-libedit             --with-zlib             --with-kerberos             --with-openssl             --with-mysql=mysqlnd             --with-mysqli=mysqlnd             --with-pdo-mysql=mysqlnd             --with-pdo-sqlite             --with-pdo-pgsql             --enable-phpdbg             --with-readline             --with-freetype             --with-zlib-dir             --with-jpeg             --with-mcrypt             --with-xsl             --with-tidy             --with-xmlrpc             --with-gettext=shared             --enable-gd             --with-webp             --with-avif             --with-pear             --enable-sockets             --enable-exif             --with-zip             --enable-soap             --enable-sysvsem             --enable-cgi             --enable-sysvshm             --enable-shmop             --enable-pcntl             --enable-bcmath             --enable-xmlreader             --enable-intl             --enable-opcache             --with-apxs2     && make -j "$(nproc)"     && make install     && make clean' returned a non-zero code: 100
13:57:13 Build step 'Execute shell' marked build as failure
13:57:14 Finished: FAILURE
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

https://git.drupalcode.org/project/drupalci_environments/-/commit/640ee7...

Explicitly declaring the additional dependencies largely works, except that some of them are not available in either buster or bullseye-backports, particularly these two:

https://packages.debian.org/bullseye/libdav1d4
https://packages.debian.org/bullseye/libgav1-0

I'm becoming tempted to try moving the whole image to bullseye, though I'm worried that may break all sorts of other upstream things...

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Alright after a bunbch of troubleshooting - 8.1 and 8.2 should now both be on bullseye and *hopefully* getting everything we need about libavif-dev?

Let's try these out:

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Still having library/codec issues with the base image on bullseye, going back to explicitly declaring the dependencies... we'll see.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Jut checked new 8.1/8.2 images and both can create files

$ dor drupalci/php-8.1-apache:production bash
$ php -r '$i=imagecreatetruecolor(10,20);imageavif($i,"123.avif");'
$ hexdum 123.avif
effulgentsia’s picture

effulgentsia commented

Great work on figuring out AVIF!!

Is the idea that all of the environments with PHP 8.1 and 8.2 are / will be on Bullseye? If so, that means they'll only be testing SQLite 3.34, which means the only environments capable of testing Drupal 10 will be testing SQLite 3.34. That's not ideal since Drupal 10 officially supports SQLite 3.26, and it's more likely that we'll accidentally introduce a regression for 3.26 if the minimum that we're testing is 3.34 than our prior situation of testing 3.27.

Would it be possible to retain an environment on which we have PHP 8.1 + SQLite (3.26 or only a bit higher) + AVIF? Or, if D10's intended AVIF support is optional, then perhaps we can have one environment with PHP 8.1 + SQLite 3.34 + AVIF, and another environment with PHP 8.1 + SQLite 3.27 + no AVIF? In that latter case, that would mean one environment for PHP 8.1 + Bullseye and another environment for PHP 8.1 + Buster. Or, if Buster is problematic for any reason, then CentOS 8 has SQLite 3.26 (which is why we still support that for D10).

andypost’s picture

andypost
he/him
Primary language Russian
commented

3.27.2 is available in "stretch-backports" https://packages.debian.org/search?keywords=libsqlite3-0

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

I'll work on updating both images using the min-supported SQLite version from backports.

effulgentsia’s picture

effulgentsia commented

both images

If it doesn't create too much burden on maintaining these environments, my preference would be for only PHP 8.1 to test SQLite 3.27, and to keep PHP 8.2 on SQLite 3.34. That way, we're testing a broader spectrum of SQLite versions. As an example, https://www.sqlite.org/changes.html says that in SQLite 3.32, the ESCAPE clause of a LIKE operator started overriding wildcard characters. So if we're only testing 3.27 and not anything higher than 3.31, then we could potentially introduce code that fails to account for subtle behavior differences like that.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

@effulgentsia

That makes good sense - test coverage for 2 sql lite versions with the same amount of containers we're maintaining anyway. Should be doable.

effulgentsia’s picture

effulgentsia commented

Even better would be if we could make the PHP 8.2 environment get the SQLite from Bookworm's backports (3.39), but I have no idea how feasible/reliable Bookworm backports are, so maybe that's better punted to a followup.

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

It doesn't look like packages.debian.org has a section for bookworm backports yet, https://packages.debian.org/bookworm-backports/libsqlite3-0 (404, doesn't recognize bookworm-backports as a suite yet)

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

8.1 should now be downgraded to sqlite 3.27, with 8.2 on 3.34

8.2 is also upgraded to rc4

andypost’s picture

andypost
he/him
Primary language Russian
commented

So the remaining tasks
- wait for GA release (in month) https://wiki.php.net/todo/php82
- add daily or on-commit testing for 10.1.x and 10.0.x branches

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Added: PHP 8.2-rc5 & MySQL 8
as a daily to both 10.0.x and 10.1.x

https://www.drupal.org/pift-ci-job/2509778
https://www.drupal.org/pift-ci-job/2509778

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Testbots are updated with php 8.2-rc6:

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

8.2 env has been updated to php 8.2-rc7

andypost’s picture

andypost
he/him
Primary language Russian
commented

9.5.x still reports 2,917 test failures

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Status: Needs work » Fixed

PHP 8.2 updated to release version 8.2.0:

Since this issue is for the environment I'm thinking we close this one and maybe open a follow up for the 9.5.x issues?

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.