Problem/Motivation

New major release is out https://www.postgresql.org/about/news/postgresql-14-released-2318/

There's official docker images https://hub.docker.com/_/postgres

Proposed resolution

Add PHP8 + pgsql 14.0 container to allow core to work on compatibility issues for D10

I bet we need both versions in CI because
- 14.0 used to add support for openssl 3.0 so 13.x is the last release with 1.x requirement
- another big change is libicu upgrade which affects multilingual sites a lot

Remaining tasks

tbc

User interface changes

tbd

API changes

no

Data model changes

no

Issue fork drupalci_environments-3240346

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

andypost created an issue. See original summary.

andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#3118249: Fix PostgreSQL 12 container for building after PostgreSQL 13 release, +#3104007: Postgresql 12 DrupalCI environment needed for core testing
andypost’s picture

andypost
he/him
Primary language Russian
commented
Title: Add Postgresql 14.0 to CI » Add Postgresql 13/14.0 to CI

andypost opened merge request !3

andypost’s picture

andypost
he/him
Primary language Russian
commented
Status: Active » Needs review

Pushed commits for 13.4 and 14.0, configs of new dockerfiles are changed accordingly

andypost’s picture

andypost
he/him
Primary language Russian
commented
Issue summary: View changes
daffie’s picture

daffie commented

@andypost: Is there any special reason why you selected PostgreSQL 13.4 instead of an earlier 13.x version? Usualy we go for something like 13.0 or 13.1. I am not saying that it is the wrong choose, just curious about why.

If we go for PostgreSQL 13.4, then the MR is RTBC for me.

andypost’s picture

andypost
he/him
Primary language Russian
commented

The reason is security, every 13.x got some CVE except 13.3 yet https://www.cvedetails.com/vulnerability-list/vendor_id-336/product_id-5...

daffie’s picture

daffie commented
Status: Needs review » Reviewed & tested by the community

As we are not testing the minimum required version, lets go with PostgreSQL v13.4.

The moment that the database versions we use on the testbot become a problem the guys and/or girls from the DA will lets us know.

The MR looks good to me. I have tested that the docker images work. For me it is RTBC.

@andypost: Thank you for the work on this issue.

andypost’s picture

andypost
he/him
Primary language Russian
commented

There's security releases so I did add upgrade commit to not add insecure versions

https://www.postgresql.org/about/news/postgresql-141-135-129-1114-1019-a...

mradcliffe made their first commit to this issue’s fork.

hestenet made their first commit to this issue’s fork.

andypost’s picture

andypost
he/him
Primary language Russian
commented
Status: Reviewed & tested by the community » Fixed

I think it fixed now! Thank you a lot

andypost’s picture

andypost
he/him
Primary language Russian
commented

@hestenet I think you can delete issue fork and branch, as I have no permissions

hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented
Status: Fixed » Needs work

Trying to figure out why these environments consistently result in CI Error:

9.5:

10.0:

10:36:56 Recording test results
10:36:56 ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?

andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#3108620: Allow test runner to use local images

The issue in CI codebase, looks related to #3108620: Allow test runner to use local images as the patch adds check for the method $response instanceof CallbackStream

20:20:38 chmod -R 777 /var/www/html
20:20:40 PHP Fatal error:  Uncaught Error: Call to undefined method GuzzleHttp\Psr7\Response::onFrame() in /opt/drupalci/testrunner/src/DrupalCI/Build/Environment/Environment.php:384
20:20:40 Stack trace:
20:20:40 #0 /opt/drupalci/testrunner/src/DrupalCI/Build/Environment/Environment.php(322): DrupalCI\Build\Environment\Environment->pull('drupalci/pgsql-...')
20:20:40 #1 /opt/drupalci/testrunner/src/DrupalCI/Build/Environment/Environment.php(258): DrupalCI\Build\Environment\Environment->startContainer(Array)
20:20:40 #2 /opt/drupalci/testrunner/src/DrupalCI/Plugin/BuildTask/BuildStep/StartContainers/RunContainers.php(72): DrupalCI\Build\Environment\Environment->startServiceContainerDaemons(Array)
20:20:40 #3 /opt/drupalci/testrunner/src/DrupalCI/Plugin/BuildTaskBase.php(166): DrupalCI\Plugin\BuildTask\BuildStep\StartContainers\RunContainers->run()
20:20:40 #4 /opt/drupalci/testrunner/src/DrupalCI/Build/Build.php(503): DrupalCI\Plugin\BuildTaskBase->start()
20:20:40 #5 /opt/drupalci/testrunner/src/DrupalCI/Build/Build.php(505): DrupalCI\Build\Build->processTask(Array)
20:20:40 #6 /opt/drupalci/testrunner/sr in /opt/drupalci/testrunner/src/DrupalCI/Build/Environment/Environment.php on line 384
20:20:40 Build step 'Execute shell' marked build as failure
hestenet’s picture

hestenet
He/Him
Location Portland, OR 🇺🇸
commented

Got #3108620: Allow test runner to use local images committed, but still stumped as to what's happening here.

andypost’s picture

andypost
he/him
Primary language Russian
commented
Title: Add Postgresql 13/14.0 to CI » Add Postgresql 13/14/15.0 to CI

As https://www.postgresql.org/about/news/postgresql-15-released-2526/ it needs to explore compatibility

andypost’s picture

andypost
he/him
Primary language Russian
commented
Title: Add Postgresql 13/14/15.0 to CI » Add Postgresql 15.4/16.0 to CI

Nor sure we need to update existing images, probably better to rename 14.1 to 14.9

andypost opened merge request !22

andypost’s picture

andypost
he/him
Primary language Russian
commented

Using version 15.4 as latest and have to update config https://www.percona.com/blog/postgresql-15-stats-collector-gone-whats-new/

postgres@a5a67edc90c0:~$ bash /opt/startup.sh 
PGSQL VERSION: 15.4
rebuilding PostgreSQL database cluster
Error: specified cluster does not exist
Creating new PostgreSQL cluster 15/main ...
/usr/lib/postgresql/15/bin/initdb -D /var/lib/postgresql/15.4/main --auth-local peer --auth-host scram-sha-256 --no-instructions
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/15.4/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
Ver Cluster Port Status Owner    Data directory                Log file
15  main    5432 online postgres /var/lib/postgresql/15.4/main /var/log/postgresql/postgresql-15-main.log
CREATE ROLE
CREATE DATABASE
CREATE EXTENSION
CREATE SCHEMA
2023-09-16 21:36:17.353 GMT [91] LOG:  skipping missing configuration file "/var/lib/postgresql/15/main_data/postgresql.auto.conf"
2023-09-16 21:36:17.353 GMT [91] LOG:  unrecognized configuration parameter "stats_temp_directory" in file "/etc/postgresql/15/main/postgresql.conf" line 621
2023-09-16 21:36:17.353 GMT [91] FATAL:  configuration file "/etc/postgresql/15/main/postgresql.conf" contains errors
pgsql died at Sat Sep 16 09:36:17 PM UTC 2023

andypost’s picture

andypost
he/him
Primary language Russian
commented
Status: Needs work » Needs review

Dev images pushed and here's testing https://git.drupalcode.org/project/drupal/-/merge_requests/4804

andypost’s picture

andypost
he/him
Primary language Russian
commented

Config is picked from

- https://github.com/postgres/postgres/blob/REL_15_4/src/backend/utils/mis...
- https://github.com/postgres/postgres/blob/REL_16_0/src/backend/utils/mis...

andypost’s picture

andypost
he/him
Primary language Russian
commented
Status: Needs review » Fixed
Related issues: +#3387732: Add postgresql 15/16 to testing matrix

Added to 10.2-dev core #3387732: Add postgresql 15/16 to testing matrix

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

andypost’s picture

andypost
he/him
Primary language Russian
commented

Looks it needs to upgrade this images for new security releases

andypost’s picture

andypost
he/him
Primary language Russian
commented

pushed 15.6/16.2 to dev

https://git.drupalcode.org/project/drupalci_environments/-/commit/81120a...

andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#3421217: Upgrade Postgresql 15.6/16/2