[policy] Decide on MySQL/MariaDB/Percona Server version support status for Drupal 9

Added the release dates to the distributions.

Added the database release dates.

Looks RH is a blocker to get rid of 5.5

The newest versions of Debian 10 and Red Hat Enterprice Linux (RHEL) 8 are gong to be almost a full year old when Drupal 9 will be released. For me that is long enough to be used as a base distribution for Drupal 9. Ubuntu 18.04 LTS will be by then 2 years old. All 3 of those distribions support MySQL 8.0.19. MySQL 8.0 is by the time of the release of Drupal 9 already 2 years out. Which implies that it should be very stable.
For MariaDB the lowest supported version across the 3 mentioned distributions is MariaDB 10.1.29. Which will be by the release date of Drupal 9 almost 6 years old. For me that is just a bit too old. Let's set the minimum version for MariaDB 10.3. Both Debian 10 and RHEL 8 support MariaDB 10.3 and Ubuntu 18.04 supports MySQL 8.0.

According to @Dries: "Drupal is for ambitious digital experiences". Supporting only modern databases helps with that goal.

For me the minimum supported version for MySQL should be 8.0.19 and for MariaDB it should be 10.3.17.

Iirc mysql better to split into mysql (legacy) and mysql8 cos very different set of reserved words

I think #9 might be too aggressive, but I'm open to further discussion on that, especially if we have evidence of hosting companies almost universally making those versions available to customers.

Here's a less aggressive proposal:

Per http://www.oracle.com/us/support/library/lifetime-support-technology-069...

             Premier support end      Extended support end
MySQL 5.5    Dec 2015                  Dec 2018
MySQL 5.6    Feb 2018                  Feb 2021
MySQL 5.7    Oct 2020                  Oct 2023
MySQL 8.0    Apr 2023                  Apr 2026

Per https://mariadb.com/kb/en/mariadb-server/

               Supported until
MariaDB 5.5    Apr 2020
MariaDB 10.1   Oct 2020
MariaDB 10.2   May 2022
MariaDB 10.3   May 2023
MariaDB 10.4   June 2024

Drupal 7/8 sites don't have to upgrade to Drupal 9 until November 2021. Therefore, I think it's reasonable for Drupal 9.0 to drop support for MySQL versions whose extended support by Oracle will end before November 2021, as well as for MariaDB versions whose support will end before November 2021.

That would place Drupal 9.0's minimum at MySQL 5.7 and MariaDB 10.2. Which also happens to be the versions (specifically MySQL 5.7.8 and MariaDB 10.2.7) that support the json datatype, which I think would be nice to be able to start using during D9's lifetime.

Also, per https://www.percona.com/services/policies/percona-software-platform-life..., Percona for MySQL follows Oracle's EOL dates, so it too will EOL 5.6 in Feb 2021.

The issue summary says that Ubuntu 16.04 and 18.04 ship with MySQL 5.7 and MySQL 8. Is this true? I can't find confirmation of that anywhere. E.g., https://packages.ubuntu.com/bionic/mysql-server makes no mention of MySQL 8 anywhere.

However, it does look like Ubuntu 20.04 plans to ship with MySQL 8 per https://packages.ubuntu.com/focal/mysql-server.

@effulgentsia: According to https://pkgs.org/download/mysql-server is Ubuntu 16.04 and 18.04 supporting Mysql 5.7 and 8.

I think that shows packages that you can optionally install onto the corresponding distro, but not ones that already come pre-installed on that distro. Or am I misreading that?

For MySQL 8 you need to add and download MySQL APT repository. So MySQL 5.7 is the default versions that comes with ubuntu 18.04 and MySQL has an APT repository for MySQL 8.

@effulgentsia: You only look at the EOL as the only selection criteria. Even though that is a very impostant selection criteria, I think we must also have other criteria.

According to @Dries: "Drupal is for ambitious digital experiences" and in his last driesnote he spoke about that he expects that Drupal site will have to work with a 100 times more data then it has to do now.

For the last 19 years of Drupal we have used database normalization as the only way to store data in a database. The disadvantage of this is that we have now a large amount of tables in which the Drupal data is stored. This in combination with a large amount of data will result slow queries. For example take #2950869: Entity queries querying the latest revision very slow with lots of revisions. That issue is already critical. How critical will that issue become if we have Drupal sites with 100X more data. A lot of other issues that are now not jet critical will become critical when we sites with a 100x more data in it. Not everything can be solved by adding more caching and sticking to using database normalization as the only way to store data in a database.

One of the new features of modern relational databases is that they support JSON datatype. Drupal can use this datatype to store data in a more document like way (like MongoDB does). We do not have to go storing entity data in a full document way, we can also do only for the parts that help performance.

Lets use the modern capabilities of databases to help solve the performance issues that the Drupal projects has now and in the near future.

I agree with looking towards modern database features, like support for the json datatype. #12 recommends a minimum of MySQL 5.7 and MariaDB 10.2, which both support that. #9 recommends a minimum of MySQL 8.0 and MariaDB 10.3. What capabilities are missing from MySQL 5.7 and MariaDB 10.2 that are in MySQL 8.0 and MariaDB 10.3 that you're concerned will be needed?

Like we've done with PHP, I think we should consider a spin-off issue to raise the minimum MySQL versions to MySQL 5.7 and MariaDB 10.2 asap, since that seems non-controversial, while we continue discussion here about requiring higher versions than those.

I think the issue is #2985788: Add a separate MariaDB driver

Also not clear about testing environment

I opened #3109340: [policy] Decide whether to require json support for all database drivers for Drupal 10 to specifically discuss the json datatype question across all database drivers.

In comment #19 @effulgentsia asked the following question:

What capabilities are missing from MySQL 5.7 and MariaDB 10.2 that are in MySQL 8.0 and MariaDB 10.3 that you're concerned will be needed?

Regarding the JSON functionality we would like to use in Drupal 9:
If you do a search on the page https://dev.mysql.com/doc/refman/8.0/en/mysql-nutshell.html for the term "json" you get 123 results. A bunch of new json method are added to MySQL 8. No idea if we are going to need those methods in Drupal 9. But I would like to highlight one change:

As part of this work, the JSON_MERGE() function has been renamed JSON_MERGE_PRESERVE(). JSON_MERGE() continues to be recognized as an alias for JSON_MERGE_PRESERVE() in MySQL 8.0, but is now deprecated and is subject to removal in a future version of MySQL.

One of the things about supporting multiple major versions of databases that we've learned during Drupal 8 is that they like to add new reserved words. It would be great to get #2986452: Database reserved keywords need to be quoted as per the ANSI standard done.

I think that MySQL 8/MariaDB 10.3 may be too high for now based on what's actually available on hosting providers, and not necessary given the EOL dates. While the new features could be useful, it's not a strong reason to make the core requirements more disruptive than they need to be.

The one concern I have is that MariaDB 10.2 is EOL in 2022 rather than 2023. It'd be useful to have data on hosting providers and what databases they support now, to verify (or not) my conclusion that MySQL 8 is a bridge too far. (There might be concerns about using MySQL 8 or non-standard installs of the distro, and I know Acquia at least is using 5.6 and even the 5.7 upgrade requires a lot of investment.)

I added the MySQL EOL dates to the IS because they were missing.

The one concern I have is that MariaDB 10.2 is EOL in 2022 rather than 2023.

I share that concern. However, the latest version of SUSE Enterprise Linux (15 SP1) ships with MariaDB 10.2 as its default. The 15 series will be supported until 2028 or later, with new service packs coming out every year. I think their past behavior has been to sometimes update a database version in a service pack, so it's conceivable they'll do that in SP2 or SP3 or SP4, but SP2 is coming out in June 2020, and their draft release notes for it do not yet indicate a MariaDB update.

If we get indication that SP2 will update MariaDB to 10.3 or later, then I'd be +1 for making 10.3 the minimum for Drupal 9. However, without that happening, I don't think it's reasonable to set the minimum higher than what's installed on the latest version of a major distro.

We do run the risk, however, of MariaDB 10.2 becoming EOL and not security supported by any distro during Drupal 9's lifetime, and I don't know how we want to plan for that.

I brought this up with @daniel.bosen who raised an interesting point about Amazon's Aurora database that has engines that are compatible with both Postgres and MySQL. It looks like they don't support MySQL8 yet either - https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL...

Here's the list for Postgres - https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/AuroraPostg...

I've just committed #3109534: Raise the minimum MySQL version to 5.7.8 and MariaDB version to 10.2.7 in Drupal 9. We can continue discussing whether to raise requirements further here.

As someone who is currently using Amazon Aurora MySQL 5.6 in production, I would not like us to push to MySQL 8 just yet. I can see why we might want to go to 5.7 but that is still going to cause some underlying upgrades for a number of people who have been on 5.6 or below for years.

For me 5.7 seems like a good version to land on as well.

Updating the proposed resolution and marking NR.

The one concern I have is that MariaDB 10.2 is EOL in 2022 rather than 2023....If we get indication that SP2 will update MariaDB to 10.3 or later, then I'd be +1 for making 10.3 the minimum for Drupal 9.

That happened! Therefore: #3120124: Raise the minimum MariaDB version to 10.3(.7) in Drupal 9.

This policy is resolved I think since we have the fallback driver and everything. There are some related and followup issues that can continue separately, but we've resolved everything related to Drupal 9 core's requirements for MySQL, MariaDB, and Percona.

Thanks everyone!