Problem/Motivation

In the parent, it was noticed how query execution through prepared statements prevents async query execution.

Proposed resolution

Introduce a connection option to configure whether the Statement class should use mysqli prepared statements when executing queries, or not. By default, set it to use prepared statements (i.e. not to use the feature).

Adjust the statement class so that placeholders are replaced by escaped strings.

Query the database via mysqli::query().

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3571186

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

mondrake created an issue. See original summary.

mondrake’s picture

Status: Active » Needs review

This was surprisingly simpler than I would have expected. Some tests to fix but most are green.

mondrake’s picture

Title: [experimental] mysqli - skip prepared statement to allow async query execution » mysqli - skip prepared statement to allow async query execution
Issue summary: View changes
mcdruid’s picture

As a quick test, I ran sqlmap turned up to 11 (--risk 3 --level 5) on a simple parameterised query with the MR applied to the mysqli driver.

It didn't find a way to achieve SQLi.

This is not a guarantee that it's impossible, but it does suggest that it's not trivial to bypass the escaping.

smustgrave’s picture

Does this need explicit test coverage for the change though?

greggles’s picture

query execution through prepared statements prevents async query execution

Is this something that's theoretically possible, but not supported by some upstream code (either MySQL or PHP or...?)? Is there an issue for it, or could we file one?

catch’s picture

Is this something that's theoretically possible, but not supported by some upstream code (either MySQL or PHP or...?)? Is there an issue for it, or could we file one?

PDO doesn't support async query execution at all. That was the reason for adding the mysqli driver to core, because it does. However we didn't realise at the time that mysqli doesn't support async with prepared statements, which is this issue.

We could theoretically open an issue against PHP to add async support to the mysqli driver, however I think it's unlikely that will go anywhere while the 'PHP true async' RFC process is happening: https://wiki.php.net/rfc/true_async

If a version of that RFC gets accepted, then eventually, async query execution will be supported within PDO drivers too (essentially all i/o becomes non-blocking without each case needing to individually support async, if I understand the RFC, which I might not). Then they won't add async support to individual drivers because they won't need to.

If the RFC gets completely rejected, then it's possible a fallback would be to add/improve async in mysqli and/or PDO and we could try to push towards that then.

The amphp project built a mysql driver entirely in userland code (e.g. no PDO, no mysqli) because of this same limitation https://github.com/amphp/mysql - this would be hard or impossible for us to adapt.

greggles’s picture

Thanks for the explanation in #8!

mondrake’s picture

#6, well this is so low level that if skipping the use of the prepared statement would cause issues, the current test suite will throw sparks everywhere. However we are now missing explicit test for the case of USING the prepared statement, so I'll look into adding one.

andypost’s picture

catch’s picture

One thing - we'll need a connection pool to make use of this, because mysqli can only handle one async query per connection (e.g. you can do other stuff while you wait, as long as that other stuff isn't executing any database queries on the same connection).

Because async queries will have to go via a connection(s) that's specifically for issuing async queries, the 'main'/non-async connection can continue to use prepared statements by default. Might be worth switching the default back in the MR then we can set it explicitly when it's really needed.

https://github.com/php/php-src/issues/20640 looks like a great issue to link to from this, thanks for finding that @andypost.

ghost of drupal past’s picture

Do we know any external security experts who could verify this approach? It's not a lot of work hopefully it wouldn't take a lot of money. Does anyone for example whether i0n1c is still around and whether he would mind looking at this?

mondrake’s picture

Let's change the default then, now with the added kernel test we have test coverage for both queries with and without use of prepared statements.

smustgrave’s picture

Should this get a CR?

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Going to mark it but still have the open question of a CR, ignore if not needed.

mondrake’s picture

Issue tags: +Needs change record

#16 it certainly needs one, yes

smustgrave’s picture

Status: Reviewed & tested by the community » Needs review

Going to put back for that to be reviewed

mondrake’s picture

A question for @catch maybe: here we are setting the use (or not) of prepared statements at the entire connection level. Is that OK or shall we look into making it an option for the single query execution?

catch’s picture

@mondrake so we will need a separate 'async' connection anyway, this should mean we're able to leave the main database connection using prepared statements, and then the async connection(s) skip prepared statements, but they'd only be used for async queries. Given that, it theoretically shouldn't make any difference at all.

mondrake’s picture

Added draft CR.

#20 so let's leave this as is, and if a query-level settings will be needed in the future, let be it dealt in a separate issue. Thanks

mondrake’s picture

Issue tags: -Needs change record
mondrake’s picture

Issue summary: View changes