Problem/Motivation
In the parent, it was noticed how query execution through prepared statements prevents async query execution.
Proposed resolution
Introduce a connection option to configure whether the Statement class should use mysqli prepared statements when executing queries, or not. By default, set it to use prepared statements (i.e. not to use the feature).
Adjust the statement class so that placeholders are replaced by escaped strings.
Query the database via mysqli::query().
Remaining tasks
User interface changes
Introduced terminology
API changes
Data model changes
Release notes snippet
Comments
Comment #3
mondrakeThis was surprisingly simpler than I would have expected. Some tests to fix but most are green.
Comment #4
mondrakeComment #5
mcdruid commentedAs a quick test, I ran sqlmap turned up to 11 (
--risk 3 --level 5) on a simple parameterised query with the MR applied to the mysqli driver.It didn't find a way to achieve SQLi.
This is not a guarantee that it's impossible, but it does suggest that it's not trivial to bypass the escaping.
Comment #6
smustgrave commentedDoes this need explicit test coverage for the change though?
Comment #7
gregglesIs this something that's theoretically possible, but not supported by some upstream code (either MySQL or PHP or...?)? Is there an issue for it, or could we file one?
Comment #8
catchWe could theoretically open an issue against PHP to add async support to the mysqli driver, however I think it's unlikely that will go anywhere while the 'PHP true async' RFC process is happening: https://wiki.php.net/rfc/true_async
If a version of that RFC gets accepted, then eventually, async query execution will be supported within PDO drivers too (essentially all i/o becomes non-blocking without each case needing to individually support async, if I understand the RFC, which I might not). Then they won't add async support to individual drivers because they won't need to.
If the RFC gets completely rejected, then it's possible a fallback would be to add/improve async in mysqli and/or PDO and we could try to push towards that then.
The amphp project built a mysql driver entirely in userland code (e.g. no PDO, no mysqli) because of this same limitation https://github.com/amphp/mysql - this would be hard or impossible for us to adapt.
Comment #9
gregglesThanks for the explanation in #8!
Comment #10
mondrake#6, well this is so low level that if skipping the use of the prepared statement would cause issues, the current test suite will throw sparks everywhere. However we are now missing explicit test for the case of USING the prepared statement, so I'll look into adding one.
Comment #11
andypostQuick search showing
- https://github.com/php/php-src/issues/19777
- https://github.com/php/php-src/issues/20640
Comment #12
catchOne thing - we'll need a connection pool to make use of this, because mysqli can only handle one async query per connection (e.g. you can do other stuff while you wait, as long as that other stuff isn't executing any database queries on the same connection).
Because async queries will have to go via a connection(s) that's specifically for issuing async queries, the 'main'/non-async connection can continue to use prepared statements by default. Might be worth switching the default back in the MR then we can set it explicitly when it's really needed.
https://github.com/php/php-src/issues/20640 looks like a great issue to link to from this, thanks for finding that @andypost.
Comment #13
ghost of drupal pastDo we know any external security experts who could verify this approach? It's not a lot of work hopefully it wouldn't take a lot of money. Does anyone for example whether i0n1c is still around and whether he would mind looking at this?
Comment #14
mondrakeLet's change the default then, now with the added kernel test we have test coverage for both queries with and without use of prepared statements.
Comment #15
smustgrave commentedShould this get a CR?
Comment #16
smustgrave commentedGoing to mark it but still have the open question of a CR, ignore if not needed.
Comment #17
mondrake#16 it certainly needs one, yes
Comment #18
smustgrave commentedGoing to put back for that to be reviewed
Comment #19
mondrakeA question for @catch maybe: here we are setting the use (or not) of prepared statements at the entire connection level. Is that OK or shall we look into making it an option for the single query execution?
Comment #20
catch@mondrake so we will need a separate 'async' connection anyway, this should mean we're able to leave the main database connection using prepared statements, and then the async connection(s) skip prepared statements, but they'd only be used for async queries. Given that, it theoretically shouldn't make any difference at all.
Comment #21
mondrakeAdded draft CR.
#20 so let's leave this as is, and if a query-level settings will be needed in the future, let be it dealt in a separate issue. Thanks
Comment #22
mondrakeComment #23
mondrake