#925764: SQL connection string for empty password is broken was challenging to fix because _drush_sql_get_credentials() is difficult to understand and hard to modify. The attached patch refactors it to;

  • assemble the command line parameters in an associative array
  • use full parameter names instead of abbreviations, e.g. "host" instead of "h"
  • only escape values that are passed into the function, and not the string constants that are defined in the function
  • better document oddities and nuances
  • remove old and redundant documentation
  • make it more maintainable and extensible
  • include #925764: SQL connection string for empty password is broken

I don't have Postgres or psql installed, so used the manual, which referred on to the man documentation, which I found via Google. The postgres changes have not been tested at all. Other changes have only been tested mildly and may have impact on other drush features that assume that the database name is an option, and not a parameter (--database="the_drupal_db"), or expect syntax in the string that is no longer present.

Comments

Bevan’s picture

StatusFileSize
new6.29 KB

site-install assumed that database name is preceded by a space character, and blatantly only supported MySQL by using the string constant 'mysql' for the command, as well as 'information_schema' for the database and -e for single-query-execution.

This patch removes any assumption about the syntax used in the credentials string, and also removes some of the explicit MySQL features in drush_core_pre_site_install(). There are probably other such assumptions in site-install related to the scheme and MySQL — I have not tested with other schemes, and was depending on the Postgres documentation linked in the description for some of these changes.

This patch also includes changes from the preceding patch, and in addition allows _drush_sql_get_credentials() to return credentials for a connection to the database server, but not the database. It uses MySQL's built in information_schema database for this. Postgres already supported this with the template1 database.

greg.1.anderson’s picture

Assigned: Unassigned » greg.1.anderson
Status: Needs review » Needs work

I like the code, but you've left me with a lot of testing to do. Could you at least expand you coverage re: "may have impact on other drush features..." etc.? If you do this much, I'll test the postgres branch.

greg.1.anderson’s picture

Posted #2 before I saw #1. Let me know when you've got pretty good coverage on the mysql testing.

Bevan’s picture

Understood. Is there a test suite for drush?

greg.1.anderson’s picture

Bevan’s picture

Status: Needs work » Needs review

I'll use drush with this patch applied and aim to keep this issue node up to date. It would be great if others could also test in this way.

Perhaps we could apply to HEAD to encourage further testing?

greg.1.anderson’s picture

I would prefer it if each code path that uses _drush_sql_get_credentials was tested at least once before committing...

Bevan’s picture

Hmmm. This is rather challenging, since any command that bootstraps to DRUSH_BOOTSTRAP_DRUPAL_DATABASE or higher, as well as several other commands (like site-install) uses _drush_sql_get_credentials(). Testing this manually would be somewhat tedious and time-consuming. Perhaps I am missing something?

greg.1.anderson’s picture

Okay, "each code path" was a bit strong, but please test it to a level where you're fairly confident that it's working.

I agree, drush needs unit tests...

(Edit: For further clarification, by each code path I meant test DRUSH_BOOTSTRAP_DRUPAL_DATABASE once, not every command that boots to that level, and so on for each caller of _drush_sql_get_credentials. But I'll settle just for a higher confidence level than in #1. Then I'll test the pgsql commands, and if they work, I'll commit it.)

Bevan’s picture

Oh I've done that already. I've used most of the drush pm-* commands and various others already. It just needs pgsql testing before it can be RTBC then?

greg.1.anderson’s picture

Yes, thanks; I'll test the pgsql code as soon as I get a chance.

moshe weitzman’s picture

Title: Improve maintainability of _drush_sql_get_credentials() » Refactor _drush_sql_get_credentials()
StatusFileSize
new6.92 KB

Attached fixes a couple problems on mysql

1. get_credentials() needed a space at beginning to match old behavior. Without it, several commands failed (sqlq, sql-connect, sql-dump) when run with --debug.
2. sql dump failed since --databases= is not supported by mysqldump. 'databases ' is supported so I implemented that in sql-dump. Not pretty, but OK.

sql-sync looks like it is running the right commands but the final import yields no tables in destination.

I noticed on postgres that -U is gone. Not sure if that’s deliberate. Needs testing on postgres.

Bevan’s picture

Moshe; Good catch.

Yes, -U is gone and that is intentional. -U is replaced by --user="bevan", much like like -ubevan" becomes --username="bevan" in MySQL.

How is PgSQL testing going? I asked Josh Waihi (fiasco) to give this a spin too. How thoroughly do we need to test this before it is RTBC for CVS HEAD?

I'm now using HEAD with Moshe's patch applied with MySQL. drush site-install, at least, is good.

greg.1.anderson’s picture

My drush work is stalled pending a personal commitment. Hope to get back into it soon.

Bevan’s picture

Assigned: greg.1.anderson » Bevan
Status: Needs review » Needs work
StatusFileSize
new6.09 KB

Re-rolled for head. This patch includes a change that was committed to head to add backticks around the db name and escape them with backslashes. Actually the original patch here was already adding backticks, but not backslashes.

Before re-rolling I found a bug where the port number is wrapped in quotes. It shouldn't be.

Bevan’s picture

Title: Refactor _drush_sql_get_credentials() » Refactor _drush_sql_get_credentials() to allow empty passwords
Assigned: Bevan » Unassigned
Status: Needs work » Reviewed & tested by the community
StatusFileSize
new6.87 KB

The backslashes introduced in #15 are not needed. I don't know why they are in HEAD (Bug?).

This patch removes them, reverting back to patch from comment #12, so site-install works again

In addition it fixes the issue raised in #15. This was not related to the port number specifically being wrapped in quotes, but arises with a bash command like $ `drush sql-connect` < file.sql, because the backtick operator on the command line doesn't play nicely with quotes in the output of the command which the backticks wrap around.

This patch fixes that by not wrapping $value from --$key='$value' in single quotes. The original issue (#925764: SQL connection string for empty password is broken) is still fixed because --password= is equivalent of --password=''. (However Shorthand -p has no equivalent of --password='').

We are starting to find only very edge-case arcane bugs here now. Both Moshe and I have been using drush with this patch for a while. It's now ready to be committed to HEAD in my humble opinion. This is important to introduce even more testing before it gets put into a stable release or branch.

greg.1.anderson’s picture

I agree this is rtbc; if there -are- any postgres bugs, I'll be forced to fix them once this goes in. :)

With only a handful of critical bugs in d7, I agree that we need to drive to completion on some of these issues so we can get drush-4 stable out.

greg.1.anderson’s picture

Status: Reviewed & tested by the community » Fixed

Tested on postgres and committed. Thanks.

Bevan’s picture

Yay! Thanks! (:

greg.1.anderson’s picture

You're welcome. There was one critical bug in this patch that I didn't catch before I committed it, but I already fixed that in #963850: Repair broken mysqldump.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.