While thinking about #418302: Copy default.settings.php to settings.php during install IFF webserver owns files (FTP on shared hosting) and watching some people in IRC trying out the new Update manager functionality in D7, it occurs to me that we should just skip the whole authorize.php step entirely if we're running on a site with the insecure configuration where httpd is running as the same user who owns all the files. In this case, the authorize.php step doesn't actually help anything, and potentially hurts (slows down the process, might not work at all if the site has neither ssh nor ftp running locally, potentially exposes credentials over http, etc).

In terms of implementation details, I wonder if a LocalFileTransfer subclass makes the most sense, that just assumes everything is local and directly writable. Then, in update.manager.inc, when we're about to call system_run_authorized(), we have a little file ownership test. If the file owners of what we downloaded and where we want to put it don't match (yay!), we call system_run_authorized() like we do now. However, if they match (boo!), we do the following:

  1. include update.authorize.inc
  2. instantiate a LocalFileTransfer
  3. directly invoke update_authorize_run_update() with the LocalFileTransfer object and the arguments we were going to hand off to system_run_authorized()

We could even still use authorize.php for our batch page, both to reuse the code, and to still do the update via the lower bootstrap level. So, I *think* it'd require no changes to update.authorize.inc nor authorize.php at all. Just a ~10 line change to the submit handlers for the update and install pages. Well, and a whole LocalFileTransfer class. ;) But, that should be trivial.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dww’s picture

Assigned: Unassigned » dww
Status: Active » Needs review
FileSize
6.67 KB

This handles the update case. Reviews welcome. We should probably fix the install case the same way. That should be trivial, but it's 5am here, so I really need to sleep before I finish this off. ;)

dww’s picture

Btw, I'm not sure we want this new Updater::getCurrentDirectory() stuff I added here. Maybe the ownership test should just compare the extracted directory and the owner of the target install directory, although it's possible that the new target (e.g. sites/default/modules) doesn't exist. Maybe I should just check the owner on conf_path()? Honestly, it doesn't make that much sense to test the ownership on each project separately... everything we extract is going to be owned by the same UID, and I guess we could just compare the first one with the owner of conf_path() and be done with it, no?

This really "needs work", but I'm going to leave it needs review to at least see what the bot has to say.

dww’s picture

- Fixes the ownership check so we only do it once, and just compare the last thing we extracted with the owner of conf_path().
- Added a similar check for the new install path.

Assuming the bot's still happy, I don't have any reason to think this "needs work" at this point.

JacobSingh’s picture

Status: Needs review » Needs work

Sorry for the brevity. It's late, gotta get up early.

-J

+++ modules/update/update.manager.inc	25 Oct 2009 12:20:43 -0000
@@ -418,14 +418,30 @@ function update_manager_confirm_update_f
+    if (fileowner($project_real_location) == fileowner(conf_path())) {
+      module_load_include('inc', 'update', 'update.authorize');
+      $filetransfer = new FileTransferLocal(DRUPAL_ROOT);
+      update_authorize_run_update($filetransfer, $updates);
+    }
+    // Otherwise, go through the regular workflow to prompt for FTP/SSH
+    // credentials and invoke update_authorize_run_update() indirectly with
+    // whatever FileTransfer object authorize.php creates for us.
+    else {
+      system_run_authorized('update_authorize_run_update', drupal_get_path('module', 'update') . '/update.authorize.inc', array($updates));
+    }

Shouldn't we check for is_writable here? It could be owned by someone else but 777.

+++ modules/update/update.manager.inc	25 Oct 2009 12:20:43 -0000
@@ -565,13 +581,29 @@ function update_manager_install_form_sub
+  if (fileowner($project_real_location) == fileowner(conf_path())) {
+    module_load_include('inc', 'update', 'update.authorize');
+    $filetransfer = new FileTransferLocal(DRUPAL_ROOT);
+    call_user_func_array('update_authorize_run_install', array_merge(array($filetransfer), $arguments));
+  }

This is sorta a cut & paste job, isn't it? Any better way to do this?

+++ includes/filetransfer/local.inc	25 Oct 2009 10:28:45 -0000
@@ -0,0 +1,86 @@
+  protected function removeDirectoryJailed($directory) {

Should this use a RecursiveDirectoryIterator? (it is only like this in the FTP Wrapper class because RecursiveDirectoryIterators don't work properly there).

I'm on crack. Are you, too?

dww’s picture

Status: Needs work » Needs review

A) Shouldn't we check for is_writable here? It could be owned by someone else but 777.

No. There are all kinds of UX problems that come from having the files owned by the httpd user if that's not you. It means that the user can't actually modify these files at all any more, *other* than in the limited way supported by this UI. It's the same problem as #225880: non-writablilty of settings.php when created by webserver

B) This is sorta a cut & paste job, isn't it? Any better way to do this?

Do you have a suggestion? The actual thing we invoke is different in the two cases. The only cut & paste is the verbose code comments. Otherwise, it's just 4 shared lines. Moving this into a shared helper function would create more code, I think, since the way you invoke the operation is different depending on if it's install vs. update.

C) Should this use a RecursiveDirectoryIterator?

We don't do so anywhere else in core besides filetransfer.inc. Not sure it's really an improvement in readability to do this:

foreach (new RecursiveIteratorIterator(new RecursiveDirectoryIterator($source), RecursiveIteratorIterator::SELF_FIRST) as $filename => $file) {
}

;)

JacobSingh’s picture

Status: Needs review » Needs work

Okay, A and B make sense to me (although I'd put a @see in the code comments for maintainability).

RecursiveDirectoryIterator is there exactly for this purpose, I think we should use it. It is easier to read (IMO) but also less code and the way it is "supposed to be done". Using the php4 method doesn't really buy us anything.

If we change this, I'd say RTBC.

Crell’s picture

The advantage of RecursiveDirectoryIterator and friends, as I see it, is to centralize the ugly parts of the code into the setup for the loop so that the actual logic of the loop is cleaner. It also makes it easier to make the body of the loop interchangeable using various OO techniques such as the visitor pattern. If your loop is simple, then it's frequently unnecessary. I use scandir() all the time, still.

RecursiveDirectoryIterator here wouldn't give us a great deal, I think, aside from consistency with a foreach/iterator approach (DBTNG is all about the iterators) and some utility routines such as $dir->isDot() to replace if ($entry == '.' || $entry == '..'). I'd have to see the implementation to say for sure which I would find "cleaner", which is admittedly subjective. I'd probably lean toward the iterator approach, but that's me. (Now, if we were using splFile and friends as well I'd absolutely say we should be using the full iterator suite. SPL has extensive file handling niceness that we don't use as often as we perhaps should. sdboyer could talk to SplFile more than I can.)

As long as I'm here, this makes no sense to me at all:

+ static function factory($jail, $settings) {
+ return new FileTransferLocal($jail);
+ }

Factories are supposed to be centralized points of contact so that one system can request "some implementation" of another system. Why have a trivial factory per implementation? Besides, static methods suck for inheritance. :-)

JacobSingh’s picture

Yeah, of course it is subjective, both work. One is:

* Shorter in bytes and commands
* Probably faster (not that it matters much).
* Meant for doing exactly this

The other is:

* Old.

re: the silly factory.

My original plan was different than this, but this is sorta a casualty of participating slightly outside the Drupal eco system. I agree, we could keep our factory in Drupal land if wanted to.

However, we need a factory of some sort because an anonymous array of settings is not a good thing to pass to a constructor (@see the new hook_theme_* for the opposite of good). Currently, the factories deal with setting defaults and instantiating themselves. In some cases though (like FileTransferFTP::factory) it actually is more like a "normal" factory in that it is figuring out the subclass.

In this case, it's just following the expected interface, and because this is a "pseudo" FileTransfer anyway (99% of other ones would require some settings), I'm fine with it.

The alternatives:
* Pass $settings to a constructor (bad)
* Have the factory in Drupal - requires that each implementation register a separate function and handle through that. That function would have logic which really belongs in the class most likely. This is redundant and harder to maintain, the class knows what its defaults, etc should be and how to find child classes.

Anyway, this issue is really not about changing the whole design pattern, so let's try to stay off that one until later if it needs discussion.

Thanks,
J

dww’s picture

Okay, I'm willing to re-roll this using the RecursiveDirectoryIterator so we can compare. I'll post something in a little while, I'm just in the middle of something else right now...

dww’s picture

Now with RecursiveDirectoryIterator, both the patch and the interdiff vs. patch #3 for comparison. Didn't touch update.manager.inc, especially since it's going to conflict with #610290: system_run_authorized() shouldn't always drupal_goto() so it can be used in submit handlers anyway...

JacobSingh’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me!

Although one small point, if $file->isDir() returns false, I'm pretty sure that means $file->isFile() is going to return true.

-J

dww’s picture

Yeah, I was just being paranoid. Seems like it's better to be safe than sorry in this code. ;)

For comparison, I worked up a version that uses a shared helper for deciding how to invoke the operation (to address #4.B). I'm attaching the patch and the interdiff. I think it's a net loss doing it this way in terms of readability. It's going to get even weirder once #610290: system_run_authorized() shouldn't always drupal_goto() so it can be used in submit handlers lands, since we'd need to also pass in &$form_state to this helper. Ugh. So, I'd rather #10 was committed... But, since I did the work just to see what it'd be like, I'm attaching here so no one can accuse me of ignoring Jacob's concerns. ;)

JacobSingh’s picture

Nice of you to post it, but yeah, helper function not desired in this case, and any refactoring not worth it. I was only concerned, not saying we shouldn't. Duped comment is no biggie.

dww’s picture

Right. So, note to the committer:

- Be sure to use patch #10, not patch #12.

- After the patch, before the commit:

cvs add includes/filetransfer/local.inc
webchick’s picture

Status: Reviewed & tested by the community » Fixed

It struck me as odd that none of the functions in local.inc are documented, but dww pointed out that these are in the base class, and the common pattern is not to document overridden methods in derived classes, which is true.

We also discussed whether or not this warrants a warning, either on this screen or in the status messages list that basically says "Your host is an idiot and should not have stuff set up this way." But in the end, figured that if someone is in such a situation, there's really nothing that they can do it about it, and probably neither can their host or they would've set it up properly in the first place. :P

It's concerning me to change this much code without any changes in the tests. I've committed this to HEAD for now to keep things moving, but is there any way of automated testing this kind of stuff, or not really since it's so host-specific?

webchick’s picture

Aw, crap. I committed 12 instead of 10. :( I had it right the first time and then went and committed the button case fix one since it was easier. Anyway, fixed it. I think.

Can you guys check to make sure this got through properly?

dww’s picture

Yup, the code in HEAD now looks right. Sorry for the mix-up, and thanks for getting this in!

Re tests: Partly we just need people to step up and help with #605276: Add tests for the update manager, partly, I'm trying to prioritize issues that need to happen before 11/15 unlike tests which can happen later, and partly yeah, it's a little tricky to test this specific issue since the tests are always going to be running as a single UID, and if we don't have root (THANK GOD!), we can't actually chown files and simulate the different cases we're testing for in this code. That said, we could certainly add unit tests for FileTransferLocal, and all of FileTransfer for that matter. There's pretty sparse test coverage of FileTransfer from the original issue, and that's one of the areas of this whole system I've spent the least time looking at closely and cleaning up. *That said*, unit or functional tests for the FTP and SSH subclasses are going to be hard/impossible to write if they're trying to connect to a real ftpd or sshd. :/

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

David_Rothstein’s picture

Status: Closed (fixed) » Needs work

Questions:

  1. Does this mean that "administer software updates" is now one of the most dangerous permissions on a Drupal site? As far as I can tell, before this patch, that permission didn't let you do anything too dangerous - unless you also happened to know the server login credentials, in which case it doesn't matter :) Whereas with this patch, it seems like on most popular shared hosting providers (Bluehost, Dreamhost, whatever) this permission is now pretty much equivalent to "execute arbitrary PHP code".

    If so, we need to document the permission description much better. (It would be a separate issue, but before filing one I want to make sure I understand the situation correctly.)

  2. Is checking the fileowner() really good enough? It seems like in most cases, it would be, but are there any server configurations in which the webserver-created file could wind up with more permissive group ownership that the sites/default directory (or perhaps even wind up world-writable), even though they have the same owner? I remember now that this is the kind of thing I was concerned about when thinking about #418302: Copy default.settings.php to settings.php during install IFF webserver owns files (FTP on shared hosting) and the other issues that led up to that.

    If that is a concern, we need to change something here. And if not, it would be really great to have the code comment here expanded significantly to explain exactly why this check is sufficient and safe (and ideally even have that check broken out into a separate function that returns a boolean). So either way, I'm tentatively putting this back to needs work.

dww’s picture

Status: Needs work » Closed (fixed)

Re: #19.1: Yes. That was the original intention of naming that permission "administer software updates", which is what you came up with at the original issue: #67234-46: Update script access rights

For the permission name, I went with "administer software updates" -- it's simple, clear, and could easily be extended to the Plugin Manager if necessary.

;)

Re: #19.2: I don't understand your concern. If the webserver has no umask at all (lord help you), I fail to see how prompting for your ftp/ssh credentials makes life any better. So long as the files are owned by the same person, a FileTransferLocal is just as powerful as a FileTransferFTP, and arguably safer, since we didn't prompt you for your login credentials (likely over http, not https). It's at least chmod'ing some install directories back to 755 for you.

If you want to add a bunch of code to try to detect world-writable files and fix them during install/update, that's a new feature/task, and it needs to happen in the FileTransfer classes themselves, regardless of which one we're using. Although, I'd potentially argue it's bloat. If all the files in your account are already world-writable by default, Drupal's not really going to be able to save you. But, if you're interested in this case, feel free to submit an issue and advocate for it (ideally with data about actual server configurations in the wild where this is happening). I just don't think it has anything to do with this check in here, but I might be wrong. Please clarify if you think I'm missing something.

I think this should be closed. If you think we need follow-up issues for anything, please submit them and add links here as "FYI related". Or, if I'm totally misunderstanding your point (quite possible given the fever I'm currently running), feel free to reopen and explain.

Cheers,
-Derek

David_Rothstein’s picture

Very belated reply :)

For 1, yes, my concern was not with the permission being given a "dangerous" name (that certainly makes sense), but rather that it breaks other assumptions Drupal 7 has about when and how arbitrary PHP code execution is allowed via the UI - e.g., by someone who stumbles across a computer where an administrator has left themselves logged in. We may wind up deciding it's just a documentation issue, but we'll see. I've created an issue for that here: #932110: On some servers, the Update Manager allows administrators to directly execute arbitrary code even without the PHP module

For 2, having thought about it more, I agree with you that there are probably no (realistically secure) server setups under which this could be a problem.