Problem/Motivation

Using FTP or SSH to install modules in Drupal 8 is broken. It is trying to change immutable config and the form throws an EnforcedResponseException that is not handled as expected by authorize.php.

This is a critical bug because installing modules is one of the first tasks new users want to do. Whilst experience Drupal user or web developers are unlikely to use this method to install modules other types of users are and it is completely broken.

Proposed resolution

Stop saving transfer details in system.authorize config. These settings should not be deployed between environments and these details will be remembered by a user's browser - storing them in Drupal is wrong really.

Fix authorize.php to handed EnforcedResponseExceptions.

Add tests so we don't break this ever again.

Remaining tasks

User interface changes

None

API changes

None

Data model changes

system.authorize config is meaningless. Filed #2715145: Remove system.authorize config to remove it in 8.2.x.

Original report

The original bug this issue was meant to solve has moved to #2717951: Update manager sometimes can't install modules using FTP due to file ownership issues...

I'm testing the latest HEAD on a Ubuntu server, and I've tried installing Views by pasting the url into the update manager.
After I input my ftp info (which btw, it doesn't ask for in my Windows install?), it downloads the file, but fails in the end with the message:


# Failed: Error installing / updating
# Failed: File Transfer failed, reason: Cannot create directory /home/bojan/public_html/drupal/sites/all/modules/views

The modules/ dir is chmoded 777. The owner is my user, "bojan".

What could be the problem? Might be good to have a note in the documentation...

NOTE: A very similar problem can be caused by a host configuration error on your file permissions and/or ownership. However, in rare circumstances, it can also happen when all the permissions and ownership appear to be correct. The root cause of this bug has not been found.

If you are having this problem, please read through this issue and fix any permission/ownership issues you find. If that doesn't fix the problem, then report any troubleshooting information you have. Don't bother to report that fixing permission problems solved the issue for you.

Files: 

Comments

PHermansson’s picture

I have a similar problem with Alpha 1.

"Failed: Error installing / updating
Failed: File Transfer failed, reason: Cannot create directory /mnt/filer/www/drupal7/drupal-7.0-alpha6/sites/all/modules"

The modules directory already exists, so it's natural that the creation fails.

tungsd’s picture

Title: Update manager can't install module -> permissions on a Ubuntu server » Update manager can't install module -> permissions on a Fedora 13 server

I have a similar problem with Alpha 1.

tungsd’s picture

I have a similar problem with Alpha 1.

Scott M. Sanders’s picture

Title: Update manager can't install module -> permissions on a Fedora 13 server » Update manager can't install module -> permissions on a Linux server
Version: 7.x-dev » 7.0
Component: update system » update.module
Issue tags: +install

I get this too, even after "chmod 777 modules" on Ubuntu 10. I am guessing it is at least not distro-specific, probably not OS-specific either.

mikyatope’s picture

Same here, my modules folder is 777 too and I get the error

kleinhev’s picture

Priority: Normal » Critical
Issue tags: +available updates error

This thread is nearly one year old. Why has noone addressed this issue which affects the very core of DRUPAL? My server runs SuseLinux 11.1 / PHP Version 5.2.9 / MySQL 5.0.67 and I have the same problem. A workaround is to delete the subdirectory and install the module from scratch. Deleting the subdirectory, fortunately, does not erase the module settings.

Scott M. Sanders’s picture

It's not critical -- just annoying.

You can just install modules manually like every version of Drupal before 7.

bfroehle’s picture

Priority: Critical » Major

I think we'd all be happy to fix the problem if we knew what caused it. Just saying that it doesn't work isn't necessarily enough to go off of. Drupal is a Do-ocracy, for better or for worse, and this usability issue has unfortunately slipped off the radar.

I've attempted to reproduce this a few times but have never been able to. The update-via-FTP feature works fine in many configurations.

As Scott said, the problem isn't critical as you can still upgrade modules by hand. Sorry. :(

GiorgosK’s picture

Category: support » bug

The problem I believe is with some modules

I succesfully updated modules such as views/admin_menu/wysiwyg etc

Error message
Update failed! See the log below for more information. Your site is still in maintenance mode.
backup_migrate

    Installed backup_migrate successfully
context
    Installed context successfully
filefield_paths
    Installed filefield_paths successfully
gmap
    Installed gmap successfully
google_analytics
    Error installing / updating
    File Transfer failed, reason: Cannot create directory /public_html/sites/all/modules/google_analytics
pathauto
    Installed pathauto successfully
simplenews
    Installed simplenews successfully
wysiwyg
    Installed wysiwyg successfully

why did it need to CREATE the google_analytics directory it was being update so it was already created ...

don't know how the update manager works but could it have something to do with the google analytics module itself ? as far as I can tell all the module directories have the same permissions so its not my server's problem

GiorgosK’s picture

Version: 7.0 » 7.2

forgot to change version

snuffles’s picture

You *MAY* need to apply this patch http://drupal.org/files/issues/filetransfer_chroot_other_fixes-528326-44... as described on this page http://drupal.org/node/528326

You *MAY* also need to make sure that you do *NOT* have DefaultRoot defined or set in your /etc/proftpd.conf file. (or whatever FTP server you use)

This will allow you to install modules and themes and directly ftp to your Server, but it still does not allow you to update themes or modules, this is still an issue.

Cheers
Douglas

catch’s picture

Version: 7.2 » 8.x-dev
Issue tags: +needs backport to D7

Looks similar to #1215090: Update manager / Module installation falling back to FTP/SSH mode, similar enough that I'm going to merge the issues, despite the unnecessarily snide tone on the other issue, it does have some steps to reproduce.

Copying here:

The actual problem is that for some beyond retarded reason in modules/update/update.manager.inc on line 706 (in my present setup, drupal 7.2) the following check is performed:

if (fileowner($project_real_location) == fileowner(conf_path()))

with the comments explaining that conf_path() resolves to e.g. sites/default (why the "e.g." is beyond me - could it be something else?!? or is "i.e." what the authors meant to say?).

The $project_real_location (how real could it be, it is a temporary location underneath temporary:/") is where the file is downloaded and extracted.

Depending on the server setup, this can be randomly complicated by a number of factors. Fact is, even if the web server user has all the rights to write all over the place, it wouldn't make any difference. Enjoy wasting your time trying to make it do so.

And why select a location that is generally created manually prior to installation to check its ownership against files actually created by the web server user. There must be a better way to deal with this.

Eventhough Wordpress is using a similar mechanism (someone is bound to have borrowed it in/from Drupal), they do have a configuration option that overrides this. This is in particular a problem when users can manipulate their setup, but cannot change ownership of files, and especially critical when performing backup, migration, or other maintenance tasks.

Even if there is no easy workaround for this, there has to be at least a hint clarifying exactly what the problem is and how to go about this in case it occurs.

Also note that the web server user may vary. On our setup it was assigned to games (id=5) for whatever reason, so intuitively changing the ownership to www-data (id=33) did not make any difference.

If there is absolutely no way around this, then include an empty file that serves this sole purpose - controlling the write access, but for goodness sake, do not rely on some odd side effects.

catch’s picture

Title: Update manager can't install module -> permissions on a Linux server » Update manager can't install module due to file ownership issues

Also re-titling, it looks like it is explicitly not permissions on the file that's causing this.

organizedhome’s picture

Subscribing

Bojhan.core’s picture

Status: Active » Needs work
catch’s picture

Status: Needs work » Active

Moving back to active, there's no patch here.

catch’s picture

Status: Active » Postponed (maintainer needs more info)
Issue tags: -install, -available updates error

We still need solid steps to reproduce here before anyone can work on it, marking needs more info.

Danny Crossley’s picture

FileSize
16.74 KB

I'm getting this error message no matter what module I try to install. This is all the information it gives me.

Error message
Installation failed! See the log below for more information.
danland

Error installing / updating
File Transfer failed, reason: Cannot create directory /var/www/drupal-7.9/sites/all/themes/danland

Next steps

The the themes directory has 777 permissions.

PatchRanger’s picture

I had the same issue on my machine with Ubuntu 11.10 and self-configured LAMP-server.
bojanz, PHermansson, Scott M. Sanders, Danny Crossley, it looks like you also use Linux with LAMP, don't you?
The problem in my case was in incorrect FTP-connection to localhost FTP-server.
Probaly your connection is anonymous - that's why we had problem with write-privilegies.

Solution:
Through Filezilla (or your favourite FTP-client) try and find the connection that works.
Host : localhost (or name of your virtual host)
User, password - one of your UNIX-users
Port : 21

Use found connection for Update manager - and everything is working like a charm!
No more such issue.

Glad if I could help.

stradivari’s picture

Had the same ugly issue. Definitely permissions related.
*snuffles* suggestion to remove DefaultRoot in /etc/proftpd.conf worked.
Just commented the line and module installed successfully

valdir.marcos’s picture

I am using Drupal 7.10 on two environments: (1) Debian Linux test server using root for everything and (2) hosted sites on third-party companies all using Linux limited users.
Drupal 7 update manager works correctly on situation 1.
On situation 2, I got the message below:
•Error installing / updating
•File Transfer failed, reason: Unable to change to directory /sites/all/modules/pathauto.

Are limited users provided by host companies a problem to this update method?
MySQL user and Hosting/SSH/FTP user are different names/passwords. Does it matter?
www/sites/all/modules/ is drwxr-xr-x. Should it be 777?

valdir.marcos’s picture

Is it possible these scripts provide username, password, source path, source file, source path permission, source file permission, destiny path, destiny file, destiny path permission, destiny file permission, error on reading or writing and any more information to help developers to identify and solve the problem when we report here?

If any one can provide this hack, I can test on the sites where I have the problem always happening.

Since 01/07/2010, it is more than 18 months. I insist on this feature because Drupal 7 release has a focus on user experience, accesibility and usability of the system to make it easier to use.
http://www.linuxforu.com/2011/12/dries-buytaert-interview-drupal-8-busin...
http://www.centernetworks.com/interview-drupal-founder-dries-buytaert
http://devworks.thinkdigit.com/Features/Interview-Dries-Buytaert--Creato...

bojanz’s picture

Who says it's even the same problem during the same 18 months? D7 was in alpha when I reported this.
So far we have 10 people reporting a problem that sounds the same, with no debug info or hints for fixing this.

A patch with debug info sounds like a good idea, in any case.

valdir.marcos’s picture

I volunteer myself to make tests on some environments where the problem happens to provide debug info or hints.
Can you provide hacks or scripts to make those tests?

valdir.marcos’s picture

Maybe, it would also be good if the patch inform os (operacional system: Linux, Windows, MacOS, etc), total ram, free ram, free space on disk, etc.

valdir.marcos’s picture

Maybe I have a clue on the error: File Transfer failed, reason: Unable to change to directory /sites/all/modules/pathauto.
The correct path is /home/user_name/www/sites/all/modules/pathauto and not /sites/all/modules/pathauto.
Could the update script consider a wrong path for any reason?

VeryCool78’s picture

My solution for all the issues that are related to permission or ftp connection is

on centos from the shell cd to your drupal 7 site root directory and do the following:

  1. sudo chown apache sites/all/themes/ #to fix themes installation issue
  2. sudo chown apache sites/all/modules/ #to fix modules installation issue
  3. sudo chown apache sites/default # to fix ftp issue.

for other servers that are not running centos or redhat replace apache with www-data

cheers

PatchRanger’s picture

valdir.marcos, there are modules that provide something similar to what you are looking for : Site Documentation, System Information and may be Devel Info
May be it will be interesting to you also : I have opened an issue some time ago, that is related to debug info posting - Posting bugreports directly to drupal.org

valdir.marcos’s picture

VeryCool78, I tried both users www-data and apache, but the hosting service uses a different username and does not inform which is... they said I have to work with my own regular user since it works perfectly on any ssh and ftp program (it really works!). They also said this a Drupal's update tool problem that does not happen on Wordpress update tool.

valdir.marcos’s picture

Staratel, thanks for your help. I will check the modules you informed.

But I still believe that Update Module itself should provide more information to help administrators that are not php developers and cannot debug the sources to solve the problem or at least help those administrators to fill up bugreports with all needed information to identify and fix the problem.

I believe that important part of the problem is that I do not have control over apache, ftp, etc servers because my site is hosted on a third party hosting company.

VeryCool78’s picture

Glad my tips put you on the right way at least.

Tapio Holopainen’s picture

I just started using Drupal (7.12) on local Ubuntu (11.10) installation and had the problem described in the original post.

In my case the solution was to change the ftp configuration in /etc/vsftpd.conf which by default did not allow any ftp writing.
As a root do the following.
1. Change the lines:

# Uncomment this to enable any form of FTP write command.
# write_enable=YES

to

# Uncomment this to enable any form of FTP write command.
write_enable=YES

2. Restart vsftpd

#service vsftpd restart

If this the cause for you, root privigeges are needed to fix it.
Anyway, not really a Drupal bug in this case.
br,
Tapio

Vc Developer’s picture

#32 worked for me except this error "File Transfer failed, reason: Unable to remove to directory" and "File Transfer failed, reason: Cannot create directory". I have group as "www" and owner as me. Both have full read and write. I will try changing the owner to wwwrun and see

valdir.marcos’s picture

Drupal update manager message:
ckeditor
Error installing / updating
File Transfer failed, reason: Unable to change to directory /sites/all/modules/ckeditor

Using https://webftp.xxx.net/index.php tool, I see another path...
/www/sites/all/modules/ckeditor (permission is rwxrwxrwx)

How (or where) can I set a diffent ftp home dir on Drupal update manager?

I have tried on http://www.xxx.org.br/users/valdir#overlay=admin/reports/updates/settings but could not find this kind of information.

Taesto’s picture

I had the same problem.

It seems that for the fast update method to work, you need to have the ftp user with which you login to be the owner of the modules' files and directories that you want to have updated.

So the problem in my opinion is not with the update manager itself.

For me what happened is that at first it didnt work because the first time I installed the modules, I copy/pasted them from a different place on the server, and so they were owned by "root", and later the update failed.

Then I tried to change the permissions, but some sub-directories had their permissions unchanged, so I had modules that were half-uninstalled with some directories remaining and the update still failed.

I deleted everything and re-uploaded everything from ftp. I then used the same ftp user to update and it worked.

syaman’s picture

@VeryCool78 Thank you your tip helped resolve my frustration

VeryCool78’s picture

You are welcom

ls4680’s picture

Title: Update manager can't install module due to file ownership issues » Update manager can't install module
Version: 8.x-dev » 7.15
Status: Postponed (maintainer needs more info) » Active

RESOLVED: see below

I have 2 FreeBSD servers one in production, the other used for testing. I have root access to both.
On my production server Drupal 7.15 module installer works as documented, but on my testing server any attempt to install a module fails with the following message:
Error installing / updating
File Transfer failed, reason: Cannot create directory /public_html/sites/all/modules/pathauto/.
Note the "." at the end
FTP LOG FILE:
Aug 23 10:05:04 home ftpd[29162]: connection from localhost (127.0.0.1)
Aug 23 10:05:04 home ftpd[29162]: FTP LOGIN FROM localhost as smith
Aug 23 10:05:04 home ftpd[29162]: session root changed to /home/smith
Aug 23 10:05:05 home ftpd[29163]: connection from localhost (127.0.0.1)
Aug 23 10:05:05 home ftpd[29163]: FTP LOGIN FROM localhost as smith
Aug 23 10:05:05 home ftpd[29163]: session root changed to /home/smith
Aug 23 10:05:05 home ftpd[29163]: mkdir /public_html/sites/all/modules/pathauto (wd: /; chrooted)
Aug 23 10:05:05 home ftpd[29163]: mkdir /public_html/sites/all/modules/pathauto/. (wd: /; chrooted)

The update manager correctly creates the directory but no files are uploaded.
drwxr-xr-x 2 smith smith 512 Aug 23 10:05 pathauto/

If I try and install the module again after Drupal has created the directory I get the following error message:
Error installing / updating
File Transfer failed, reason: Cannot create directory /public_html/sites/all/modules/pathauto
Note now there is no "." at the end.
FTP LOG FILE:
Aug 23 10:07:21 home ftpd[29167]: connection from localhost (127.0.0.1)
Aug 23 10:07:21 home ftpd[29167]: FTP LOGIN FROM localhost as smith
Aug 23 10:07:21 home ftpd[29167]: session root changed to /home/smith
Aug 23 10:07:22 home ftpd[29168]: connection from localhost (127.0.0.1)
Aug 23 10:07:22 home ftpd[29168]: FTP LOGIN FROM localhost as smith
Aug 23 10:07:22 home ftpd[29168]: session root changed to /home/smith
Aug 23 10:07:22 home ftpd[29168]: mkdir /public_html/sites/all/modules/pathauto (wd: /; chrooted)

FTP LOG FILE of system that works:
Aug 23 10:41:08 ns1 ftpd[82608]: connection from localhost (127.0.0.1)
Aug 23 10:41:08 ns1 ftpd[82608]: FTP LOGIN FROM localhost as admin
Aug 23 10:41:08 ns1 ftpd[82608]: session root changed to /home/admin
Aug 23 10:41:08 ns1 ftpd[82609]: connection from localhost (127.0.0.1)
Aug 23 10:41:08 ns1 ftpd[82609]: FTP LOGIN FROM localhost as admin
Aug 23 10:41:08 ns1 ftpd[82609]: session root changed to /home/admin
Aug 23 10:41:08 ns1 ftpd[82609]: mkdir /domains/mydomain.net/public_html/sites/all/modules/pathauto (wd: /; chrooted)
Aug 23 10:41:08 ns1 ftpd[82609]: put /domains/mydomain.net/public_html/sites/all/modules/pathauto/pathauto.admin.inc = 16991 bytes (wd: /; chrooted)
. . .

Production system that works:
FreeBSD 8.2
Apache/2.2.22 (FreeBSD) DAV/2 PHP/5.3.10 with Suhosin-Patch mod_ssl/2.2.22 OpenSSL/0.9.8q
PHP Version 5.3.10
MySQL 5.5.20
Firewall: NONE

Testing system that does not work:
FreeBSD 7.3
Apache/2.2.22 (FreeBSD) PHP/5.4.5 mod_ssl/2.2.22 OpenSSL/1.0.1c DAV/2
PHP Version 5.4.5
MySQL 5.1.54
Firewall: Server is behind a DSL Router. I can manually FTP upload (put) to the server from a local client and also manually FTP download (get) to the server from other servers outside the firewall.

Because I have 2 servers, one that always works, and one that always gets this error believe that this is a settings/config error. I can compare any settings if I can just get some guidance on where to look, OS, FTP, Apache, PHP, MySQL, etc. I have root shell access to both servers.

RESOLVED: I was able to resolve this be removing PHP 5.4.5 and installing PHP 5.3.15. I am not sure if the problem was 5.4.5 or just the PHP5.3 extensions that I used. It now works as documented.

dan.hu’s picture

same problem here, with v 7.15

Leeteq’s picture

Version: 7.15 » 7.x-dev
ecommercium’s picture

Version: 7.x-dev » 7.15

Error installing / updating
File Transfer failed, reason: Cannot remove directory /srv/http/sites/all/...

Leeteq’s picture

This appears to be related to PHP 5.4.x, and not specifically to Drupal 7.15...

@ecommercium:
Please confirm if you are experiencing this on PHP 5.4.x.
(and if possible, also confirm if it is also happening on PHP 5.3.x or not)

thorsten.’s picture

@Leeteq:
Drupal 7.15 with PHP 5.4.6 or PHP 5.3.13 here but everytime the same:

# Failed: Error installing / updating
# Failed: File Transfer failed, reason: Cannot create directory ...

FTP works ... -.-

cheese_monkey’s picture

Version: 7.15 » 7.14

With 7.14 and PHP 5.3.16, I get

comment_notify
    Error installing / updating
    File Transfer failed, reason: Cannot chmod /usr/share/drupal7/sites/all/modules/comment_notify.

Steps for me were:

  1. As my site administrator, went to /admin/modules/install
  2. In the "Install from URL", I pasted the full link of the module I wanted to use, in this case, http://ftp.drupal.org/files/projects/comment_notify-7.x-1.1.tar.gz
  3. Clicked Install. Boom, error.

Looking in the filesystem, I see something that strikes me as odd:

$ ls -lF sites/all
total 8
drwxr-xr-x 2 www-data www-data 4096 Sep 10 22:10 modules/
drwxr-xr-x 3 www-data www-data 4096 Sep  8 19:47 themes/

$ ls -lF sites/all/modules
total 100
-rw-r--r-- 1 www-data www-data  1739 Sep 10 22:10 INSTALL.TXT
-rw-r--r-- 1 www-data www-data 18092 Sep 10 22:10 LICENSE.txt
-rw-r--r-- 1 www-data www-data    86 Sep 10 22:10 comment_notify-rtl.css
-rw-r--r-- 1 www-data www-data    86 Sep 10 22:10 comment_notify.css
-rw-r--r-- 1 www-data www-data  8661 Sep 10 22:10 comment_notify.inc
-rw-r--r-- 1 www-data www-data   537 Sep 10 22:10 comment_notify.info
-rw-r--r-- 1 www-data www-data  6514 Sep 10 22:10 comment_notify.install
-rw-r--r-- 1 www-data www-data   364 Sep 10 22:10 comment_notify.js
-rw-r--r-- 1 www-data www-data 27563 Sep 10 22:10 comment_notify.module
-rw-r--r-- 1 www-data www-data  6975 Sep 10 22:10 comment_notify.test
-rw-r--r-- 1 www-data www-data  2167 Sep 10 22:10 comment_notify.tokens.inc

The ownership is correct, but shouldn't the modules files have been unpacked into a "comment_notify" subdirectory? If other code is trying to chown a subdir that never got created, that would explain the error.

I just checked the .tar.gz file itself, and there is a subdir inherent in the tarball, so it's something to do with the unpacking, not the file itself.

Other comments in this thread say "oh well, people can just install modules by hand". As I am a new Drupal user and this was my first attempt at installing any module ever, that's not the most helpful advice ever, especially as the only current "how to install a module" documentation I can find says to use the method I just tried.

I'll keep looking...

tim.plunkett’s picture

Version: 7.14 » 8.x-dev

Moving back to D8

jainparidhi’s picture

Hello VeryCool78,

This is related to comment #27. Could you please tell me where I enter the code you have provided?

I am facing the same issue. I am hosting on HostMonster. Not sure where I can enter this, or how to navigate to the appropriate directory. Is there a program I have to use? Sorry, I am total newbie.

Thank you in Advance,

PJ

valdir.marcos’s picture

Version: 8.x-dev » 7.15

The third party hosting company said tha Drupal 7 update module should change permission right after files are uploaded:
---------------
move_uploaded_file($_FILES['attachement']['tmp_name'], $uploadfile)
chmod($uploadfile, 0777);
---------------

That is because my site user and Apache are different and my user does not have root power to change file owner or file permission on /tmp/ directory of a hosting shared server...

On Drupal Admin area, I have change temp directory so I could see uploaded file owner and permission... The files uploaded have apache owner (not regular site user as owner). And the problem still persists because of owner/permission settings after upload .

tim.plunkett’s picture

Version: 7.15 » 8.x-dev

Please stop changing the version.

jainparidhi’s picture

I used Drush to solve my problem of updating modules.

Thanks,
PJ

jurjenn’s picture

For me, this solved the problem:

Go to /includes/filetransfer/ftp.inc replace:

 protected function createDirectoryJailed($directory) {
    if (!ftp_mkdir($this->connection, $directory)) {
      throw new FileTransferException("Cannot create directory @directory", NULL, array("@directory" => $directory));
    }
  }

on line 70

with:

 protected function createDirectoryJailed($directory) {    
	if (!ftp_chdir($this->connection, $directory) && !ftp_mkdir($this->connection, $directory)) {
		  throw new FileTransferException("Cannot create directory @directory", NULL, array("@directory" => $directory));		
	}	
  }
G’s picture

Thank you very much VeryCool78, your solution our problems in a jiffy.
I have made some slight adjustments however:

Posted by VeryCool78 on January 12, 2012 at 7:07pm
My solution for all the issues that are related to permission or ftp connection is
on centos from the shell cd to your drupal 7 site root directory and do the following:

sudo chown apache:apache drupal/sites/all/themes/ #to fix themes installation issue
sudo chown apache:apache drupal/sites/all/modules/ #to fix modules installation issue
sudo chown apache:apache drupal/sites/default # to fix ftp issue.
for other servers that are not running centos or redhat replace apache with www-data

cheers

and in addition make sure that you also...

sudo chown apache:apache drupal/modules/ #to fix modules installation issue that install direct into Drupal root (like Superfish)
sudo chown apache:apache drupal/themes/

And ensure that the correct read write execute commands are permitted (755).

The other helpful thing which I found was changing the apache settings.
APC settings in php.ini
change
apc.stat=1
apc.shm_size=64M
to
apc.stat=0
apc.shm_size=128M

Thanks to Bennos and T-34 for the information. Original posts by Bennos Apache problems and APC fragmentation and T-34 Unable to allocate memory pool

cheese_monkey’s picture

Please stop changing the version.

The guidelines say "Choose the version where you noticed the problem," and it's a bug in 7.x. If you need to fix it in 8 first, that's great, but don't be surprised when people finding the bug in the latest released version keep reporting it against that version...

In practice, marking these kinds of bugs as "version X" means they're forgotten about and never backported to X-1, which is what we're all using.

webchick’s picture

"The guidelines say "Choose the version where you noticed the problem," and it's a bug in 7.x. If you need to fix it in 8 first, that's great, but don't be surprised when people finding the bug in the latest released version keep reporting it against that version..."

Can you point out where you read this text? We definitely need to fix it, because that's incorrect.

"In practice, marking these kinds of bugs as "version X" means they're forgotten about and never backported to X-1, which is what we're all using."

Actually, while it might sound strange, that's really not true at all. The vast majority of the core development team looks solely at the 8.x queue right now, since 8.x is in active development. So putting things against 8.x does indeed get it the most eyes the fastest.

cheese_monkey’s picture

"Can you point out where you read this text? We definitely need to fix it, because that's incorrect."

The issue queue handbook, linked to from the issue settings grid ("Descriptions of the priority and status values..."). Follow that link, then go to the "Issue submission form fields" subsection.

"The vast majority of the core development team looks solely at the 8.x queue right now,"

Yeah, that's the problem. People look solely at 8.x, bugs get fixed in 8.x and then forgotten about. Bugs reported against 7.x are ignored, or closed and answered with "use 8.x instead". Not everybody is free to upgrade out of the stable version to the development version.

tim.plunkett’s picture

I've fixed the documentation (http://drupal.org/node/314328).

It now says "Choose the current development version of the module, theme, or Drupal itself. Don't just pick the version you noticed the problem in. For more information, see Drupal's backport policy."

Now, let's continue fixing this problem!

kingfisher64’s picture

#27 worked for me lovely thanks so much VeryCool78. This issue really started to get to me.

Can use update manager on ubuntu 12.04 lovely.

webchick’s picture

Category: bug » support

If file/folder permissions are the issue then this seems to be a support request, not a bug report?

Would someone be willing to update the docs to be more clear? http://drupal.org/documentation/modules/update

valdir.marcos’s picture

Category: support » bug

It is a bug report (at least, it is a feature request) on shared host third party companies.
It would help a lot if this module provided more information on such errors to aid building a solution.

Backport policy
Where I put "needs backport to D7" issue tag?

tim.plunkett’s picture

Category: bug » feature

It's already tagged as such.

webchick’s picture

Category: feature » support
Priority: Major » Normal

If #27 is the solution, there's absolutely nothing we can do in Drupal to solve this. To me, this needs a documentation update and that's it.

highmastdon’s picture

As mentioned here you can change permissions or either change owner (chown). I'd recommend to use chown user:www-data or chown user:apache when you're using something like webmin/virtualmin in order to make your permissions equal sorted out.

Pycouz’s picture

Same problem here.

Solved by changing PHP version from 5_4 to 5_3 in OVH .htaccess.

DanZ’s picture

Category: support » bug
FileSize
27.05 KB

The original issue is clearly a bug which still exists. It is NOT a permissions/ownership issue. Mind you, some of the posters had permissions issues that caused similar problems, but that's not what's happening here:

[drupal@danlinux modules]$ ls -ld panels scroll_to_top social-share wysiwyg 
drwxr-xr-x. 12 drupal drupal 4096 Dec 11 21:33 panels
drwxr-xr-x.  2 drupal drupal 4096 Dec 11 21:33 scroll_to_top
drwxr-xr-x.  2 drupal drupal 4096 Jul 12 07:24 social-share
drwxr-xr-x.  5 drupal drupal 4096 Dec 11 21:33 wysiwyg

The social-share update failed. The other three succeeded (see attachment). The error was "File Transfer failed, reason: Cannot create directory /var/www/html/drupal_test/sites/all/modules/social-share". Can you see a permissions difference? I can't.

Before you complain about the directory name, ubercart does the same thing on another one of my installs.

This is PHP 5.3.18, Drupal 7.17. I'd be happy to provide any other troubleshooting information you'd like.

DanZ’s picture

In the interest of actually making some progress here (instead of just complaining that it's broken), the error is caught at the end of Updater::update() in includes/Updater.inc.

The exception is thrown from FileTransferFTPExtension::createDirectoryJailed.

The actual error occurs on a call to ftp_mkdir().

So, the first question is this: Why would this fail on some directories, and not others, when it's not a permissions or ownership issue? The second question is this: Why should the PHP version matter?

DanZ’s picture

The error shows up in the log. The second message is clearly related. The first one, I'm not sure, but it happened only a second before, and is clearly due to a bug.

Dec 11 21:32:54 danlinux drupal: http://localhost/drupal_test|1355290374|php|127.0.0.1|http://localhost/drupal_test/batch?id=4&op=do|http://localhost/drupal_test/batch?op=start&id=4|1||Notice: Undefined variable: finished in _batch_process() (line 344 of /var/www/html/drupal_test/includes/batch.inc).

Dec 11 21:33:54 danlinux drupal: http://localhost/drupal_test|1355290434|php|127.0.0.1|http://localhost/drupal_test/authorize.php?batch=1&id=6&op=do|http://localhost/drupal_test/authorize.php?batch=1&op=start&id=6|1||Warning: ftp_mkdir(): Create directory operation failed. in FileTransferFTPExtension->createDirectoryJailed() (line 71 of /var/www/html/drupal_test/includes/filetransfer/ftp.inc).
Donovan’s picture

@VeryCool78 You rock!

Thank you. I've been plagued by permissions-related issues on a fresh D7.19 install. Your suggestion saved the little bit of hair I have left!

Thank you,

Donovan

drupalsilval’s picture

I was having the same problem like @valdir.marcos in #34 while trying to update a module. Drupal tried to cd to /sites/all/modules/... during the update.
The (not-so-ideal-but-temporarily-solved-my-problem) solution was to:
1) Have the ownership of all files in my site changed to Apache's user group. In my case: "nobody". Similar to what was described in #27 above. See more details in http://drupal.org/node/244924

2) create a "sites" logical link in the ftp root directory of my hosted drupal site space:
[your-ftp-root-dir]$ ln -s www/sites sites
this made Drupal correcly cd to the right directory when trying to update it.
(I could not find any Drupal ftp config to make it add that "www" prefix to the ftp root dir while updating a module)

3) In some cases, for some modules (I haven't figure out why yet) I had to give read permissions to everyone to the existing module directory:
Example: to update module "search_api": [search_api] $ chmod -R 774 *
Then, after the update I changed the permissions back to file permissions recommendations described in (1)

Hope it helps.

grindlay’s picture

I use suPHP on all sites so changing permissions as per #27 not an option (sites runs with permission of their owners). So the problem is the Apache user doesn't have permission to create/remove directories - nothing to do with permissions of the directories themselves.
What I'm looking for is a module to authenticate with the site's FTP user that has full read/write/rename/delete permission on folders. Can't find this in default install - is it available asa module ?
Also, issue *seems* to have appeared after going PHP 5.3.x -> 5.4.x

grindlay’s picture

DrMiaow’s picture

I can confirm that under Windows IIS 7.5/SQL2008 Drupal 7.19 the solution is to switch from PHP 5.3 to PHP 5.4 to update modules.

Why I don't know - but it works.

Is Drupal 'ready' for PHP 5.4?

Dkids’s picture

DrMiaow -

I have run in to this very same issue with IIS. I upgraded tonight to PHP 5.4 and still get the same error. Did you happen to make any other config chnages prior to re-attempting the updates?

rosewoodmarketing’s picture

...it does seem like it might have something to do with permissions...

I've also started receiving the "Error installing / updating" "File Transfer failed" "Cannot create directory" messages and am not able to update some modules. We just updated our server to PHP 5.4.9 recently (before these error messages started).

One thing I started doing recently is maintaining a folder on my desktop with all the modules I normally use. Rather than downloading them every time I just upload the whole sites/all/modules folder and I get them all. It seems like my grief must be coming from file permissions created somewhere in that upload. Some of my newer sites I've been uploading with Coda rather than using the cPanel File Manager or Filezilla. Not sure where the weak link is.

I just deleted some of the sites/all/modules folders I was having issues with, downloaded from Drupal the older versions of the modules again, uploaded them to sites/all/modules (via Coda), then re-ran the admin/modules/update script on those modules.

Now everything works. No errors.

DanZ’s picture

There's no need to post further messages here if you've fixed your problem by adjusting permissions.

...it does seem like it might have something to do with permissions...

Yes, a bad file permission configuration can produce this problem or a similar one. However, that's a configuration issue, and separate from the issue reported here.

See #63. This problem can happen when all the permissions are correct. There's something else going on here, and a root cause has not yet been discovered.

If your permission are correct but you're still seeing this, please do report what troubleshooting information you can.

DanZ’s picture

Issue summary: View changes

Added that it's more than a permission problem.

DrMiaow’s picture

In reply to Dkids

No. Switching back PHP 5.3 just for an upgrade does 'solve' it for me though.

Krizalys’s picture

Facing the same issue as reported in comment #41 under Drupal 7, I did some testing, and it does look like the directory removal issue may also be caused by the PHP version you are using. I tested with PHP 5.2, 5.3 and 5.4, all on the same Linux machine, same PHP configuration, file permissions, Drupal installation, and only PHP 5.4 is facing this issue.

After further investigation (http://drupal.org/node/1785216, http://drupal.org/node/1313560, http://drupal.org/node/1915088), I discovered that RecursiveDirectoryIterator is behaving differently in PHP 5.4 than it used to be in PHP 5.2/5.3. The old behavior, which allows Drupal to work as expected, is indeed the one that looks buggy to me, and I reported a bug to the PHP group. If this bug is confirmed, Drupal would be found relying on it to operate properly and would have to be fixed to accomodate newer PHP versions, starting from 5.4.

Krizalys’s picture

Following my report, the PHP bug has been confirmed and the fix has been scheduled for PHP 5.3.23, meaning that starting from this version, the update manager may start failing for the same reasons as PHP 5.4.

This issue is primarily affecting Drupal 7, a patch has been proposed here: http://drupal.org/node/1915088. I tested it successfully on PHP 5.4.11.

For Drupal 8, the current code should keep using FilesystemIterator::SKIP_DOTS, or the PHP requirements should be lifted up to PHP 5.3.23 since previous versions always assume this flag.

athos99’s picture

When i update a module ( infomaniak provider), i've the error message.

Error installing / updating
File Transfer failed, reason: Unable to change to directory

After some investigation I found the problem, at login the FTP is set to a default directory ( the function ftp_pwd($this->connection); return "web/", a standard ftp is set to root folder)

In file /include/filetransfer/filetransfert.inc there is a bug in function findChroot()

while (count($parts)) {
$check = implode($parts, '/');
if ($this->isFile($check . '/' . drupal_basename(__FILE__))) {
// Remove the trailing slash.
return substr($chroot, 0, -1);
}
$chroot .= array_shift($parts) . '/';
}

In while loop, we check if the variable $check is a file. But the '/' is not assigned a first position of filename. In my case, i'have a default pwd and the filename ($check) is append to default pwd.
The $check filename must be prefixed with a '/'.
A possible correction is :

while (count($parts)) {
$check = '/'.implode($parts, '/'); //*****<- prefixed with a /
if ($this->isFile($check . '/' . drupal_basename(__FILE__))) {
// Remove the trailing slash.
return substr($chroot, 0, -1);
}
$chroot .= array_shift($parts) . '/';
}

maxkam1@hotmail.com’s picture

Amend VSFTPd.conf to allow Drupal updates to work. Following is my solution, works for me.

Go to command mode

1. su root

2. nano /etc/vsftpd.conf to edit

3. change
# Uncomment this to enable any form of FTP write command.write_enable=YES# Uncomment this to allow local users to log in.local_enable=YES

maxkam1@hotmail.com’s picture

Amend VSFTPd.conf to allow Drupal updates to work.

Go to command mode

1. su root

2. nano /etc/vsftpd.conf to edit

3. change
# Uncomment this to enable any form of FTP write command.
write_enable=YES

# Uncomment this to allow local users to log in.
local_enable=YES

srece’s picture

Getting same issue...was looking at the Module folder after trying to install "admin_menu" module through the install module on D7 and I noticed the subdirectories of the mod were install at the root of the modules folder. So I tried installing something else like views, and it worked fine.

So it looks like nothing more than some modules are zipped incorrectly, so I just installed manually through FTP and it worked fine and most other modules will install so it seems to me if you run across this check your modules folder first the error message is misleading.

d.cochran’s picture

How I Fixed My Issue

Now this may not be the solution for everyone here, but it might help some.

The culprit (for me): Disk Quota Space for "web user" exceeded.

We fell into a lot of the same criteria as people here:

  • Drupal's upload manager would fail
  • Cannot create directory
  • Permissions were correct
  • Ownership was correct
  • Some modules would work and others would not
  • Able to upload via SSH Shell

For some accounts on our server, we restrict their Disk Space to 250MB. Once they reach this quota, they can not create anything else on the server via their "web user". This prevented Drupal from creating directories as it operates under their "web user".

This also explained why some modules would work and others would not. If they had 300k left on their disk space and the module was under 300k, it would work. However, once the module topped the disk space - ERROR!

It also explained why I was able to upload via SSH Shell. When I use SSH Shell, probably like most people here, I upload via my server's "root user". After the files are uploaded, I then simply change ownership. I was able to upload all the files because my "root user" was not restricted to the Disk Quota like my "web user".

I increased the Disk Quota Space for the domain and - vola! - success.

~dc

hip’s picture

iprok’s picture

It seems that update modules checks directory (sites/) owner not permissions just like it's described here https://drupal.org/node/1215090. I just don't understand why it's done in such way. I have complex acl settings on my hosting and owner of all files is root and write permissions are granted using acls. So why should I change owner just to make update module work without ftp\ssh and so on?

Can it be fixed in drupal 7.*?

Fix seems to be just replacing
if (fileowner($project_real_location) == fileowner(conf_path()))
to
if (is_writeable(conf_path()))

filippo.piffaretti’s picture

Great athos99!!!

Your suggestion #77 solved my the problem smoothly!

"Error installing / updating
File Transfer failed, reason: Unable to change to directory"

Hellsyl’s picture

Version: 8.x-dev » 7.23
Component: update.module » file system

Thx athos99, #77 work fine for me !
Is this solution only works at infomaniak?

(this is my first post and I made a mistake in this form post that I filled)

tim.plunkett’s picture

Version: 7.23 » 8.x-dev
Component: file system » update.module
rhodie’s picture

jurjenn's fix in #50 fixed my problem updating google_analytics. (The error was "File Transfer failed, reason: Cannot create directory.") I was updating a number of modules at the same time, but google_analytics was the only one that gave the error.

valdir.marcos’s picture

Priority: Normal » Major

Shouldn't the priority of this bug report be elevated to MAJOR after more than 3 year of reports on both D7 and D8?
Marking it as major would make it appear on the radar of core developers on this special moment that Drupal 8 is about to be released.
This Update Module will be really important before migration from D7 to D8.

valdir.marcos’s picture

Issue summary: View changes

Adjusted wording.

Mech45’s picture

Thanks a lot !

It works HARD !

valdir.marcos’s picture

Problem still persists on Drupal 8 alpha 9 and 10 on a shared hosting company where I can not change Linux server settings.

I am trying to install http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz or http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip or (blog-8.x-2.x-dev.tar.gz or blog-8.x-2.x-dev.zip) via uploaded file.

On Drupal 8 alpha 9, I receive the error message:

English:
"Archivers can only operate on local files: temporary://update-cache-22399eda/blog-8.x-2.x-dev.tar.gz not supported"
Brazilian Portuguese:
"Arquivadores só podem operar em arquivos locais: temporary://update-cache-22399eda/blog-8.x-2.x-dev.tar.gz não é suportado"

-------------
https://github.com/jrholowka/drupal-opencart/blob/master/web/profiles/st...
msgid "Archivers can only operate on local files: %file not supported"
msgstr "Arquivadores só podem operar em arquivos locais: %file não é suportado"
-------------

On Drupal 8 alpha 10, I receive no error message and nothing is installed and no error message appears on http://xxx.yyy.br/drupal8a10/admin/reports/dblog

valdir.marcos’s picture

Issue summary: View changes

I am using Drupal 8 Alpha 10.

The shared hosting company says that Drupal has many php files trying to use the directory "temporary://" which is not available. Drupal should request an alternative path when directory "temporary://" is not available.

I have created a new temporary directory /home/xxx/tmp/ on my FTP home user with permission 777.

I have changed the temporary path in http://xxx.yyy.br/drupal8a10/admin/config/media/file-system to the new temporary path.

Then I try to install the blog module again using tool on http://xxx.yyy.br/drupal8a10/admin/modules/install:
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

On the first time, the files are correctly downloaded to /home/xxx/tmp, but nothing is extracted:
---------------
ls -lh tmp/
total 0
drwxrwxrwx 2 user group 88 Apr 22 18:54 update-cache-be72b049
drwxrwxrwx 3 user group 72 Apr 22 18:54 update-extraction-be72b049

ls -lh tmp/update-cache-be72b049/
total 36K
-rw-rw-r-- 1 user group 14K Apr 22 18:54 blog-8.x-2.x-dev.tar.gz
-rw-rw-r-- 1 user group 18K Apr 22 18:56 blog-8.x-2.x-dev.zip

ls -lh tmp/update-extraction-be72b049/
total 0
---------------

From the second time on, I receive the error message:
---------------
Warning: file_exists(): open_basedir restriction in effect. File(/home) is not within the allowed path(s): (/home/xxx/:/tmp:/usr/local/lib/php:./) in drupal_mkdir() (line 1548 of core/includes/file.inc).
Warning: mkdir(): open_basedir restriction in effect. File(/home) is not within the allowed path(s): (/home/xxx/:/tmp:/usr/local/lib/php:./) in _drupal_mkdir_call() (line 1579 of core/includes/file.inc).
blog-8.x-2.x-dev.zip does not contain any .info.yml files.
---------------

Also the file is downloaded on the wrong place:
---------------
ls -lh tmp/
total 16K
-rw-rw-r-- 1 user group 13K Apr 22 18:56 blog-8.x-2.x-dev.tar.gz
drwxrwxrwx 2 user group 128 Apr 22 18:56 update-cache-be72b049
drwxrwxrwx 2 user group 48 Apr 22 18:56 update-extraction-be72b049
---------------

On Drupal 8 alpha 10, I receive these error messages on http://xxx.yyy.br/drupal8a10/admin/reports/dblog
All the messages say that the user was anonymous, but I was logged as Valdir:
-------------------
Tipo sistema de arquivos
Data terça-feira, abril 22, 2014 - 18:32
Usuário Anônimo (não verificado)
Localização
Referência

Mensagem Não foi possível apagar o arquivo temporário "temporary://blog-8.x-2.x-dev.tar.gz" durante a limpeza.
Free Translation: It is not possible to delete the temporary file "temporary://blog-8.x-2.x-dev.tar.gz" during the ccleaning.

Severidade Erro
Nome do servidor
Operações
-------------------
Tipo agendador de tarefas
Data terça-feira, abril 22, 2014 - 18:32
Usuário Anônimo (não verificado)
Localização
Referência

Mensagem Symfony\Component\DependencyInjection\Exception\RuntimeException: You have requested a synthetic service ("request"). The DIC does not know how to construct this service. em service_container_prod->getRequestService() (linha 3411 de /home/xxx/www/drupal8a10/sites/default/files/php/service_container/service_container_prod.php/3cf8da017fe6afa8e262984f583051030c6c8e3e31a694ac2b48f1d58c1d9a4e.php).

Severidade Erro
Nome do servidor
Operações
-------------------
Tipo agendador de tarefas
Data terça-feira, abril 22, 2014 - 18:32
Usuário Anônimo (não verificado)
Localização
Referência

Mensagem As tarefas agendadas já terminaram de rodar.
Free Translation: Cron tasks finished.
Severidade Nota
Nome do servidor
Operações
-------------------
Tipo php
Data terça-feira, abril 22, 2014 - 18:56
Usuário Valdir
Localização http://xxx.yyy.br/drupal8a10/admin/modules/install
Referência http://xxx.yyy.br/drupal8a10/admin/modules/install

Mensagem Warning: file_exists(): open_basedir restriction in effect. File(/home) is not within the allowed path(s): (/home/xxx/:/tmp:/usr/local/lib/php:./) em drupal_mkdir() (linha 1548 de /home/xxx/www/drupal8a10/core/includes/file.inc).

Severidade Aviso
Nome do servidor 201.13.191.65
Operações
-------------------
Tipo php
Data terça-feira, abril 22, 2014 - 18:56
Usuário Valdir
Localização http://xxx.yyy.br/drupal8a10/admin/modules/install
Referência http://xxx.yyy.br/drupal8a10/admin/modules/install

Mensagem Warning: mkdir(): open_basedir restriction in effect. File(/home) is not within the allowed path(s): (/home/xxx/:/tmp:/usr/local/lib/php:./) em _drupal_mkdir_call() (linha 1579 de /home/xxx/www/drupal8a10/core/includes/file.inc).

Severidade Aviso
Nome do servidor 201.13.191.65
Operações
-------------------

Is there any other information that I could provide to help?

valdir.marcos’s picture

Problem still persists on Drupal 8 alpha 11 on a shared hosting company where I can not change Linux server settings.

I have changed the temporary path in http://xxx.yyy.br/drupal8a11/admin/config/media/file-system to the new temporary path.

Then I try to install the blog module again using tool on http://xxx.yyy.br/drupal8a10/admin/modules/install:
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

On the first time, the files are correctly downloaded to /home/xxx/tmp, but nothing is extracted:
---------------
ls -lh /home/xxx/tmp
total 0
drwxrwxrwx 2 xxx xxx 88 Apr 23 11:55 update-cache-2a331153
drwxrwxrwx 2 xxx xxx 128 Apr 22 18:56 update-cache-be72b049
drwxrwxrwx 3 xxx xxx 72 Apr 23 11:55 update-extraction-2a331153
drwxrwxrwx 2 xxx xxx 48 Apr 22 18:56 update-extraction-be72b049

ls -lh /home/xxx/tmp/update-cache-2a331153/
total 16K
-rw-rw-r-- 1 xxx xxx 14K Apr 23 11:55 blog-8.x-2.x-dev.tar.gz

ls -lh /home/xxx/tmp/update-cache-be72b049/
total 36K
-rw-rw-r-- 1 xxx xxx 14K Apr 22 18:54 blog-8.x-2.x-dev.tar.gz
-rw-rw-r-- 1 xxx xxx 18K Apr 22 18:56 blog-8.x-2.x-dev.zip

ls -lh /home/xxx/tmp/update-extraction-2a331153/
total 0
drwxrwxrwx 4 xxx xxx 264 Apr 23 11:55 blog

ls -lh /home/xxx/tmp/update-extraction-be72b049/
total 0
---------------

On Drupal 8 alpha 11, on http://xxx.yyy.br/drupal8a11/admin/modules/install after trying to install I receive the message "Você deve fornecer uma URL ou enviar um arquivo para instalar." free translated as "Your must provide an URL or send a file to install.".
Nothing is installed and no error message appears on http://xxx.yyy.br/drupal8a10/admin/reports/dblog

aangel’s picture

Can confirm this error and it is occurring on a new MAMP 3.0.5 install on OS X 10.9.2.

jurjenn’s picture

FileSize
769 bytes

This patch fixed it for me, version Drupal 7

valdir.marcos’s picture

I am using Drupal 8 Alpha 12 on a shared hosting company.

The option INSTALL NEW MODULE disappeared from the Extend menu on http://www.xxx.yyy.zz/drupal8a12/admin/modules.

Instead we see a new message:
"(free translation) Download extra modules to extend Drupal functionalities.
(original in English) Regularly review available updates to maintain a secure and current site. Always run the update script each time a module is updated. Enable the Update Manager module to update and install modules and themes."

When I try to enable installing new modules on Update Manger settings (http://www.xxx.yyy.zz/drupal8a12/admin/reports/updates/settings), I receive the following error message on:
http://www.xxx.yyy.zz/drupal8a12/admin/reports/event/33
-------------------------
Type php
Date friday, may 30, 2014 - 14:52
User valdir
Place http://www.xxx.yyy.zz/drupal8a12/admin/reports/updates/settings
Reference http://www.xxx.yyy.zz/drupal8a12/admin/modules
Message InvalidArgumentException: Class "\Drupal\update\UpdateSettingsForm" does not exist. in Drupal\Core\DependencyInjection\ClassResolver->getInstanceFromDefinition() (line 29 of /ddd/xxx/www/drupal8a12/core/lib/Drupal/Core/DependencyInjection/ClassResolver.php).
Severity Error
Server name xxx.xxx.xxx.xxx
Operations (none)
-------------------------

Now, I can not even try to install automaticaly any new module on Drupal 8 Alpha 12.

valdir.marcos’s picture

I am using Drupal 8 Alpha 13 on a shared hosting company.

During Drupal 8 Alpha 13 installation process, I receive this error message:

http://www.xxx.yyy.zz/drupal8a13/core/install.php?langcode=pt-br&profile...

Installing Drupal

The installation has encountered an error.
Please continue to the error page

An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: http://www.xxx.yyy.zz/drupal8a13/core/install.php?langcode=pt-br&profile...
StatusText: Internal Server Error
ResponseText:
Erro no servidor - Erro 500
* { margin: 0; padding: 0; font-family: Arial, Helvetica, sans-serif;}
html { font-size: 62.5%; }
body {
font-size: 100%;
background: #F3F3F3;
text-align: center;
}
div.box {
position: absolute;
border: 1px solid #C0C4C0;
width: 450px;
height: 350px;
left: 50%;
top: 50%;
margin-top: -160px;
margin-left: -450px;
text-align: left;
background-color: #FFF;
background-image: url('http://static.kinghost.net/img/404.jpg');
background-repeat: no-repeat;
background-position: -90px 0px;
padding-left: 400px;
padding-right: 30px;
-moz-border-radius: 12px;
-webkit-border-radius: 12px;
border-radius: 12px;
}
h1,h2 {
font-size: 210%;
font-weight: normal;
display: block;
margin-bottom: 30px;
}
img {
display: block;
margin-top: 35px;
margin-bottom: 20px;
height: 56px;
}
a, a:visited {
color: #2CB0FF;
}
p {
font-size: 125%;
margin-top: 12px;
}
p.inner-box {
background: #EEF7F8;
padding: 12px 8px 12px 8px;
display: inline;
}
p.error {
margin-top: 24px;
margin-left: 8px;
}
Desculpe, mas esta p�gina est� com erro interno no seu c�digo de programa��o
Tente acessar novamente em alguns instantes.
Se o problema persistir, contate o desenvolvedor do site.
Hospedado por KingHost
Crie sua p�gina ou hospede seu site conosco - xxx.yyy.zz
---------------

Going to the suggest error page, I cannot see the error page, but instead I get back to site installation page on the next step and I can finish the installation of the new site.

valdir.marcos’s picture

I am using Drupal 8 Alpha 13 on a shared hosting company.

Error on Comment #92 still persists and I cannot install automatically:
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
nor
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

Nothing is done and I receive NO error messages on module install page (http://www.xxx.yyy.zz/drupal8a13/admin/modules/install) nor on http://www.xxx.yyy.zz.br/drupal8a13/admin/reports/dblog.

valdir.marcos’s picture

I am using Drupal 8 Alpha 14 on a shared hosting company.

I am trying to install the module BLOG as a user perspective.
Error on Comment #92 still persists and I cannot install no module automatically:
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
nor
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

Nothing is done and I receive NO error messages on module install page (http://www.xxx.yyy.zz/drupal8a14/admin/modules/install) nor on http://www.xxx.yyy.zz.br/drupal8a14/admin/reports/dblog.

valdir.marcos’s picture

I am using Drupal 8 Alpha 15 on a shared hosting company.

I am trying to install the module BLOG on a user perspective.
Error on Comment #92 still persists and I cannot install no module automatically:
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
nor
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

Nothing is done and I receive NO error messages on module install page (http://www.xxx.yyy.zz/drupal8a15/admin/modules/install) nor on http://www.xxx.yyy.zz.br/drupal8a15/admin/reports/dblog.

Observation:
https://www.drupal.org/documentation/modules/blog
https://www.drupal.org/project/blog

Andre-B’s picture

same problem here as described in #92 for Drupal 8 Alpha 15.

watching the tmp folder I could see that the module got downloaded and extracted. I don't know how I could further debug this.

UpdateManagerInstall::submitForm() triggers update_authorize_run_install which should trigger a batch for execution, but apparently it does not happen.

as far as I can tell, batch job is created and is stored in the database.

valdir.marcos’s picture

I am using Drupal 8 Beta 1 on a shared hosting company.

I am trying to install the module BLOG on a user's perspective.
Error on Comment #92 still persists and I cannot install any module automatically:
neither
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
nor
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

Nothing is done and I receive NO error messages on module install page (http://www.xxx.yyy.zz/drupal8b1/admin/modules/install) nor on http://www.xxx.yyy.zz/drupal8b1/admin/reports/dblog.

Observation:
https://www.drupal.org/documentation/modules/blog
https://www.drupal.org/project/blog

joelpittet’s picture

Issue tags: +beta target, +Amsterdam2014
BarisW’s picture

ceng’s picture

I've added a description of some of the problems in 8.0.x branch on the other thread #2311691: Updater UI for installation is broken

valdir.marcos’s picture

@ ceng
I am writing extra information there. Thanks.
Should this issue be closed as duplicate?

valdir.marcos’s picture

I am using Drupal 8 Beta 4 (released 17-dec-2014) on a shared hosting company.

I am trying to install the module BLOG on a user's perspective.
Error on Comment #92 still persists and I cannot install any module automatically:
neither
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
nor
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

Nothing is done and I receive NO error messages on module install page (http://www.xxx.yyy.zz/drupal8b4/admin/modules/install) nor on http://www.xxx.yyy.zz/drupal8b4/admin/reports/dblog.

Observation:
https://www.drupal.org/documentation/modules/blog
https://www.drupal.org/project/blog

cilefen’s picture

Status: Active » Closed (duplicate)
David_Rothstein’s picture

Title: Update manager can't install module » Update manager sometimes can't install modules using FTP due to file ownership issues
Status: Closed (duplicate) » Postponed

Some of the comments certainly are, but I think it's actually a separate issue. (Or maybe like 20 issues rolled into one...)

How about we restore the title to something more similar to #13 to try to focus this. It looks like #94 is the patch to review (a solution which was also mentioned in #50). That's a Drupal 7 patch which needs porting to Drupal 8; however, there's no possible way this can be practically tested in Drupal 8 without #2042447: Install a module user interface does not install modules (or themes) being fixed first, so postponing on that.

bigZ_again’s picture

Maybe cause by anonymous, try another user.

You can check it, just like this:

[root@localhost all]# ftp localhost
Trying ::1...
Connected to localhost (::1).
220 (vsFTPd 3.0.2)
Name (localhost:root): anonymous
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd /var/www/drupal-7.38/sites/all/
550 Failed to change directory.            << be careful!
ftp> exit
221 Goodbye.
[root@localhost all]# ftp localhost
Trying ::1...
Connected to localhost (::1).
220 (vsFTPd 3.0.2)
Name (localhost:root): another_user
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd /var/www/drupal-7.38/sites/all/
250 Directory successfully changed.       << is OK!
ftp> exit
221 Goodbye.
[root@localhost all]#
bigZ_again’s picture

Maybe cause by anonymous, try a normal user.

You can check it, just like this:

[root@localhost all]# ftp localhost
Trying ::1...
Connected to localhost (::1).
220 (vsFTPd 3.0.2)
Name (localhost:root): anonymous
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd /var/www/drupal-7.38/sites/all/modules
550 Failed to change directory.            << be careful!
ftp> exit
221 Goodbye.
[root@localhost all]# ftp localhost
Trying ::1...
Connected to localhost (::1).
220 (vsFTPd 3.0.2)
Name (localhost:root): a_normal_user
331 Please specify the password.
Password:
230 Login successful.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> cd /var/www/drupal-7.38/sites/all/modules
250 Directory successfully changed.       << is OK!
ftp> exit
221 Goodbye.
[root@localhost all]#
David_Rothstein’s picture

Status: Postponed » Active
David_Rothstein’s picture

Status: Active » Needs work

Moving to "needs work", actually, since there's an older patch at #94 that needs discussion and presumably a reroll for Drupal 8.

valdir.marcos’s picture

I am using Drupal 8 rc 2 (released 2015-Oct-21) on a shared hosting company.
I have only a normal user (not root).

I have tried to install the module BLOG on a user's perspective and it worked out automatically and perfectly:
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.tar.gz
and
http://ftp.drupal.org/files/projects/blog-8.x-2.x-dev.zip

To me this ticket can be closed because the automatic installation of new modules from HTTP or OS ocurred CORRECTLY.

Observation:
https://www.drupal.org/documentation/modules/blog
https://www.drupal.org/project/blog

hansfn’s picture

OK, I think I have some useful info here - from testing Drupal 8.0.2 on a shared server (where the web server is running as a separate user "www-data" and the FTP server is unjailed).

  • NB! Setting the owner of sites/default to "www-data" cause the update manager to not use FTP at all. (I think this fact is important and hasn't been noted.)
  • The update manager with FTP actually works if you comment out some code in core/lib/Drupal/Core/FileTransfer/Form/FileTransferAuthorizeForm.php ... I'll list the problems below.
  • If you don't modify core/lib/Drupal/Core/FileTransfer/Form/FileTransferAuthorizeForm.php, the update manager fails with the very little help ful message:
    The website encountered an unexpected error. Please try again later.
    Drupal\Core\Form\EnforcedResponseException: in Drupal\Core\Form\FormBuilder->buildForm() (line 349 of core/lib/Drupal/Core/Form/FormBuilder.php).

First, lets post the problematic code in FileTransferAuthorizeForm.ph

        try {
          $connection_settings = array();
          [...]

          // Set this one as the default authorize method.
          $this->config('system.authorize')->set('filetransfer_default', $filetransfer_backend);
          // Save the connection settings minus the password.
          $this->config('system.authorize')->set('filetransfer_connection_settings_' . $filetransfer_backend, $connection_settings);

          $filetransfer = $this->getFiletransfer($filetransfer_backend, $form_connection_settings[$filetransfer_backend]);

          // Now run the operation.
          $response = $this->runOperation($filetransfer);
          if ($response instanceof Response) {
            $form_state->setResponse($response);
          }
        }
        catch (\Exception $e) {
          // If there is no database available, we don't care and just skip
          // this part entirely.
        }
  1. NB! If you actually catch the exception above and just print the message, you get:
    Can not set values on immutable configuration system.authorize:filetransfer_default. Use \Drupal\Core\Config\ConfigFactoryInterface::getEditable() to retrieve a mutable configuration object
  2. Based on the "immutable configuration" exception, I decided to comment out the two $this->config()->set() lines. Then I actually got back the "EnforcedResponseException" exception - there must be a problem with the $form_state->setResponse line I wondered ...
  3. Based on that idea, I added
              var_dump($response);
              die();

    right after the $this->runOperation line. Voila! It actually work. Yes!!

To summarize: There is a problem with the $this->config()->set() calls and with the $form_state->setResponse() call. I don't have the time or (currently) the skills to investigate further.

Hopefully this can help this bug being fixed. I have 40 students "unable" to install modules right now ... (OK, I just write a shell script installing the modules for all 40 sites in one go using Drush.)

hansfn’s picture

Added "patch" - he-he.

antiart’s picture

I still had the problem with 8.0.2 on a website hoster (Campusspeicher, so my configuration options are limited), I read through half this discussion, tried changing permissions, tried changing php version settings, tried different ftp users/settings... nothing helped much until...

There is a setting in sites/default/settings.php:

/**
 * Access control for update.php script.
 *
 * If you are updating your Drupal installation using the update.php script but
 * are not logged in using either an account with the "Administer software
 * updates" permission or the site maintenance account (the account that was
 * created during installation), you will need to modify the access check
 * statement below. Change the FALSE to a TRUE to disable the access check.
 * After finishing the upgrade, be sure to open this file again and change the
 * TRUE back to a FALSE!
 */
$settings['update_free_access'] = TRUE;

This is a temporary workaround. If this was mentioned on the top of this page, it would have saved me a few hours of my sleep. Also: the error message is still obscure :[

EDIT:
Setting this to true does still not work when installing a module from url and setting my hosting options to: Run PHP as Apache module , PHP version 5.5.9 (Authentification form still displays). It only works for PHP version 5.6.17 as FastCGI app (auth is skipped). I can't test other options, but I hope this helps...

hansfn’s picture

That settings is only for access to update.php - the script that updates the database after installing an updated module. This issue is mainly about the actual installation of modules, but thx for the feedback anyway.

Hm, changing that setting to TRUE shouldn't change anything for the installation of modules. Are you 110% sure that it affects "PHP version 5.6.17 as FastCGI"?

Skin’s picture

Subscribing, I've still this problem with 8.0.3

If I try to install a new module I have this error:

The website encountered an unexpected error. Please try again later.
Drupal\Core\Form\EnforcedResponseException: in Drupal\Core\Form\FormBuilder->buildForm() (line 349 of core/lib/Drupal/Core/Form/FormBuilder.php).

hansfn’s picture

Did you you try my ugly patch in comment 116 / 117

Skin’s picture

With ugly-hack.patch #117 I can install new modules, but for a moment I see a blank page with some errors ( see attached file).

hansfn’s picture

With ugly-hack.patch #117 I can install new modules, but for a moment I see a blank page with some errors ( see attached file).

That's why it's called ugly-hack.patch and not beautiful-clever.patch ;-) Thx for testing and confirming that we have tracked down the problem. I might have time to make a less ugly patch in a few days.

BondD’s picture

Seems some core files not updated to new config calls.
I use a dirty trick to avoid EnforcedResponseException. Not sure if it comply with Drupal coding standard.

BondD’s picture

Title: Update manager sometimes can't install modules using FTP due to file ownership issues » Update manager can't install modules using FTP due broken FileTransferAuthorizeForm
Status: Needs work » Needs review

Change a title slightly

The last submitted patch, 94: patch.diff, failed testing.

The last submitted patch, 117: 842620-116-updatemanager-ugly-hack.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 124: not_so_ugly.patch, failed testing.

BondD’s picture

FileSize
2.93 KB
BondD’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 129: not_so_ugly_fix.patch, failed testing.

BondD’s picture

Status: Needs work » Needs review
hitesh-jain’s picture

Assigned: Unassigned » hitesh-jain
Issue tags: +drupalconasia2016
hitesh-jain’s picture

Assigned: hitesh-jain » Unassigned
Status: Needs review » Needs work

Warning when apply patch from #132
not_so_ugly_fix_test.patch:47: trailing whitespace.
warning: 1 line adds whitespace errors.

BondD’s picture

Status: Needs work » Needs review
FileSize
2.74 KB

Seems some typo whitespace are leak to previews patch file. Fixed.

solidwebcode’s picture

patch workes like a charm :)

Skin’s picture

I confirm, #136 it is working :-)

hansfn’s picture

Thank you very much, BondD, for finding the correct fixes. However, your patch has some issues:

  1. The config values aren't saved since you missed the save() function after set()
  2. You introduced getEditable everywhere - also where the config is just read
  3. Minor coding standard issue - missing linebreak before elseif

I have attached an updated patch.

Status: Needs review » Needs work

The last submitted patch, 139: 842620-139-updatemanager-final-final.patch, failed testing.

hansfn’s picture

Attached the patch again. Now in the format expected by the testing system.

hansfn’s picture

Status: Needs work » Needs review

Forgot to switch status back to "needs review" after the test bot complained. Sorry about the noise.

BondD’s picture

1) No sure about save() function after set(). I'm try to learn Drupal by digging in code (because I'm not sole programmer and need to understand how things work from inside), and seems save() used if you need to add things to static cache. But in this form caching are not welcomed (???). If it use cache, than old code should look like:
$this->config('system.authorize')->set('filetransfer_default', $filetransfer_backend)->save(TRUE);
Certainly I can be wrong.
2) OK. I'm a little overdo it :)
3) Admit. Guilty. :)

hansfn’s picture

  1. I'm 110% sure about the save thing. It saves the username to the next time you use FTP when uploading a theme/module. Very neat. (The old code is irrelevant as it was outdated - and caused fatal errors.) I prefer reading the API and the digging around in the code.
xjm’s picture

@hitesh-jain and I also found this related issue while reproducing this:
#2672684: "+ Install new module" button doesn't appear after installing Update manager module

(So if anyone else has trouble when testing, be sure to clear the cache manually if the button does not appear).

xjm’s picture

(Updating issue credit for the major triage.)

xjm’s picture

(Updating issue credit for the major triage, and adding the relevant tag since we've confirmed the issue still exists.)

Skin’s picture

is working, thanks.

xjm’s picture

Issue tags: -beta target

Thanks all for helping triage and test this!

This issue was marked as a beta target for the 8.0.x beta, but is not applicable as an 8.1.x beta target, so untagging. This is a bug fix so still potentially eligible for patch releases.

alexpott’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

It would be good to add some test coverage if possible.

+++ b/core/lib/Drupal/Core/FileTransfer/Form/FileTransferAuthorizeForm.php
@@ -220,15 +221,18 @@ public function submitForm(array &$form, FormStateInterface $form_state) {
-          $this->config('system.authorize')->set('filetransfer_default', $filetransfer_backend);
+          $this->configFactory()->getEditable('system.authorize')->set('filetransfer_default', $filetransfer_backend)->save();
           // Save the connection settings minus the password.
-          $this->config('system.authorize')->set('filetransfer_connection_settings_' . $filetransfer_backend, $connection_settings);
+          $this->configFactory()->getEditable('system.authorize')->set('filetransfer_connection_settings_' . $filetransfer_backend, $connection_settings)->save();

This should be chained... like so...

$this->configFactory()->getEditable('system.authorize')
  ->set('filetransfer_default', $filetransfer_backend)
  ->set(filetransfer_connection_settings_' . $filetransfer_backend, $connection_settings)
  ->save();

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Skin’s picture

Hello, on the latest Drupal 8.1.0 stopped working, if I try to install a module I see an HTTP 500 ERROR.

hansfn’s picture

Status: Needs work » Needs review
FileSize
2.36 KB

Patched updated to work with Drupal 8.1.0. I also used chaining as suggested by alexpott.

PS! I find it very sad that this patch hasn't been applied yet just because it doesn't have a test. I do understand that test coverage is important, but ...

Skin’s picture

Hello, I've tried Patch #153, it works as expected.

I hope that someone is going to apply this patch to the dev release.

Thanks

sic’s picture

thanks for #153

hansfn’s picture

Status: Needs review » Reviewed & tested by the community

We now have at least 3 successful tests (including me) and alexpott's review - let's RTBC this.

FredQ9’s picture

I installed version 8.1.0 two days ago, using an Amazon EC2-HVM instance, Ubuntu, running Apache2 server. After considerable effort (e.g., resolving PHP5 extension missing error), I managed to get it up and running fine, but with a couple of lingering issues.

The main one is that in this thread, when I go to install a module using uploads to the automated install, the website simply reverts to where it started, with this in the browser line "http://[IP address]/index.php/core/authorize.php?batch=1&id=13&op=start". Per one of the earlier comments, I tried installing through http FTP route but got the same result. Obviously, Drupal 8.1.0 is not terribly useful if new modules cannot be installed. For example, just for security purposes, I need to install Captcha.

I have been running two websites with Drupal 7, and have a lot of experience with that. I realize that Drupal 8 still has some bugs to work

If someone can help, I would be happy to upload additional data.

Thanks

FredQ9’s picture

I would add that my log shows that the errors I am getting multiple "page not found" errors, and that this may be a function of pages being directed to "index.php". I have looked at different solutions for this, but in going to the .htaccess file in the drupal-8.1.0 directory, it is not clear how to specifically resolve this. There are instructions in various forums regarding changing RewriteBase, but I am reluctant to try to do that before I am sure of the proper solution.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/lib/Drupal/Core/FileTransfer/Form/FileTransferAuthorizeForm.php
    @@ -214,16 +215,21 @@ public function submitForm(array &$form, FormStateInterface $form_state) {
    -          if ($response instanceof Response) {
    +          if ($response instanceof RedirectResponse) {
    +            $response->send();
    +          }
    

    Something about this does not look right... it seems really weird to just send the redirect response during a form submission. Ah authorize.php needs to support enforced responses. This is why we need test coverage. When broke this when we introduce enforce responses into the Form API.

  2. +++ b/core/lib/Drupal/Core/FileTransfer/Form/FileTransferAuthorizeForm.php
    @@ -342,7 +348,7 @@ protected function runOperation($filetransfer) {
    -    require_once $this->root . '/' . $operation['file'];
    +    require_once $operation['file'];
    

    I don't understand why this change was introduced in #153 but without it it does not work.

alexpott’s picture

Status: Needs work » Needs review
FileSize
2.02 KB
2.7 KB
FredQ9’s picture

Fixed by reinstalling from scratch. This time was successful in enabling AllowOverride All in httpd.conf. Tricky. Thx

alexpott’s picture

So this is totally testable! (yay)

We broke this in two ways in D8 - we introduced enforced responses in FAPI and immutable config in CMI.

Discussed the system.authorize config with @catch in IRC. This config is saving the user's choices in the authorize form where they select SSH or FTP and some of the connection details - ie. user name. This has no business being in config. At best it is state but in reality we shouldn't be saving this at all - just leave for the browser to remember anything a user has entered into the fields. I tried to not make the changes to config here but I had to change the config lines because of immutable config. Therefore doing it properly in this patch. We should file a followup for 8.2.x to completely remove the unused system.authorize config as after this patch it has no meaning.

alexpott’s picture

alexpott’s picture

+++ b/core/modules/update/src/Tests/UpdateViaAuthorizeTest.php
@@ -0,0 +1,44 @@
+    $edit = [
+      'project_url' => 'https://ftp.drupal.org/files/projects/config_inspector-8.x-1.0-beta1.tar.gz',
+    ];

So this actually goes and downloads this module... going to see if and how we can use something more reliable.

alexpott’s picture

Fixing #164 and also the test where I referenced my local environment :)

The last submitted patch, 162: 842620-162.test-only.patch, failed testing.

The last submitted patch, 162: 842620-162.patch, failed testing.

The last submitted patch, 165: 842620-165.test-only.patch, failed testing.

David_Rothstein’s picture

The tests look like they get the job done, but there is a lot of overlap with the existing tests in UpdateUploadTest::testUploadModule() (which are more comprehensive).

A possibly better test would involve extending those, by which I mean:

  1. Move everything from here to the end of the function to a helper method.
  2. Make the helper method do this at the appropriate place:
      if (\Drupal::state()->get('test_uploaders_via_prompt', FALSE)) {
        $edit = [
          'connection_settings[authorize_filetransfer_default]' => 'system_test',
          'connection_settings[system_test][update_test_username]' => $this->randomMachineName(),
        ];
        $this->drupalPostForm(NULL, $edit, t('Continue'));
      }
    
  3. Have one test call the helper method with the state variable set to TRUE and one without.
  4. For this to work the tests or helper method may need to be changed to clean up after itself (i.e. manually delete the downloaded module from sites/simpletest at the end) - not positive about that.
 /**
  * Mocks a FileTransfer object to test the settings form functionality.
+ *
+ * This class extends \Drupal\Core\FileTransfer\Local to make module install
+ * testing via authorize.php possible.
+ *
+ * @see \Drupal\update\Tests\UpdateViaAuthorizeTest
  */
-class MockFileTransfer {
+class MockFileTransfer extends Local {

This is very nice. But I'm not sure MockFileTransfer is a good name anymore. Maybe it should be renamed to something like TestFileTransferWithSettingsForm?

David_Rothstein’s picture

By the way, this looks like it is fixing a real Drupal 8 bug, but I don't think it has anything to do with the problems this issue was originally opened for (except that both affect the FTP form). As I recall from #110 the earlier ones were about file ownership issues...

This does need to get fixed first, but not sure what we should do with this issue after it's committed - since the original problems are probably still there.

alexpott’s picture

@David_Rothstein I intentionally kept it out of UpdateUploadTest::testUploadModule() to do the most minimal test possible. I think it fine that the UpdateUploadTest::testUploadModule() has more comprehensive testing of what it is doing. This new test adds coverage of things it does not test. @David_Rothstein I'd rather open a followup to address points 1 to 3 since I think a big refactor of UpdateUploadTest and testing what happens when the file owner and permission means you don't have to use FTP or SSH is out of scope here.

Patch attached fixes point 4.

Re #170 yep there are definitely problems with issue scope here. The patch attached does not solve the original concerns of the issue but from #116 it has become about the bugs in Drupal 8. Totally unsure of what to do here now.

David_Rothstein’s picture

I guess a followup issue is OK - I was suggesting it here though because I think it would be a small refactor (with fewer changes than adding a whole new test) that would test the FTP/SSH scenario more comprehensively as a side benefit.

The interdiff looks good. I can't say I understand the overall patch well enough to RTBC it...

alexpott’s picture

Priority: Major » Critical
Issue summary: View changes
Issue tags: -needs backport to D7

After discussing how to manage this issue with @xjm, I've updated issue summary to reflect the bug that is being fixed. Opened #2717951: Update manager sometimes can't install modules using FTP due to file ownership issues to handle the bug @bojanz opened all those years ago.

I've promoted this to a critical bug because installing modules is one of the first tasks new users want to do. Whilst experienced Drupal user or web developers are unlikely to use this method to install modules other types of users are and it is completely broken. (Note that I did not discuss promoting this to critical with @xjm)

xjm’s picture

Issue tags: +Triaged D8 critical

This issue was critical before release and regressed. @alexpott, @catch, @effulgentsia and I agreed with it being critical again now.

Thanks to @hitesh-jain for helping triage this!

hansfn’s picture

Status: Needs review » Reviewed & tested by the community

I have tested/reviewed the installation code - it works. The fix seems better/more general than one I took part in creating. I agree that saving the (FTP) connection username to config was wrong. It was nice to have it prefilled, but of course the browser auto-completes, when you start typing. Looking forward to getting this fixed.

I have not reviewed the test.

And, yes, this is critical. It almost killed my Drupal class this spring since the students weren't able to install modules. Luckily, they had a clever teacher - me :-)

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 171: 842620-171.patch, failed testing.

hansfn’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
11.71 KB

Updated patch (adding a blank line) so it applies.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 177: 842620-177.patch, failed testing.

alexpott’s picture

Status: Needs work » Reviewed & tested by the community

  • catch committed a9666b4 on 8.2.x
    Issue #842620 by alexpott, hansfn, BondD, jurjenn, Skin, xjm, hitesh-...

  • catch committed 00c3ef3 on 8.1.x
    Issue #842620 by alexpott, hansfn, BondD, jurjenn, Skin, xjm, hitesh-...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.2.x and cherry-picked to 8.1.x. Thanks!

xjm’s picture

(Removing major triage tag since it got promoted to critical.) :)

cavalcami’s picture

thanks, I resolve with your help.

I have comment Default root string in my
i /etc/proftpd/proftpd.conf config file
restart ftp server and all done.

Ps. I have Raspberry pi-3 with DietPI
and my ftp server is : proftp

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.