Improve logic for restoring backups

I can confirm the exact same issue.

Also for the encryption I get prompted for a passcode? I see no obvious documentation on this. We need to add documentation if we are going to encourage use of encryption for future use.

I will have a word with Damien next week Paul.

My private files directory is not set outside of web root:

$settings['file_private_path'] = 'sites/default/files/private';

in settings.php

And I am experiencing the same issue on restore attempt still.

Red colored message is displayed inside admin message area:
"The file could not be imported."

The strangest thing is that there is no any log error message recorded in drupal dblog or at server.

Here is part of code displaying the error message:

src/Core/Main/BackupMigrate.php at line 150:

     // Do the actual source restore.
      $import_result = $source->importFromFile($file);
      if (!$import_result) {
        throw new BackupMigrateException('The file could not be imported.');
      }

I was able to var_dump() $file object passed to importFromFile() method and here is the var_dump($file) output:

object(Drupal\backup_migrate\Core\File\ReadableStreamBackupFile)#576(6) {
  ["handle":protected]=>NULL
  ["fileInfo":protected]=>NULL
  ["path":protected]=>string(32) "private://backup_migrate/manual/"
  ["name":protected]=>string(6) "manual"
  ["ext":protected]=>array(0) {}
  ["metadata":protected]=>array(2) {
    ["filesize"]=>int(4096)
    ["datestamp"]=>int(1599693367)
  }
}

Maybe someone will figure out from this output why $source->importFromFile($file); failed.

This is out of my knowledge scope. :)

My config: PHP 7.3, BAM v5.0.x-dev, CLOUDLINUX 7.8

With latest RC2 today the message at shared server online is the same "The file could not be imported." without any log error message.

Locally at my wamp server the message is: "Cannot open file".

And there is an error log message:

Warning: fopen(private://backup_migrate/): failed to open stream: "Drupal\Core\StreamWrapper\PrivateStream::stream_open" call failed in Drupal\backup_migrate\Core\File\ReadableStreamBackupFile->openForRead() (line 88 of D:\wamp\www\mysite\modules\contrib\backup_migrate\src\Core\File\ReadableStreamBackupFile.php)

But this could be related with my local wamp settings... or not...

Maybe it would be good to add this issue to RC2 release notes "Known issues" list together with current:

Known issues
#3154716: Unable to delete backups

Temporary workaround for this issue is to download backup file and upload it at "Restore" tab (admin/config/development/backup_migrate/restore).

No error messages in this case and the backup is successfully restored.

When I go to the Saved Backups page at /admin/config/development/backup_migrate/backups I get a WSOD that says "The website encountered an unexpected error. Please try again later." Logs report:

Exception: Cannot open file. in Drupal\backup_migrate\Core\File\ReadableStreamBackupFile->openForRead() (line 90 of /PATH/TO/docroot/modules/contrib/backup_migrate/src/Core/File/ReadableStreamBackupFile.php).

And...

Warning: fopen(private://backup_migrate/backup-2019-11-28T00-05-23.mysql.gz.info): failed to open stream: "Drupal\Core\StreamWrapper\PrivateStream::stream_open" call failed in Drupal\backup_migrate\Core\File\ReadableStreamBackupFile->openForRead() (line 88 of /PATH/TO/docroot/modules/contrib/backup_migrate/src/Core/File/ReadableStreamBackupFile.php)
#0 /PATH/TO/docroot/core/includes/bootstrap.inc(301): _drupal_error_handler_real(2, 'fopen(private:/...', '/mnt/www/html/n...', 88, Array)
#1 [internal function]: _drupal_error_handler(2, 'fopen(private:/...', '/mnt/www/html/n...', 88, Array)
#2 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Core/File/ReadableStreamBackupFile.php(88): fopen('private://backu...', 'r')
#3 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Core/File/ReadableStreamBackupFile.php(160): Drupal\backup_migrate\Core\File\ReadableStreamBackupFile->openForRead()
#4 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Core/Destination/SidecarMetadataDestinationTrait.php(31): Drupal\backup_migrate\Core\File\ReadableStreamBackupFile->readAll()
#5 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Core/Destination/DestinationBase.php(41): Drupal\backup_migrate\Core\Destination\DirectoryDestination->loadFileMetadataArray(Object(Drupal\backup_migrate\Core\File\ReadableStreamBackupFile))
#6 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Drupal/Destination/DrupalDirectoryDestination.php(99): Drupal\backup_migrate\Core\Destination\DestinationBase->loadFileMetadata(Object(Drupal\backup_migrate\Core\File\ReadableStreamBackupFile))
#7 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Controller/BackupController.php(119): Drupal\backup_migrate\Drupal\Destination\DrupalDirectoryDestination->queryFiles(Array, 'datestamp', 3, 5)
#8 /PATH/TO/docroot/modules/contrib/backup_migrate/src/Controller/BackupController.php(41): Drupal\backup_migrate\Controller\BackupController->listDestinationBackups(Object(Drupal\backup_migrate\Drupal\Destination\DrupalDirectoryDestination), 'private_files', 5)
#9 [internal function]: Drupal\backup_migrate\Controller\BackupController->listAll()
#10 /PATH/TO/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)
#11 /PATH/TO/docroot/core/lib/Drupal/Core/Render/Renderer.php(573): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#12 /PATH/TO/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#13 /PATH/TO/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)
#14 /PATH/TO/docroot/vendor/symfony/http-kernel/HttpKernel.php(158): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#15 /PATH/TO/docroot/vendor/symfony/http-kernel/HttpKernel.php(80): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#16 /PATH/TO/docroot/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#17 /PATH/TO/docroot/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#18 /PATH/TO/docroot/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#19 /PATH/TO/docroot/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#20 /PATH/TO/docroot/core/modules/ban/src/BanMiddleware.php(50): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#21 /PATH/TO/docroot/modules/contrib/shield/src/ShieldMiddleware.php(115): Drupal\ban\BanMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#22 /PATH/TO/docroot/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\shield\ShieldMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#23 /PATH/TO/docroot/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#24 /PATH/TO/docroot/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#25 /PATH/TO/docroot/core/lib/Drupal/Core/DrupalKernel.php(705): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#26 /PATH/TO/docroot/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#27 {main}

Is this the same issue that others are having here?

Re: #10

Is this the same issue that others are having here?

Yes. The same error message.

There is a temporary workaround. You can download the file and restore it back by uploading it using the "Restore" tab.

Thanks for the reply. That's assuming one has access to those saved backups. In my case, the screen crashes. So, one has to use the command line to access those files. This seems like a problem that needs to be fixed.

This may be a related issue: https://www.drupal.org/project/backup_migrate/issues/3169587

I am experiencing this as well.

I have a different report in logs than #10. Mine says:

Notice: fread(): read of 8192 bytes failed with errno=21 Is a directory in Drupal\Core\StreamWrapper\LocalStream->stream_read() (line 211 of /var/www/titan8.roztr.org/web/core/lib/Drupal/Core/StreamWrapper/LocalStream.php)
 
#0 /var/www/example.com/web/core/includes/bootstrap.inc(600): _drupal_error_handler_real()
#1 [internal function]: _drupal_error_handler()
#2 /var/www/example.com/web/core/lib/Drupal/Core/StreamWrapper/LocalStream.php(211): fread()
#3 [internal function]: Drupal\Core\StreamWrapper\LocalStream->stream_read()
#4 /var/www/example.com/web/modules/contrib/backup_migrate/src/Core/File/ReadableStreamBackupFile.php(149): fgets()
#5 /var/www/example.com/web/modules/contrib/backup_migrate/src/Core/Source/MySQLiSource.php(279): Drupal\backup_migrate\Core\File\ReadableStreamBackupFile->readLine()
#6 /var/www/example.com/web/modules/contrib/backup_migrate/src/Drupal/Source/DrupalMySQLiSource.php(26): Drupal\backup_migrate\Core\Source\MySQLiSource->_readSqlCommand()
#7 /var/www/example.com/web/modules/contrib/backup_migrate/src/Core/Main/BackupMigrate.php(151): Drupal\backup_migrate\Drupal\Source\DrupalMySQLiSource->importFromFile()
#8 /var/www/example.com/web/modules/contrib/backup_migrate/backup_migrate.module(68): Drupal\backup_migrate\Core\Main\BackupMigrate->restore()
#9 /var/www/example.com/web/modules/contrib/backup_migrate/src/Form/BackupRestoreForm.php(93): backup_migrate_perform_restore()
#10 [internal function]: Drupal\backup_migrate\Form\BackupRestoreForm->submitForm()

This happens both with the latest development version, and the tagged release 5.0.0-rc2.

Is it not an permission issue with the private file system, as version 8.x-4.1 works as one would expect with the same private file system as source.

My environment:
- Drupal: 8.9.10
- Ubuntu: 20.04 LTS
- PHP: 7.4.3, 128M memory limit.
- MySQL: 8.0.22-0ubuntu0.20.04.2

I just submitted a PR (first time using the feature!) to address both this issue and #3154716: Unable to delete backups.

The reason why download, restore and delete were failing was because the path contained a file name, so Drupal interpreted that as a direct path to a file and was trying to serve it, but the file doesn't exists on that location. I'm replacing the "." in the path for "-DOT-" so the path can be evaluated by Drupal's routing mechanism and therefore backups can be deleted, restored and downloaded again.

There was also a typo on a parametre name for the restore and delete confirmation pages which I've also amended.

I hope it all helps.

Nice fix, thank you.

Let's add some test coverage to make sure the delete function works correctly.

Am adding test coverage in #3154716.

I've tested the merge request introduced in comment #19, and can confirm that it works with 5.0.x-dev. Thanks!

Adding tests would be nice, but since RC2 of this project is in effect broken, I humbly request that this is committed, and that the addition of tests for this is tracked in another issue.

Could we possibly get an updated Dev version out there?

The bare fixes were committed elsewhere, so let's work on trimming this back to focus on improving the logic.

This doesn't need to be in the 5.0.0 release.

The changes need to be improved upon, hence the status "needs work".

Could you please edit the issue summary to highlight what the proposed changes are (that can be improved upon).

What I see in the issue summary is no "logic", it is just a bug report that I believe no longer can be reproduced if you run the latest dev snapshot. I think the PR in #19 fixed it, but it may also been fixed elsewhere. re. # 24. No changes are proposed in the issue summary, and it is hard to figure out if there are changes to the logic proposed in any of the many comments.

I love this module, and want to help out, but a little clarity would be nice.

I'm also encountering this on trying to import/upload manually through the UI (as suggested in #7).

I've used both 5.0.1 and 5.0.x-dev on DDEV (WSL2) and have updated php.ini values to be well above the export size (compressed backup is <12MB and have limits at 200MB). Error logs only show warnings for using a temporary file: "a client request body is buffered to a temporary file."

Edit: Issue was that my local environment was setup to use mariadb, not mysql - changing ddev settings resolved the issue!

#28 What values have you changed on your php.ini ?