I have an absurdly large database. I am using Backup and Migrate via Drush to restore it. I receive timeout errors as follows:

Are you sure you want to perform the restore? (y/n): y
Could not perform the restore because the script timed out. Try      [error]
increasing your PHP max_execution_time setting.
WD backup_migrate: Could not perform the restore because the script  [error]
timed out. Try increasing your PHP max_execution_time setting.
The operation timed out. Try increasing your PHP max_execution_time  [error]
setting.
WD backup_migrate: The operation timed out. Try increasing your PHP  [error]
max_execution_time setting.
session_encode(): Cannot encode non-existent session                 [warning]
backup_migrate.module:1647
Cannot modify header information - headers already sent by (output   [warning]
started at
/home/webmaster/.composer/vendor/drush/drush/includes/output.inc:38)
common.inc:698

Setting max_execution_time to 0 in my php.ini (in /etc/php5/cli/) has no effect; the error message remains the same.

CommentFileSizeAuthor
#4 backup_migrate-prevent_overwriting_unlimited_max_execution_time-7.x-3.x-2569707-4.patch1.76 KBhgoto

Comments

duncan246 created an issue. See original summary.

duncan246’s picture

duncan246 commented

I discovered that max_execution_time was being overwritten by the following code, which appears in the functions backup_migrate_perform_restore and backup_migrate_perform_backup:

  // If not in 'safe mode', increase the maximum execution time:
  if (!ini_get('safe_mode') && strpos(ini_get('disable_functions'), 'set_time_limit') === FALSE && ini_get('max_execution_time') < 1200) {
    set_time_limit(variable_get('backup_migrate_backup_max_time', 1200));
  }

The workaround was to add the value set_time_limit to the directive disable_functions in the php.ini. It took two hours but, after this, the database restored successfully. (Remember that the php.ini file to be changed may vary according to whether php is being called via the browser or via the command line.)

claudio mateluna’s picture

claudio mateluna commented

Hi, any chance that you can explain a step by step for the solution you find? It will be much appreciated

hgoto’s picture

hgoto commented
Version: 7.x-3.1 » 7.x-3.x-dev
Assigned: duncan246 » Unassigned
Category: Support request » Bug report
Status: Active » Needs review
StatusFileSize
new backup_migrate-prevent_overwriting_unlimited_max_execution_time-7.x-3.x-2569707-4.patch1.76 KB

@duncan246, thank you for sharing this issue.

Here is a patch for the version 7.x-3.x to avoid overwriting unlimited max time (0). Since Backup and Migrate for Drupal 8 doesn't use this logic, the version 7.x-3.x is the latest one we can create a patch, I believe.

I hope someone would test the patch with Backup and Migrate version 7.x-3.x. Thank you in advance.

couturier’s picture

couturier commented
Status: Needs review » Needs work

This patch will need to be reviewed based on the latest 7.x-3.2 release.

junro’s picture

junro commented

Hello,

I have exactly the same problem.

I can't restore my database.

Could not perform the restore because the script timed out. Try increasing your PHP max_execution_time setting.
The operation timed out. Try increasing your PHP max_execution_time setting.

I increase max_execution_time = 3600 & max_input_time = 3600.

I have the error message 20/25 minutes after the restoration begin.

So I guest it's not a problem with max_execution_time...

With Drupal 6.x, I didn't have this problem.

My website error_log file :

[Fri Jan 05 17:52:01 2018] [error] [client xxx] Script timed out before returning headers: index.php, referer: ww.xxx.com/admin/config/system/backup_migrate/settings/destination/list/restorefile/manual/xxx-2018-01-05T14-48-26.mysql.gz

I tried with backup & migrate 7.3.3 & 7.3.dev

junro’s picture

junro commented

Patch works , but not with the 3.3 version, only with the dev version.

AND I can't backup anymore, I have to delete the code patch...

couturier’s picture

couturier commented

Sounds like this patch is going to have to be rewritten.

junro’s picture

junro commented

HTTP ERROR 500 after 3.5 upgrade.

couturier’s picture

couturier commented

This isn't a solution, just an observation: with the release of Drupal 8, most of the larger websites using Drupal are beginning to move toward alternate backup options rather than the Backup and Migrate module which was so popular for D7 and previous versions. If someone's database is so large that it is timing out with a Backup and Migrate backup, perhaps they would have alternate command line options for backup that would better suit the needs of the database?

Quite a few of us who are intensely interested in seeing Backup and Migrate move forward for D8 and beyond don't have command line access and/or use a shared hosting environment where a module for backup is imperative. With development resources limited for this module, I suggest we work on this bug only if it also applies to the D8 version, since D7 will eventually become unsupported.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 4: backup_migrate-prevent_overwriting_unlimited_max_execution_time-7.x-3.x-2569707-4.patch, failed testing. View results

HaloFX’s picture

HaloFX commented

From reading #2, it sounds like this effects backup and restore functions, am I correct?
My apologizes if my situation is not related.

I have a D7 site currently using BaM 3.5. I am backing up the database, and the files in separate tasks to S3. The database is about 5mb and backups fine in a matter of seconds. The files are just over 50gb. They backup and transfer to S3 in about 30 minutes. I get 2 messages from the task, one stating it completed, and a few seconds later a message stating the backup failed due to a PHP max_execution_time error. PHP max_execution_time is currently set to 3600 in /etc/php/7.0/apache2/php.ini.

couturier’s picture

couturier commented

@HaloFX Do I understand correctly that you are working off of an Amazon system? It could be something related to that if so. The Backup and Migrate issues queue has several issues related to problems with that. If it is a simple timeout error then this patch might help if we can get it working. Note that we do also have several other issues related to timeout errors, so you might want to check the queue to see if anything else might provide some help.

HaloFX’s picture

HaloFX commented

In my case, the site is on Digital Ocean, and backups to S3.
I also realized I didn't make it clear that in most cases, the backup appears to be OK, but still reports a timeout error.

couturier’s picture

couturier commented
Related issues: +#2751899: Support for the AWS S3 encryption at rest, +#1714696: Make use of amazons3 file stream wrapper and official php sdk, +#2404113: Different host for AWS type server

It is likely a timeout issue as your error states, but I am concerned that if you're backing up to S3, you could be having additional issues. See these other discussions as linked to rule them out. DigitalOcean coordinates with Amazon, and we've seen problems with the Backup and Migrate module version 7.x working well with that.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented

Please test out the latest 7.x-3.x codebase, see if the recent changes help any.

As mentioned above, it's also worth considering just using native database tools to restore the backup, they don't run through the web server and PHP so it should be less likely to trigger timeouts.

brankoc’s picture

brankoc commented
Related issues: +#2374567: Timed out error, but backup successful backup, +#1381836: Warning: session_encode(): Cannot encode non-existent session in backup_migrate_shutdown() (line 861 of /var/www/html/sites/all/modules/contrib/backup_migrate/backup_migrate.module).

The issue from comment #15 by HaloFX seems more closely related to "Timed out error, but backup successful backup".

The warnings at the end of the message from the issue starter, "Cannot modify header information - headers already sent by [...]" are unrelated to the main issue and have to do with the B&M script being called from cron - an anonymous user apparently does not establish a session. This error message is discussed in "Warning: session_encode(): Cannot encode non-existent session in backup_migrate_shutdown()" (which has a patch, but hasn't been committed at the time of writing).

brankoc’s picture

brankoc commented

I tried to reroll the patch, only to find that similar code was already present in B&M 7.x-3.6.

It appears that when issue 2829492, Big file backup breaks with "MySQL server has gone away", tried to solve a different problem, that solution also changed the one thing that the patch of the current issue changes, namely turning set_time_limit() into drupal_set_time_limit().

The current patch adds a comment "// drupal_set_time_limit() doesn't overwrite limit 0 (unlimited) since Drupal 7.40" to explain why drupal_set_time_limit() should be used, but I am not sure if the lack of that comment in the current stable module should count as a failing.

I suggest this issue be closed.

ivnish’s picture

ivnish
he/him
commented
Status: Needs work » Closed (outdated)

Drupal 7 is EOL. This issue will be closed.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.