CommentFileSizeAuthor
#53 backup_migrate-n2946311-53.patch1.16 KBdamienmckenna
#37 backup_migrate-n2946311-37.patch610 bytesdamienmckenna

Comments

k.elizabeth created an issue. See original summary.

k.elizabeth’s picture

k.elizabeth commented

I wanted to clarify that this is happening for some, but not all, sites that we manage that backup both Drupal and CiviCRM databases. Which makes this especially confusing.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Category: Bug report » Support request
Priority: Major » Normal

You might try out the setting that lets you use the mysqldump command directly, see if that helps.

k.elizabeth’s picture

k.elizabeth commented

I'm not seeing that option anywhere in 7.x-3.3, could you direct me to it?

damienmckenna’s picture

damienmckenna
Location TN, USA
commented

The option is on the profile edit form, it's under Advanced Options, "Use cli commands".

Also, make sure you're using 3.5.

couturier’s picture

couturier commented
Related issues: +#2947510: PDOException: SQLSTATE[08004] [1040] Too many connections
k.elizabeth’s picture

k.elizabeth commented

The option is on the profile edit form, it's under Advanced Options, "Use cli commands".

Also, make sure you're using 3.5.

I am using 3.5, and have selected 'Use cli commands'. When I run cron, I still get the 'Too many connections' message. 

$ drush cron -v
Executing: mysql --defaults-extra-file=/tmp/drush_cCcKIa --database=website_drupal --host=localhost --silent  < /tmp/drush_Vaxio8
Executing: mysql --defaults-extra-file=/tmp/drush_wRsGGf --database=website_drupal --host=localhost --silent  < /tmp/drush_9qmHcd
WD cron: PDOException: SQLSTATE[08004] [1040] Too many connections in backup_migrate_source_db->_get_db_connection() (line 220 of                                                [error]
/var/www/vhosts/website.org/htdocs/sites/all/modules/backup_migrate/includes/sources.db.inc).
WD cron: Cron run completed.                                                                                                                                                        [notice]
Cron run successful.

As that runs, I check the connections. They go up and up until it hits the limit (I have max_connections at 500 for testing this) then the job errors out as you see above.

$ mysqladmin -u -p extended-status | grep -wi 'threads_connected\|threads_running' | awk '{ print $2,$4}'
Threads_connected 486
Threads_running 3

After that cron job fails, the connections drop back down:

$ mysqladmin -u -p extended-status | grep -wi 'threads_connected\|threads_running' | awk '{ print $2,$4}'
Threads_connected 1
Threads_running 1

Ran it again, this time looking at the actual processes. Most of them are sleeping. (Too big to post here, would put it in a pastebin, but do not want to share client info). Here is a sample:

mysql> show processlist;
+-------+------+-----------+--------------+---------+------+-------+------------------+
| Id    | User | Host      | db           | Command | Time | State | Info             |
+-------+------+-----------+--------------+---------+------+-------+------------------+
| 46825 | root | localhost | NULL         | Query   |    0 | NULL  | show processlist |
| 46828 | root | localhost | website_drupal  | Sleep   |    1 |       | NULL             |
| 46829 | root | localhost | website_civicrm | Sleep   |  198 |       | NULL             |
| 46830 | root | localhost | website_civicrm | Sleep   |  198 |       | NULL             |

I try checking a different site we manage, on a different server. bam has been updated to 3.5. Backups are (thankfully) running, but max_connections is set at 350. When I attempt to access any of the bam settings pages, I get:

$ mysqladmin -uroot -p extended-status | grep -wi 'threads_connected\|threads_running' | awk '{ print $2,$4}'                                                      
Threads_connected 346
Threads_running 3

And then it fails once it hits 350 (the max_connections setting). This is when I attempt to navigate to the main settings tab, saved backups tab, and schedules tab.

We're concerned that something was introduced in the 3.4 update that is causing this, These two sites I am working with are samples. We are having the same trouble across all of our sites (~20+) that use bam for scheduling backups for both drupal and civi.

k.elizabeth’s picture

k.elizabeth commented
Priority: Normal » Major
k.elizabeth’s picture

k.elizabeth commented
Priority: Major » Normal
k.elizabeth’s picture

k.elizabeth commented

Just an update to share what I get when I roll backup_migrate back to 3.3 and run cron with drush (all the backups were backlogged because cron was unable to successfully run them when using 3.5):

$ drush cron -v
Executing: mysql --defaults-extra-file=/tmp/drush_yDBmgH --database=website_drupal --host=localhost --silent  < /tmp/drush_cuBNMU
Executing: mysql --defaults-extra-file=/tmp/drush_olBBmg --database=website_drupal --host=localhost --silent  < /tmp/drush_3nLrOt
WD backup_migrate: CiviCRM Database backed up successfully to website-civi-thrice-daily-2018-03-16T13-25-33 (6.09 MB) in destination Civi Thrice Daily in 22 sec.                      [notice]
WD backup_migrate: CiviCRM Database backed up successfully to website-civi-weekly-2018-03-16T13-26-21 (6.09 MB) in destination Civi Weekly in 35 sec.                                  [notice]
WD backup_migrate: Default Database backed up successfully to website-drupal-daily-2018-03-16T13-27-00 (6.86 MB) in destination Drupal Daily in 1 min.                                 [notice]
WD backup_migrate: Default Database backed up successfully to website-drupal-thrice-daily-2018-03-16T13-27-48 (6.86 MB) in destination Drupal Thrice Daily in 1 min 25 sec.            [notice]
WD backup_migrate: Default Database backed up successfully to website-drupal-weekly-2018-03-16T13-28-36 (6.86 MB) in destination Drupal Weekly in 1 min 49 sec.                        [notice]
WD backup_migrate: Default Database backed up successfully to pt_sqlsync-website-2018-03-16T13-30-25 (4.68 MB) in destination PT SQL Sync in 2 min 9 sec.                  [notice]
WD backup_migrate: CiviCRM Database backed up successfully to pt_sqlsync-website-civi-2018-03-16T13-31-04 (6.09 MB) in destination PT SQL Sync in 2 min 22 sec.            [notice]
WD cron: Cron run completed.                                                                                                                                                        [notice]
Cron run successful.                                                                                                                                                             [success]
Command dispatch complete                                                                                                                                                           [notice]

All of these backups ran without connections exceeding 150. This used precisely the same backup settings, as I have them saved as a feature.

Also, all of the settings pages load relatively quickly (rather than timing out as they do with 3.5).

couturier’s picture

couturier commented

@DamienMcKenna, correct me if I'm wrong, but I think 7.x-3.4 was the first release after the long gap of time when Ronan quit working on the module and Alex Andrascu and team had taken over. This could make the problem's origin hard to trace if it was something Ronan may have worked on in the 3.4 dev that wasn't necessarily documented in the issues queue. This wasn't a problem in 3.3 but emerged in 3.4, and oddly not on all sites using CiviCRM but only some of them.

@k.elizabeth, is there any other common factor among the sites with this issue that you could isolate?

k.elizabeth’s picture

k.elizabeth commented

I can confirm after leaving 3.5 on a few drupal/civi sites for some time, that though backups have continued successfully completed in some of these cases, in all cases we are seeing an overload to mysql connections whenever backup migrate runs. We are now having to roll back all of our sites to 3.3 to alleviate the issue.

The only common factor for this problem is that we are running backups for both Drupal and Civi. Another common factor may be the volume of backups: thrice daily civi, thrice daily drupal, daily civi, daily drupal, weekly drupal, weekly civi. Of course these do not all run at once, they run at different times in the day. All sites are using Drupal cron.

All of our sites that only run Drupal backups are handling fine, though we have not tested if there is a significant increase in connections for those sites as it has not been a top priority for us to date.

couturier’s picture

couturier commented

Thanks for the additional information @k.elizabeth. I searched the issues queue for anything filed against 7.x-3.3 specifically that might have been changed with the upgrade to 3.4 to induce a "too many connections" error, and nothing is relevant.

However, there is an interesting comment by Ronan in the following thread that indicates he was working on something related to multiple sources for backups in the 3.x branch. This makes me wonder if he changed something in the 3.x dev after 3.3 was released that would have affected the connections when 3.4 was finally put out by a different team of programmers: https://www.drupal.org/project/backup_migrate/issues/1305716

If we can't find a historical reference for something that changed between the 3.3 and 3.4 versions to have caused your error, then we'll have to tackle this problem fresh. I'm also curious if you are planning to move your sites to Drupal 8 at any point in the near future. 7.x-3.3 might work well enough for you to continue until your upgrades to D8 if this ends up being a time-consuming and mysterious problem to fix. Thanks!

k.elizabeth’s picture

k.elizabeth commented

Thanks for the follow-up, and looking forward to hear what you find out.

We have no present intention to migrate any of these sites to Drupal 8. As of now, our plan is to migrate all existing Drupal 7 sites to Backdrop over time. We only have two sites in Backdrop presently, and both use the ported backup_migrate module.

couturier’s picture

couturier commented

That's interesting you're moving to Backdrop. So, I'm assuming the ported backup_migrate for Backdrop the home page for which I just viewed here is now on its own maintenance track including new version numbers and wouldn't benefit from any fix we might find for the 7.x-3.x Drupal version. Or, do module upgrades and fixes get re-ported to Backdrop when upgrades occur for the Drupal version?

My recommendation for now would be to continue using 7.x-3.3 for your sites until they are all moved to Backdrop. The reason for that is due to extremely limited resources for the Backup and Migrate module right now and also due to lack of any prior documentation as to what may have introduced this error starting with 7.x-3.4. Most of the development effort for this module is being focused on getting a stable Drupal 8 release. However, let's leave this issue open for now and see if we have anyone else reporting the same error. I'm not sure if something in your local environment could be contributing, so if we can find other users duplicating the error, that would help.

k.elizabeth’s picture

k.elizabeth commented

I will note that this is happening on array of VPS and shared hosting environments, not in a single environment, and not all of which we provide systems administration for.

k.elizabeth’s picture

k.elizabeth commented
Issue summary: View changes
couturier’s picture

couturier commented

Thanks for the additional information. It must be something in the Backup and Migrate module that changed from 7.x-3.3 to 3.4 that is the problem, then.

couturier’s picture

couturier commented

Correction to previous comments: the official releases page shows that 7.x-3.3 was released on October 18, 2017, and was the second release for 7.x under Alex Andrascu after Ronan took leave from the project. 7.x-3.4 was released on January 24, 2018. The list of commits between those two dates include numerous fixes, but one in particular stands out to me that might have caused the "too many connections error" you experienced starting with 3.4, as follows:

Issue #2742855 by hanoii: Caching profile objects led to unwanted behaviors because of stale data.

The reason this patch might be relevant is that the reporter had multiple scheduled backups from multiple sources. I don't have the experience to be able to look into the code to see if anything happened with this fix to cause the new error, but if someone could help out with that, I think it is at least a starting point. If this commit checks out okay, then I would recommend going back to the full commit list mentioned above and looking at the other issues between the two release dates to see if anything else might have changed.

@k.elizabeth, I'm still curious if ongoing fixes within Backup and Migrate 7.x on Drupal are moved over to Backdrop or if the two projects have completely diverged at this point. Thanks.

k.elizabeth’s picture

k.elizabeth commented

Thanks for the follow up, this is all useful information. We ourselves have multiple scheduled backups from multiple sources and multiple destinations, and do not experience the issue outlined in issue #2742855 when we are using 7.x-3.3, for what it's worth.

Our solution for now has been to roll all of our sites back to 7.x-3.3, check that the security issue fixed in 7.x-3.4 does not apply to our sites, and set the .info file to claim 7.x-3.5 in order to keep our monitoring system (and our clients) from being alerted to the out-of-date module. Thank goodness for features allowing us to save all of our backup settings! This is a band-aid for us, but we would love to see a 7.x-3.6 release that rolls back whatever commit is causing this issue.

As for Backdrop, the backup_migrate module is being maintained by Jason Ramsey, and as far as I can tell from the release notes, it has diverged from the D7 backup_migrate module.

couturier’s picture

couturier commented

Thanks for the info on the Backup and Migrate module diverging for Backdrop. So, fixing this current here isn't going to affect that code, then.

Per your suggestion, I wonder if someone could manually roll back the #2742855 commit and see if it solves the problem. I understand that you aren't having the same issue, but it is the only commit i see within the time frame we've narrowed down that has anything even remotely related to your current issue as far as the elements of code that were being addressed.

As I also mentioned in Comment #19, there is a pretty narrow list of commits between the two dates that are the timeframe for this error having been created. if rolling back #2742855 doesn't work, then it's not beyond possible to test the other commits individually.

Cyberflyer’s picture

Cyberflyer commented

Some data for the maintainers:

I'm running Drupal 7.58, Commons 7.x-3.47 and CiviCRM 7.x-4.7.31 on my client's site.
Hosting is running Apache 2.4.29, PHP 7.0.28, MySQL 10.1.32-MariaDB on linux kernal 2.6.32-896.16.1.lve1.4.49.el6.x86_64.

The Status page is a green board. Everything is reporting operating and latest revisions, etc.

I installed Backup and Migrate 7.x-3.5.

I created Sources for my Drupal DB, CiviCRM DB and Site Root Files Folder.
I set up destinations for Node Squirrel, and manual and scheduled server locations at ../drupal-backup/manual and ../drupal-backup/scheduled, respectively.
I created a Settings for my site, storing structure only for cache tables.

When I go to the B&M main admin page, I get "Quick Backup" and it works.

When I go to the Node Squirrel tab, it comes up and tells me the status of NS.

NodeSquirrel Status
Ready to Backup (backup now)
Current Plan
Free Tier
Number of Stored Backups
3
Last Backup
Wed, 04/04/2018 - 12:34 (22 hours ago)
Storage Space
2.17 MB used of 5 GB (5 GB remaining)

When I go to Schedules tab, it shows me one NodeSquirrel schedule, which is disabled.

However, when I go to the Settings tab I get a simple page titled "Error" with the helpful note that:

The website encountered an unexpected error. Please try again later.

Makes me suspect permissions, but they seem proper.

Looking at recent Log messages, I see this:

Type php
Date Thursday, April 5, 2018 - 11:00
User Me
Location https://my.domain/admin/config/system/backup_migrate/settings
Referrer https://my.domain/admin/config/system/backup_migrate/schedule
Message PDOException: SQLSTATE[HY000] [1203] User CiviCRM-DB-User already has more than 'max_user_connections' active connections in backup_migrate_source_db->_get_db_connection() (line 220 of public_html/sites/all/modules/backup_migrate/includes/sources.db.inc).
Severity error
Hostname 68.108.215.39
Operations

I ran a cron and it produced this message:

Initialization Error

Array
(
   [callback] => Array
       (
           [0] => CRM_Core_Error
           [1] => simpleHandler
       )

   [code] => -24
   [message] => DB Error: connect failed
   [mode] => 16
   [debug_info] =>  [nativecode=User CiviCRM-DB-User already has more than 'max_user_connections' active connections]
   [type] => DB_Error
   [user_info] =>  [nativecode=CiviCRM-DB-User already has more than 'max_user_connections' active connections]
   [to_string] => [db_error: message="DB Error: connect failed" code=-24 mode=callback callback=CRM_Core_Error::simpleHandler prefix="" info=" [nativecode=User CiviCRM-DB-User already has more than 'max_user_connections' active connections]"]
)

Initialization Error

Array
(
   [callback] => Array
       (
           [0] => CRM_Core_Error
           [1] => simpleHandler
       )

   [code] => -24
   [message] => DB Error: connect failed
   [mode] => 16
   [debug_info] =>  [nativecode=User CiviCRM-DB-User already has more than 'max_user_connections' active connections]
   [type] => DB_Error
   [user_info] =>  [nativecode=User CiviCRM-DB-User already has more than 'max_user_connections' active connections]
   [to_string] => [db_error: message="DB Error: connect failed" code=-24 mode=callback callback=CRM_Core_Error::simpleHandler prefix="" info=" [nativecode=User CiviCRM-DB-User already has more than 'max_user_connections' active connections]"]
)

I'm hoping this is useful in debugging this issue.

Let me know if there is something I should try....

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Title: Too many connections in backup_migrate_source_db->_get_db_connection() » Too many connections in backup_migrate_source_db->_get_db_connection() (with CiviCRM)

I wonder if the fact that you have CiviCRM connecting to it too is compounding the problem? I really don't know..

couturier’s picture

couturier commented

@Cyberflyer was everything working well for you with the 7.x-3.3 release? As per comments above, there was something starting with the 7.x-3.4 release that caused the max connections issue, and there is a limited list of commits during that timeframe that could have introduced the problem. It would be great if we could roll back #2742855 in particular and see if it solves your problem. CiviCRM could indeed be compounding the problem, but if so, it's strange that the 7.x-3.3 release works fine for many sites even with CiviCRM.

k.elizabeth’s picture

k.elizabeth commented

I would look forward to any rollback of commits that could potentially be related to the max_connections issue. I would be happy to test a dev release with such rollbacks and report on the results!

couturier’s picture

couturier commented

@k.elizabeth and @Cyberflyer Can either of you help with this? We need someone to rollback the specific commit mentioned on the current release and re-test with your CiviCRM systems. Not a lot of contributors are helping with Backup and Migrate as far as writing and testing because so many of the larger sites have moved to alternate backup methods beginning with Drupal 8. It would be a great help if those of you that this error is affecting could rollback and test with your systems. If the commit mentioned has no effect, then previous comments explain how to find the half dozen or so other commits that could be the culprit.

couturier’s picture

couturier commented
Title: Too many connections in backup_migrate_source_db->_get_db_connection() (with CiviCRM) » Too many connections in backup_migrate_source_db->_get_db_connection()
Related issues: +#2949897: more than 'max_user_connections' active connections

This is fairly interesting that another user is having the same issue without using CiviCRM. We're still needing help with rolling back the commit and testing since the module works fine for most of us, so we wouldn't be able to see whether that certain commit or others within the time frame may have caused the issue. Can anyone who is having this error help with testing?

k.elizabeth’s picture

k.elizabeth commented

I will try to find some time in the next couple of weeks to roll back commits and test; I will update here with the results once I do.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Version: 7.x-3.5 » 7.x-3.x-dev
sharonknieper’s picture

sharonknieper commented

I too am seeing this issue on a Drupal/Civicrm site. The behavior I see is as k.elizabeth describes with the ability to make a manual backup just fine however trying to go to the settings page results in the following error:

"PDOException: SQLSTATE[HY000] [1203] User {database username} already has more than 'max_user_connections' active connections in backup_migrate_source_db->_get_db_connection() (line 223 of /sites/all/modules/backup_migrate/includes/sources.db.inc)."

I noticed the error when using 7.x-3.5 and tried reversing the changes in #2742855 per this thread, upgrading to the recently released 7.x-3.6, and downgrading to 7.x-3.3. None of these changes fixed the issue.

However, I did notice that if I removed the line defining the civicrm database from the 'backup_migrate_sources' mysql table I was then able to access the settings page. So I removed the source, profile and scheduled backup associated with the civicrm database. Once I added the source for the civicrm database back again I am no longer able to access the schedules page nor the settings page url (admin/config/system/backup_migrate/settings). Though if I directly visit the source settings url (admin/config/system/backup_migrate/settings/source) it does not cause the error.

I also made a new mysql user with permissions on the civicrm database only for creating a backup and migrate source but it still throws the error that there are too many connections.

I'm not sure how to debug this further but I am willing to test ideas if anyone has any.

The site doesn't have plans to move to D8 soon so a fix in 7x would be greatly appreciated.

couturier’s picture

couturier commented

@sharonknieper Thank you so much for taking time to do the testing and reporting back. Our original users involved in this issue never got back with us, so this issue had been shelved for a long time. I find it impressive that you reverted to 7.x-3.3 and were still having the problem since the previous user said that everything was working fine in that version and only changed upon moving to 3.4. So, that tells us that it's probably not related to any upgrades to the module during that time and may be another issue or combination of issues that are more complex. We'll see if anyone else can help or comment on your findings.

Also, on the topic of not planning to upgrade to Drupal 8, I know a lot of sites seem to be stuck between the huge changes that were introduced with D8 and what they consider to be a workable arrangement with D7. It does concern me what will happen once D7 is no longer supported upon release of D9. There have been a few users moving to BackDrop CMS which is a fork of D7 that will hopefully be maintained for longer than D7 itself. There are pros and cons, and I've seen discussion of porting Backup and Migrate into BackDrop but have no idea if that's proceeding at this point.

nelslynn’s picture

nelslynn commented

I can confirm I also get this error with Drupal 7 and CiviCRM. I also need to increase my PHP memory and Maximum execution time of 30 seconds or I get the following when I run /update.php then browse back to the home page:
PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /home/mysite/public_html/includes/menu.inc on line 3799
and/or
PHP Fatal error: Maximum execution time of 30 seconds exceeded in /home/mysite/public_html/includes/database/database.inc on line 2227

damienmckenna’s picture

damienmckenna
Location TN, USA
commented

@nelslynn: If your Drupal 7 site is giving out-of-memory errors loading the homepage when the memory limit is set to 512mb there's definitely room for improvement on your site and/or server, completely unrelated to Backup Migrate.

couturier’s picture

couturier commented

There were 4 new releases of CiviCRM during the time period in question when the error began occurring for our earlier posters. It would be interesting to know which version of CiviCRM they were using in combination with Backup and Migrate 7.x-3.3 that allowed things to go smoothly. I am wondering if any changes to CiviCRM affected the interaction starting with Backup and Migrate 7.x-3.4 or if we are dealing with something much more complex than software changes -- perhaps the local environment as Damien McKenna suggested.

nelslynn’s picture

nelslynn commented

My issue has been resolved. I did a couple things related to CiviCRM that may have helped resolve the issue:

- Converted Datetime fields to Timestamps
- Changed php version to 7.1

Thanks for all the help.

jackalope’s picture

jackalope commented

From @couturier on update #26 on this issue: "@k.elizabeth and @Cyberflyer Can either of you help with this? We need someone to rollback the specific commit mentioned on the current release and re-test with your CiviCRM systems."

I think I was finally able to do this myself, though not specifically via git. Here's how:

https://www.drupal.org/files/issues/2742855-n4.patch is the patch that causes the changes to includes/profiles.inc that then result in this "Too many connections" error.

I downloaded and reverted https://www.drupal.org/files/issues/2742855-n4.patch against backup_migrate-7.x-3.6; reverting that patch resolved the error.

That patch is connected to Drupal.org issue https://www.drupal.org/node/2742855, which was fixed by this commit: https://www.drupal.org/commitlog/commit/4484/0954938cedcd37f4559d035892d...

So rolling back commit 0954938cedcd37f4559d035892dc71e20e7fcc25 should fix the problem.

If y'all definitely need someone with an affected site to clone the backup_migrate git repo, roll back that specific commit and test again, let us know, we should be able to make that happen sometime soon!

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Category: Support request » Bug report
Status: Active » Needs review
StatusFileSize
new backup_migrate-n2946311-37.patch610 bytes

I've looked this over. I think we need to revert the original change and then work out when the static variable needs to be cleared. In #2742855: Caching profile objects led to unwanted behaviours because of stale data hanoii said that problems happened when multiple scheduled tasks were conflicting, so maybe the profile static variable should be cleared after each backup execution?

rrirower’s picture

rrirower commented

I've also begun to see this problem.

PDOException: SQLSTATE[HY000] [1203] User XXXXX already has more than 'max_user_connections' active connections in backup_migrate_source_db->_get_db_connection() (line 223

It occurred after I had added a new source and profile. I then tried to add a schedule and the above message now appears. For giggles, I tried the posted patch and it did not work. The database I am trying to back up is a non-Drupal database within PHPMyAdmin.

Is there any way for me to remove the source and profile I've already defined?
Is this close to being fixed?

jackalope’s picture

jackalope commented

For giggles, I tried the posted patch and it did not work.

@rrirower, Did you try applying https://www.drupal.org/files/issues/2742855-n4.patch, or some other patch? If you tried applying that patch, you need to instead revert it!

Here are the commands I used to download and revert that patch, which fixed this problem for me:

cd sites/all/modules/backup_migrate
wget https://www.drupal.org/files/issues/2742855-n4.patch
patch -p1 -R < 2742855-n4.patch
jackalope’s picture

jackalope commented

Ugh sorry @rrirower, I realize you probably meant you tried applying https://www.drupal.org/files/issues/2019-02-06/backup_migrate-n2946311-3... that Damien recently posted; I've not yet tested that patch myself!

rrirower’s picture

rrirower commented

jackalope,

Yes, that was the patch. I've already reverted it.

jackalope’s picture

jackalope commented

I tested @DamienMcKenna's patch in comment #37 above by reverting my patch-reversion-fix documented in my earlier comment #36, then applying the new patch instead. The scheduled backups that were triggering the "Too many connections" error have continued to work just fine, so I'd say the patch works, at least for the problem we're experiencing! (I can also see that the new patch is basically a reversion of https://www.drupal.org/files/issues/2742855-n4.patch so I'm not surprised it works!)

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Issue tags: +Needs tests

Thanks @jackalope.

The next step is to work out what items need to be cleared to avoid the bug reported in #2742855: Caching profile objects led to unwanted behaviours because of stale data that caused this problem in the first place.

couturier’s picture

couturier commented

@jackalope I really appreciate your finally doing this testing and finding the commit that was the culprit. Thanks for your contribution. It's good to know this was a change in Backup and Migrate rather than something that changed within CiviCRM at the time.

Anonymous’s picture

Anonymous (not verified) commented

I have just hit the same problem.
Drupal 7.63. Backup & Migrate 7.x-3.6. PHP 7.2.11. CiviCRM 5.9.1.

Tony

jackalope’s picture

jackalope commented

Thanks for the thanks @couturier and @DamienMcKenna, happy to help!

couturier’s picture

couturier commented
Status: Needs review » Needs work
Parent issue: » #2742855: Caching profile objects led to unwanted behaviours because of stale data

I'm setting this back to needs work because now we have two issues here, the original patch which fixed a caching/stale data issue but then created this second issue of "too many connections" for CiviCRM users. Damien McKenna has suggested that we go back and re-work the original patch that caused the problem.

katharine_gates’s picture

katharine_gates commented

Can confirm I am also having this problem with a combination of a Drupal and a Civi database.
Backup and Migrate 7.x-3.6, civicrm 7.x-5.10.3

All worked fine setting up scheduled backups for default database, but when I attempted to set up scheduled backups for Civi database, got this nasty error and wsod. Now I am unable to access either /admin/config/system/backup_migrate/settings or /admin/config/system/backup_migrate/schedule to even fix it!!

Can anyone give me a clue where to go in database to at least clear this up so we can reset??

bmango’s picture

bmango commented
Issue summary: View changes

@Katharine_Gates, you could try simply uninstalling Backup and Migrate, and then reinstalling a previous version. I don't think uninstalling the module will remove any of your backups.

katharine_gates’s picture

katharine_gates commented

Thank you bmango but unfortunately we have a bunch of Features modules (I am picking up from another developer, ugh!) that require B&M and they all seem to be circularly requiring each other. I can disable the Civi Source and that fixes it, but it sure would be nice to be able to backup Civi on a schedule!

bmango’s picture

bmango commented

You can still backup the Civi database on a schedule by using a bash script and then use cron to execute the script on a regular basis. I answered a similiar question on this in stack exchange and gave an example of a script you can use.

I was having the same problem as you backing up my Civi db and so I used a script to back it up instead of Backup and Migrate. I just use Backup and Migrate to backup the Drupal db.

jackalope’s picture

jackalope commented

Hi @Katharine_Gates -- until this problem is fixed in an official release of the module, you can try applying the patch posted at the top of this ticket to your site's copy of backup_migrate. Check out the Drupal documentation on applying patches for help on how to do it; here are the commands I run from my Drupal root directory to download and the patch to my $DRUPAL_ROOT/sites/all/patches directory and apply it to my module in $DRUPAL_ROOT/sites/all/modules/backup_migrate:

cd sites/all/patches
wget https://www.drupal.org/files/issues/2019-02-06/backup_migrate-n2946311-37.patch
cd ../modules/backup_migrate
patch -p1 < ../../patches/backup_migrate-n2946311-37.patch

And hopefully someone gets to tackle the work outlined in update #47 above soon so that the fix makes it into the module itself!

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Status: Needs work » Needs review
Parent issue: #2742855: Caching profile objects led to unwanted behaviours because of stale data »
Related issues: +#2742855: Caching profile objects led to unwanted behaviours because of stale data, +#2672478: Wrong destination when running multiple schedules using the same profile
StatusFileSize
new backup_migrate-n2946311-53.patch1.16 KB

@jackalope: Thank you so much for providing those instructions, I really appreciate the help.

A minor reroll and adjustments to comments.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Parent issue: » #3020775: Plan for Backup and Migrate 7.x-3.7
jackalope’s picture

jackalope commented
Status: Needs review » Reviewed & tested by the community

A minor reroll and adjustments to comments.

The reroll in #53 has worked properly for me!

damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Issue tags: -Needs tests
damienmckenna’s picture

damienmckenna
Location TN, USA
commented
damienmckenna’s picture

damienmckenna
Location TN, USA
commented
Status: Reviewed & tested by the community » Fixed

Committed. Thank you all.

  • DamienMcKenna committed 37aaeba on 7.x-3.x 
    Issue #2946311 by DamienMcKenna, k.elizabeth, couturier, jackalope,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.