New backup destination: NodeSquirrel (D8)

NodeSquirrel is currently not supported in the Drupal 8 version.

Is there a plan to have NodeSquirrel in Drupal 8? I really miss it.

Ronan who was the original creator of Backup and Migrate is also the creator of NodeSquirrel. Now that the D8 development has been handed to new maintainer Alex Andrascu as of September 2017, it is unlikely that this will be built in to the D8 version unless ronan is able to come back to the project at some point in the future and work on it. I'm marking this as closed - won't fix, and I'm also recommending that the main page documentation be updated to reflect this change so developers won't be confused.

Let's see if we can bring this back to life again. Can we have a patch for this please ?

There is a small patch that gives ability to create destination to the NodeSquirrel.
How to :
1. go to 'admin/config/development/backup_migrate/settings/destination' and add new destination choosing NodeSquirrel destination type and add your secret key to configure it.
2. Now go back to 'admin/config/development/backup_migrate' and choose the new NodeSquirrel as Destination

I thinks this just needs a re-roll after y/day release. Otherwise a good looking patch.
Thank you @Lilian for your time spent on it.

Fixed whitespaces trailling error on applying patch.

Windows is a hell, there is unix system patch

Great stuff Lilian and thank you for this.

Can anyone test this and report back with findings so we can plan for a release soon?

I can have a look at it tonight.

If you try to do a quick backup you get the error

"Could not initiate an upload to NodeSquirrel. Error: The NodeSquirrel server returned the following error: NodeSquirrel requires the php XML-RPC extension. (code: 0)"

@ikit-claw This would not be the first issue for Backup & Migrate requiring either code to activate an extension or instructions for developers to manually activate an extension as a work-around. Maybe this one can be added to the patch.

@Alex Andrascu it's great to hear another D8 release is in the works soon. I have seen recent discussion on another thread about the slow rate of D8 core adoption and how lack of a stable non-Composer dependent backup module for D8 may be part of the reason some developers are holding back. Thanks for all your work, and I see Damien McKenna and a few others have been active lately too.

@couturier I see but many will be looking for one link and it works functionality I will talk to Damien about it I am helping with the module. Am just not sure a patch that also requires system tweaks is good enough.

If we’re talking about the php xmlrpc library that wouldn’t count as a system tweak if you ask me.

Will review the patch tomorrow with the team and revert with an actionable outcome.

However, while I don’t disagree with you, it would be good to get your actual suggestions, proposals on a workaround if you think this isn’t good enough yet.

The Backup Migrate library bundled with the module already requires XML-RPC in NodeSquirrelClient::xmlrpcCall(), so I don't think it's too much to say that it's needed for the module's implementation.

OTOH I don't think this patch should be extending the uploadFile() method, if anything needs tweaks it should go into the class itself so that the module's classes are minor wrapper, nothing more.

Lastly, it should add a hook_requirements to check if the XML-RPC extension is available and then the NodeSquirrel option should be disabled if it is not.

@Alex I was just doing a quick review for the weekend we all need a break.

@Damien I didn't check the libraries for this actually my bad I just looked at the patch and then ran it. Perhaps it should display a warning if it doesn't detect it. But if it is in the libraries it should be fine.

@DamienMcKenna NodeSquirrelClient::xmlrpcCall() has just a check for function_exists() and throws an exception that doesn't really work, and i think is bad idea to check just one function instead of entire library.

@ikit-claw Something like this (check the attached screenshots) would be enough to avoid misunderstandings in the future? ( I think i provided not enough information for configuring NodeSquirrel destinations)

added hook requirements to the patch + some UI notice for xmlrpc php library

The patch itself looks good and solves the main purpose of this post. Regarding Damien's comment on #23 I think this really depends on what we plan to do with the library.

Personally I can see two options here, keep the library as it was originally intended trying to extend it at a very low level working our way around the limitations or fork it and keep it as part of the module. The first comes with all the obvious overhead though I've failed to identify any benefits. I'm not saying there aren't any just that I can't really see them at the moment. Maybe Ronan can shed some light on this and help us in taking a better, more informed decision. It could be a whole new topic on itself and I would be happy to discuss the future of the library in a separate post.

Meanwhile I would do my best to RTBC the above patch and release it by the end of this week unless the other maintainers are willing to reason against it.

Have applied the patch to both dev and rc-1, but when trying to manually backup to NodeSquirrel, I'm getting the error:

Fatal error: Interface 'BackupMigrate\Core\Service\HttpClientInterface' not found in /var/www/html/drupal/modules/backup_migrate/lib/backup_migrate_core/src/Service/PhpCurlHttpClient.php on line 17

HttpClientInterface.php appears to exist at /var/www/html/drupal/modules/backup_migrate/lib/backup_migrate_core/src/Service/HttpClientInterface.php, though.

Appears to be a casing issue, as the file is called HTTPClientInterface.php. Don't have time to throw together a patch right now, but that was the problem.

@mrweiner NodeSquirrel is also picky about extensions. This comment #10 at Support for the Encrypt module isn't a casing issue, but the reporter was surprised by the following:

. . . Because of unexpected behaviour by NodeSquirrel. It appears that NS is fussy about file extensions.

If you save a file to NS with an extension it doesn't recognise, it won't list it in the UI - neither NS's web UI, or the file list returned to B-M.

Hi all, is there any update to this issue ?

I've just retested this more carefully and it looks like this needs work. It also appears that @DamienMcKenna was right on #23 in that we shouldn't extend on NodeSquirrelClient as that's not a fully working port in itself. The problem with the above patch is that the request isn't constructed properly. The post request in NodeSquirrelClient is curl based rather then http as in 7.x and the request is missing proper constructs(headers etc). Aiming to fix this by later this week. It's a bit difficult as there's no documentation to NodeSquirrel other then the 7.x. version of this module so I had to go and see how it is done over there. It's not very far off from a good port but there's still work to do.

I hope this helps.

Thanks for the review, Alex.

I'd recommend a proper port be written based upon recreating the D7 module's operations using D8's built-in Guzzle.

I agree. I'll look into it. Thanks Damien.

@Alex Andrascu it is my understanding that Ronan Dowling was the original developer of both Backup and Migrate and NodeSquirrel and has left both projects due to health issues and job commitments that limit his time. I do not expect any further work to be done on a NodeSquirrel port unless someone else takes up the project as you and your team have done with Backup and Migrate.

IMNSHO, adding NodeSquirrel as an offline backup destination for the D8 version of the project may not be the best solution, and alternatives need to be evaluted.

I had a relationship with NodeSquirrel as a paying customer until May 2018. However, I watched this relationship deteriorate after Pantheon acquired the NodeSquirrel (May 2015), and in May 2018, I realized that NodeSquirrel could no longer be relied upon for production sites. When I left, my impression was that the service had fallen into a black hole. For instance: support@nodesquirrel.com seems to be rerouted to /dev/null. While NodeSquirrel is still sold as a service, I believe that when Ronan Dowling left, Pantheon did not replace him.

My current backup setup still use Backup and Migrate for creating schedules and profiles. I believe Backup and Migrate is a very good backup-tool, and – because is is "Drupal-aware" – superior to low-level tools such as mysqldump.

However, after I dropped NodeSquirrel, I had to resort to using a CLI tool for offline backup (s3cmd – based upon AWS S3 API). It works well enough for my purposes, so I no longer need support for offline backup to be integrated in Backup and Migrate.

However, if somebody is willing to donate their time to integrate offline backup into this project, I would recommend that you first evaluate what alternatives to NodeSquirrel exists, and maybe pick a better service provider than Pantheon.

@gisle I didn't know Ronan Dowling had left Pantheon, but I did know that he was essentially the only creator and maintainer of NodeSquirrel as well as Backup & Migrate until the new team took over for Backup & Migrate and completed the Drupal 8 port. He had posted on a few threads that he had some medical issues and work issues that prevented him from continuing on in his original capacity. My impression has been the same as yours, that NodeSquirrel has fallen into a black hole. It's good to hear from someone who was actually a customer as to what the situation is. My opinion is the same as yours, that our limited resources would be better spent integrating with other solutions. i believe that one of the primary maintainers of Backup & Migrate, Damien McKenna, is a user of NodeSquirrel himself, so he might be interested in continuing on the work since it is something that has worked for him, but NodeSquirrel is going to need a new technical team to take over if it is to survive long term.

FYI I'm going to remove the current (broken) integration from the 8.x-4.1 release (#3047223: Temporarily remove NodeSquirrel from D8 branch) so we can fix it and then add it back again, presuming that NodeSquirrel itself is fixed soon (#3037038: NodeSquirrel status?).

I've been following this issue, hopeful that support might be added in the future, but it looks like NodeSquirrel was shut down by Pantheon on September 30th of 2019.

This can probably be closed as "Won't fix"

True for those who didn't see the message https://pantheon.io/nodesquirrel-service-end-life

Closed