I think this is rather a serious issue, as it leads to data loss.

It should be obvious that we cannot use the "spoke" model for sites when thinking about the whole archticture with remotes. Because at _least_ the database and the files will be more up-to-date on the remote in 100% of all cases, we cannot think about "push from the master" in this cases - and we dont.

But it seems like we dont do this for the sites/domain/modules and themes folder.

When you:
- put a theme into the domain/themes folder on the remote
- you migrate that site ( on the same remote or somewhere else )

You end up:
- losing that theme

Thats pretty bad. Iam not sure this can be avoided when verifing before migrating ( which i did not do ), but if, we _must_ do this automatically before migrating. I would advice to deal with the "site/domain" folder the "spoke" model way, but "reverse", so the "remote" is always the master. Lets call this folder the "users playground":
- he changes the database without askign the master
- he changes the files without asking the master
- he adds themes for himself or whatever

Iam sorry if this issue is again mixing up an issue and a feature request, but i think to proper fix the bug, we should "extend" the default behavior. What do you think?

CommentFileSizeAuthor
#3 syncfix.patch908 bytesEugenMayer
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

EugenMayer’s picture

Title: Sites data get lost / dont "spoke" the site » Sites data get lost on migrate or clone / dont "spoke" the site

Just for an update, on verify, the folder does not get synced, so the modules / themes folder would get lost in any case. Marking this as critical for now, as it really could harm pages as
- both folders are created when the site folder is "initiated"
- so people are invited to use it

As far as i see, this issue would not happen when you:
- dont use remote
- when you migrate from the aegir master to a remote

but it will happen when you clone / migrate:
- from a remote to a remote

EugenMayer’s picture

Well pretty simply, themes modules and libraries are never synced back to the master, only files ( platform/backup.provision.inc ), therefore those stuff is lost when its a remote. Patch incoming, also see http://drupal.org/node/1080588

Patch incoming, take cover :)

EugenMayer’s picture

Status: Active » Needs review
FileSize
908 bytes

Should be it. Yet iam not sure what about verify. It seems (code wise) that the site/domain folder is not synced from master to remote during verify.

EugenMayer’s picture

for the record, close http://drupal.org/node/1080588 if this one gets fixed

EugenMayer’s picture

confirming this patch fixes it for migrate / clone. Verify does not touch anything, as it does not sync ( so it also does not need to be patched ).

For me, its RTBC

bwood’s picture

Sounds like you are challenging some design decisions re the hub-spoke model?

If all changes are done on the hostmaster server's platform then verify will rsync them out to the remote webservers. You probably don't want developer logging in to the hostmaster server, right? I solved this with a version control (subversion) workflow. Developers checkin changes to the site to version control, a cron job periodically loops over 'drush sa' attempting to do 'svn up' on the working copy directories that live on the hostmaster server's platforms. When 'svn up' results in new data, the script 1) emails svn up output to the client_email 2) kicks off verify which syncs the site to the remote webserver.

EugenMayer’s picture

@6: no i dont challange it.

Or are you "spoking" files? no you arent, because they are changed on the remote. Are you "spoking" DB? No, its changed on the remote. Same for privates.

So the spoke model does not mean everything comes for the master. I fine with it that "code" is that way, so platforms are alway in sync. VHOST/ Platfom configurations, also global.inc.

I see that modules / themes are a mixup here, due to the possibility that they exists in sites/all. But i just take it very simple, everything in sites/$site/ is _site_ specific, not platform specific:
- files / private files
- DB (database.sql..)
- modules / themes

That is btw the only way to have something "site specific". Call it "site level overrides". And those stuff you want to get migrated with the site (or cloned) no matter what platform. And a theme is a _perfect_ example here. It would simply not work any other way.

EugenMayer’s picture

See http://drupal.org/node/1083366 for an additional fix during the verify process

anarcat’s picture

Status: Needs review » Fixed

I agree it is inconsistent to fetch "files" but not "private" from the remote server. modules/themes/libs are probably the same, and local.settings.php...

I wonder if we shouldn't use an exclude pattern instead...

fix committed, thanks.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

  • Commit b40b6e3 on dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by anarcat:
    #1079274 by EugenMayer - fetch more files from remote server when  doing...
  • Commit 9c484db on 6.x-1.x, dev-ssl-ip-allocation-refactor, dev-1205458-move_sites_out_of_platforms, 7.x-3.x, dev-subdir-multiserver, 6.x-2.x-backports, dev-helmo-3.x by anarcat:
    #1079274 by EugenMayer - fetch more files from remote server when  doing...