This is a temporary solution to make running replications a little bit easier until we have implemented a "real" replicator in PHP (which hopefully will be worked on by Abhishek during this year's Google Summer of Code).

We basically need a few simple drush commands that will wrap the CouchDB replicator. So this version of the Drush command will require that you have CouchDB installed and running (by default at http://localhost:5984).

Here are the specification of all commands:

All commands should use a --replicator option that will point to the CouchDB replicator. This value defaults to http://localhost:5984.

All commands should use a --format option. See for example drush status which uses this option.
When the json format is used, it's important that the JSON structure is exactly the same format as the corresponding CouchDB output, for full compatibility.

We should use the Doctribe CouchDB library as our API client: https://github.com/doctrine/couchdb-client

Start and stop commands

# Starts a normal replication from site alias A to B
$ drush replication-start @site.a @site.b

# Starts a continuous replication from site alias A to B
$ drush replication-start --continuous @site.a @site.b

# Stops a replication.
$ drush replication-stop @site.a @site.b

The start and stop commands should essentially call $replicator_url/_replicate with the various parameters. Where $replicator_url is the value from the --replicator option (defaults to http://localhost:5984).

The output from both the start and stop commands should be something like the below, when using the standard key-value format. Note that no "history" data is displayed.

Status                         :  OK
Session ID                     :  1122334455
Last sequence number           :  123456
Replication protocol version   :  3

And when using the json format, it should be exactly the same output as per http://docs.couchdb.org/en/latest/api/server/common.html#replicate (including the history data).

{
    "history": [
        {
            "doc_write_failures": 0,
            "docs_read": 10,
            "docs_written": 10,
            "end_last_seq": 28,
            "end_time": "Sun, 11 Aug 2013 20:38:50 GMT",
            "missing_checked": 10,
            "missing_found": 10,
            "recorded_seq": 28,
            "session_id": "142a35854a08e205c47174d91b1f9628",
            "start_last_seq": 1,
            "start_time": "Sun, 11 Aug 2013 20:38:50 GMT"
        },
        {
            "doc_write_failures": 0,
            "docs_read": 1,
            "docs_written": 1,
            "end_last_seq": 1,
            "end_time": "Sat, 10 Aug 2013 15:41:54 GMT",
            "missing_checked": 1,
            "missing_found": 1,
            "recorded_seq": 1,
            "session_id": "6314f35c51de3ac408af79d6ee0c1a09",
            "start_last_seq": 0,
            "start_time": "Sat, 10 Aug 2013 15:41:54 GMT"
        }
    ],
    "ok": true,
    "replication_id_version": 3,
    "session_id": "142a35854a08e205c47174d91b1f9628",
    "source_last_seq": 28
}

Commands for active replications

# Prints all active replication tasks.
$ drush replication-active

# Prints information the specific active replication between site A and B
$ drush replication-active @site.a @site.b

The start and stop commands should essentially call $replicator_url/_active_tasks with the various parameters, but only list tasks with type replication.

The output from both utility commands should be something like this, when using the standard key-value format:

Source                  :  http://site-a.com
Target                  :  http://site-b.com
Started on              :  Sun, 11 Aug 2013 20:38:50 GMT
Progress                :  44
Documents read          :  123
Documents written       :  123
Revisions checked       :  123
Write failures          :  0
Process ID:             :  5678

And when using the json format, it should be exactly the same output as per http://docs.couchdb.org/en/latest/api/server/common.html#active-tasks

{
    "checkpointed_source_seq": 68585,
    "continuous": false,
    "doc_id": null,
    "doc_write_failures": 0,
    "docs_read": 4524,
    "docs_written": 4524,
    "missing_revisions_found": 4524,
    "pid": "<0.1538.5>",
    "progress": 44,
    "replication_id": "9bc1727d74d49d9e157e260bb8bbd1d5",
    "revisions_checked": 4524,
    "source": "mailbox",
    "source_seq": 154419,
    "started_on": 1376116644,
    "target": "http://mailsrv:5984/mailbox",
    "type": "replication",
    "updated_on": 1376116651
}
// ...more active tasks if no specific argument were given

Replication info commands

Retrieves information and history about old, inactive replications.

TODO

Validation

We should implement drush_hook_COMMAND_validate() to check that the couchdb package is installed.
If it's not installed, all commands should fail with an error message saying: "You need to have CouchDB installed in order to run this command."

Comments

dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
jonhattan’s picture

Version: 5.x-1.x-dev » 8.x-0.x-dev

I guess it is an 8.x thing :)

@dixon_ in your session at LA you said (or I understood that) CouchDB is not needed for such a Drupal - Drupal replication.

In the search for the command you used in the session I've found this issue, and also https://packagist.org/packages/relaxedws/replication so I finally understood CouchDB is mandatory for now, and there's a plan to get rid of it. Isn't it?

If things go well, I'll start using your cocktail in the next weeks for a project. I'll do me best to try and contribute!

  • dixon_ committed ecef012 on 8.x-1.x
    Merge pull request #1 from dickolsson/2476033-drush-commands
    
    Issue #...
jeqq’s picture

Status: Active » Closed (fixed)