Problem/Motivation

We have a bunch of unorganized XML-RPC-related code scattered throughout Drupal core.

Proposed resolution

Move all the XML-RPC code into one single XML-RPC module to help organize it all.

Remaining tasks

User interface changes

The user now sees an XML-RPC module that they can toggle to enable/disable the XML-RPC capabilities on their site.

API changes

The xmlrpc() function is only available if the XML-RPC module is enabled.

Original report by catch

Instead of a random xmlrpc.php file in the root of the Drupal directory, we can just have a menu callback for xmlrpc.php provided by a module. Handy thing that Drupal, providing menu callbacks.

Patch is 100% untested, I imagine/hope there are some xmlrpc tests in system somewhere we could move to the xmlrpc folder.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch’s picture

FileSize
60.86 KB

Missing files from the patch.

sun’s picture

Issue tags: +API clean-up

.

catch’s picture

FileSize
63.41 KB

Missed xmlprc() in common.inc

catch’s picture

Issue tags: +Framework Initiative
sun’s picture

Framework is our new tag for all of these issues.

catch’s picture

So the patch is big, but the changes are tiny:

1. Removes xmlrpc.php entirely.

2. Adds xmlrpc.module - this implements hook_menu() and registers xmlrpc.php as a callback.

3. Removes xmlrpc() from common.inc and moves it onto xmlrpc.module - modules that need to call xmlrpc() would now need to add xmlrpc module as a dependency. That is the only API change.

4. Moves xmlrpc.inc and xmlrcps.inc into the xmlrpc module directory out of /includes.

sun’s picture

2. Adds xmlrpc.module - this implements hook_menu() and registers xmlrpc.php as a callback.

Sleeping over this, there's one non-obvious consequence here: Without clean URL support, you cannot use xmlrpc.php anymore.

If we consider that to be a problem, we could provide the current/legacy xmlrpc.php within the module folder, so sites that cannot or do not want to enable clean URLs are able to copy the file into their site's root directory.

catch’s picture

That's an interesting point, I personally cannot remember seeing a real Drupal site running without clean URLs. Another possibility would be to add a hook_requirements().

cweagans’s picture

I think that clean urls should just be a requirement of using xmlrpc module. Another approach would be that we just completely remove the xmlrpc server and let contrib handle it with Services module (we've already removed the prime reason to even have an xmlrpc server, which was Blog API)

lyricnz’s picture

+1 for separate module, +1 for this not being in core (and the reason why I won't RTBC the patch)

Status: Needs review » Needs work

The last submitted patch, xmlrpc.patch, failed testing.

catch’s picture

Status: Needs work » Needs review
FileSize
63.43 KB

Re-rolled.

Obligatory diffstat:

catch@catch-laptop:~/www/8$ diffstat xmlrpc.patch
 b/includes/common.inc             |   33 --
 b/modules/xmlrpc/xmlrpc.inc       |  625 ++++++++++++++++++++++++++++++++++++++
 b/modules/xmlrpc/xmlrpc.info      |    5 
 b/modules/xmlrpc/xmlrpc.module    |   54 +++
 b/modules/xmlrpc/xmlrpc.pages.inc |   14 
 b/modules/xmlrpc/xmlrpcs.inc      |  385 +++++++++++++++++++++++
 includes/xmlrpc.inc               |  625 --------------------------------------
 includes/xmlrpcs.inc              |  385 -----------------------
 xmlrpc.php                        |   18 -
 9 files changed, 1083 insertions(+), 1061 deletions(-)

I am posting these patches just to disentangle the code base - in this case we remove a file from the root directory, two from /includes, and 33 lines from common.inc.

The fact that if we later wanted to remove xmlrpc support from core entirely it would be a simple git rm is just a happy side effect.

It's also possible that the web services initiative may eventually want to add xmlrpc support in core via that mechanism - but if it does then this makes that neither harder nor particularly easier to do so.

If you like this patch, you may also like such issues as #679112: Time for system.module and most of includes to commit seppuku.

I have not yet added the hook_requirements() for clean urls, but marking CNR for the test bot.

Status: Needs review » Needs work

The last submitted patch, xmlrpc.patch, failed testing.

catch’s picture

Status: Needs work » Needs review
FileSize
63.43 KB

Fixing the syntax error.

Status: Needs review » Needs work

The last submitted patch, xmlrpc.patch, failed testing.

catch’s picture

Status: Needs work » Needs review
FileSize
90.72 KB

Status: Needs review » Needs work

The last submitted patch, xmlrpc.patch, failed testing.

catch’s picture

Status: Needs work » Needs review
FileSize
90.72 KB

grumble, grumble.

joachim’s picture

> Another approach would be that we just completely remove the xmlrpc server and let contrib handle it with Services module (we've already removed the prime reason to even have an xmlrpc server, which was Blog API)

+1 for this option. I don't see why anyone would use core's xmlrpc server when there is Services module available.

Similarly, the code for making xmlrpc calls could be moved out to Clients module. There's no reason for core to be favouring xmlrpc over any other type of service call.

PS. Though isn't part of Services moving into core in D8 anyway? If so, won't that initiative be handling the cleanup of core's xmlrpc code?

catch’s picture

If you want to open up an issue to move xmlrpc to contrib, that will be a very easy patch once this is issue is in:

git rm modules/xmlrpc
git diff origin/8.x > remove_xmlrpc_module.patch

That's not what this issue is about, so I don't know why people keep bringing it up, you're free to open up that issue if you like.

I have no idea whether the web services initiative will have anything to do with xmlrpc by the time it's done, currently that is 100% vapourware, so I have limited interest in having mess in the root directory waiting on that.

marcingy’s picture

Services provides a wrapper around the core xmlrpc functionality so I would be against this removal for the most part as a services maintainer. If xmlrpc leaves core I can pretty much say it will die (from a services prospective) because the vast majority of services maintainer are only interested in json servers.

catch’s picture

@marcingy, services could add a requires[] = xmlrpc, or use module_exists() or whatever though if it stays in core as a module though right?

marcingy’s picture

@catch total. my comment was in response to the comments of remove xmlrpc from core.

aspilicious’s picture

+++ b/modules/xmlrpc/xmlrpc.infoundefined
+++ b/modules/xmlrpc/xmlrpc.infoundefined
@@ -0,0 +1,5 @@

@@ -0,0 +1,5 @@
+name = XML-RPC ¶
+description = Provides XML-RPC functionality.

Trailing whitespace, other docs seems to be ok

Powered by Dreditor.

joachim’s picture

> I have no idea whether the web services initiative will have anything to do with xmlrpc by the time it's done, currently that is 100% vapourware, so I have limited interest in having mess in the root directory waiting on that.

Yup, fair point. I see what you mean: clean up what we definitely can now, clean up more later if/when other things happen :)

catch’s picture

FileSize
90.72 KB

Removed the trailing whitespace.

joachim’s picture

Status: Needs review » Needs work

Just a few points -- and TBH some of it is stuff that could be looked at later.

+++ b/modules/xmlrpc/xmlrpc.module
@@ -0,0 +1,54 @@
+  $items['xmlrpc.php'] = array(

I had my doubts about this, but I just looked at xmlrpc.php and it does drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL). How much do we lose going via the menu system rather than straight in?

+++ b/modules/xmlrpc/xmlrpc.module
@@ -0,0 +1,54 @@
+    'type' => MENU_SUGGESTED_ITEM,

Surely MENU_CALLBACK?

You won't get anything visible back from this with a GET, just a white screen (or a message saying 'only POST', I forget).

+++ b/modules/xmlrpc/xmlrpc.module
--- /dev/null
+++ b/modules/xmlrpc/xmlrpc.pages.inc

Why do we need both the xmlrpc.pages.inc and the xmlrpcs.inc?

Powered by Dreditor.

catch’s picture

Status: Needs work » Needs review
FileSize
45.16 KB

There is negligible difference between a full bootstrap from a custom .php file and using the router system in terms of performance - more or less just menu_get_item() I think. Hopefully routing of requests is going to be moved to a lower level with WSCCI so it may end up cheaper overall by the time the release is out, but this is very far from critical path so I'm happy with using what we have for now.

The only issue with using the menu system is dropping support for dirty URLs, which I'd like to do in general at #1183208: Remove variable_get('clean_url') and switch to index.php/path pattern for dirty URL support anyway.

Changed MENU_SUGGESTED_ITEM to MENU_CALLBACK, I must have copied a .module file since I can never remember what to write in @file, then failed to clean up the hook_menu() properly.

Merged xmlrpcs.inc into xmlrpc.pages.inc, agreed it is not worth two separate files.

Thanks for the review!

Status: Needs review » Needs work
Issue tags: -Framework Initiative, -API clean-up

The last submitted patch, xmlrpc_module.patch, failed testing.

Eric_A’s picture

Status: Needs work » Needs review

#28: xmlrpc_module.patch queued for re-testing.

Oops. First I saw CommentBlockFunctionalTest fails, and only after requesting the retest did I notice that lines have gone missing in the latest patch...

Status: Needs review » Needs work
Issue tags: +Framework Initiative, +API clean-up

The last submitted patch, xmlrpc_module.patch, failed testing.

joachim’s picture

Those are some very weird tests that are failing.

A lot of them are to do with file_put_contents(). How can that have been affected by this?

lyricnz’s picture

More likely to be a testbot issue, perhaps out of disk space.

joachim’s picture

Status: Needs work » Needs review

#28: xmlrpc_module.patch queued for re-testing.

cweagans’s picture

Status: Needs review » Needs work

It appears that the most recent patch is not creating the xmlrpc module, but removing all traces of the xmlrpc server and client from core. I'm okay with this, but I'm certain that's not what you were aiming for, so back to CNW.

franz’s picture

Yeah, catch probably forgot to add new files on latest patch (#28)

catch’s picture

pwolanin asked why make this a module since it's currently standalone, here's the justification:

- if something only works by having a single-purpose PHP file in the root folder of the core download I don't think we can call it standalone. If I have my way we'll eventually ditch install.php and update.php too one day.

- it does a full Drupal bootstrap so there is no benefit to a one-off script compared to going via the router.

However having it as a module does add overhead if it's actually enabled compared to now, but it's definitely got issues either way.

podarok’s picture

subscribe

RobLoach’s picture

Component: base system » xml-rpc system
Status: Needs work » Needs review
Issue tags: +XML-RPC
FileSize
49.63 KB

Blog module was the only thing in Drupal core using the XML-RPC system. Now that Blog module is no longer in core, there is no reason to have XML-RPC in there.

Also, I moved xmlrpc.module, what catch was working on, to http://drupal.org/node/1285602 . We can promote it to a full project if we want. I've given a bunch of you commit rights on there.

cweagans’s picture

*cough*

:%s/Blog module/Blog API module/g

sun’s picture

Please create a separate issue for removing xmlrpc altogether. I'll explain on that one why I'm going to mark it postponed.

sun’s picture

#28: xmlrpc_module.patch queued for re-testing.

boombatower’s picture

sub

RobLoach’s picture

Please create a separate issue for removing xmlrpc altogether. I'll explain on that one why I'm going to mark it postponed.

Here ya go: #1285726: Remove XML-RPC

sun’s picture

Status: Needs review » Needs work

#26 / #28 needs to be re-rolled.

amateescu’s picture

Status: Needs work » Needs review
FileSize
9.02 KB

The patch from #26 was not re-rollable at all, so I basically recreated it and also added the changes advertised in #28 but not really added to that patch :)

amateescu’s picture

FileSize
9.03 KB

We've transferred this work to: http://drupalcode.org/sandbox/sun/1255586.git/shortlog/refs/heads/platfo...

Attached patch contains a change by @sun that renames xmlrpc.pages.inc to xmlrpc.server.inc.

pwolanin’s picture

A PHP file actually works better than a callback for fastCGI environments, since some headers are not passed through the URL rewrite process.

A separate file could actually have a different bootstrap? e.g. never need the theme layer?

I think sun had such a file for JS callbacks at one point.

Damien Tournoud’s picture

I don't think we want more then one front-end controller (ie. a PHP entry point) for now. The are refactoring going on in #1463656: Add a Drupal kernel; leverage HttpFoundation and HttpKernel so let's stick with the proposed xmlrpc.inc routing entry for now.

RobLoach’s picture

Since the functionality was enabled by default, do you think we should retain the upgrade path for it?

/**
 * Move XML-RPC functionality to its own module.
 *
 * @see http://drupal.org/node/1033818
 */
function system_update_8005() {
  module_enable(array('xmlrpc'));
}
amateescu’s picture

FileSize
537 bytes
9.55 KB

Yes, definitely. Thanks Rob :)

RobLoach’s picture

FileSize
9.5 KB
+++ b/core/modules/xmlrpc/xmlrpc.moduleundefined
@@ -0,0 +1,53 @@
+ /**
+  * @file
+  * Enables XML-RPC functionality.
+  */
+
+ /**
+  * Implements hook_menu().
+  */
+ function xmlrpc_menu() {
+   $items['xmlrpc.php'] = array(
+     'title' => 'XML-RPC',
+     'page callback' => 'xmlrpc_server_page',
+     'access callback' => TRUE,
+     'type' => MENU_CALLBACK,
+     'file' => 'xmlrpc.server.inc',
+   );
+   return $items;

There's some extra opening whitespace there. Attached patch doesn't have it. If it goes green, I'd consider this RTBC!

14 days to next Drupal core point release.

cweagans’s picture

Status: Needs review » Needs work
+++ b/core/modules/system/system.installundefined
@@ -1761,6 +1761,15 @@ function system_update_8004() {
+* Move XML-RPC functionality to its own module.
+*
+* @see http://drupal.org/node/1033818
+*/
+function system_update_8005() {
+  module_enable(array('xmlrpc'));
+}

I don't agree with this. This means that every single Drupal site that upgrades from 7 -> 8 will have xmlrpc.module enabled, even if they don't need it. I'd say let's leave it disabled, and then document an API change: if your modules depend on XML-RPC functionality, then you need to enable the xmlrpc module.

+++ b/core/modules/xmlrpc/xmlrpc.moduleundefined
@@ -0,0 +1,53 @@
+ function xmlrpc($url, $args, $options = array()) {
+   module_load_include('inc', 'xmlrpc');
+   return _xmlrpc($url, $args, $options);

Should this be named something else? Most of the other modules that we have in core don't have functions that are named the same as the module name. What about xmlrpc_request()?

RobLoach’s picture

Status: Needs work » Needs review
Issue tags: +Needs change record
FileSize
14.09 KB

I'd say let's leave it disabled, and then document an API change

Change notification it is!

What about xmlrpc_request()?

Sure!

Status: Needs review » Needs work

The last submitted patch, 1033818.patch, failed testing.

RobLoach’s picture

Status: Needs work » Needs review
FileSize
14.04 KB

Uses xmlrpc_call() instead of xmlrpc_request() since that's already defined.

fgm’s picture

Just a note: this issue, and others related to XMLRPC in D8, are part of the program for the Paris code sprint on 2012/04/28-29. More info at http://groups.drupal.org/node/223174

sun’s picture

Title: xmlrpc.module » Move xmlrpc includes into xmlrpc.module
FileSize
8.97 KB

Re-rolled and adjusted for latest HEAD.

This code lives in the platform/platform-xmlrpc-1033818-sun branch now.

Note: I've intentionally reverted the unnecessary xmlrpc() API change.

RobLoach’s picture

Status: Needs review » Needs work
Issue tags: +XML-RPC, +Framework Initiative, +API clean-up, +Needs change record

The last submitted patch, platform.xmlrpc.58.patch, failed testing.

RobLoach’s picture

Status: Needs work » Needs review
FileSize
8.51 KB

Re-roll.

Status: Needs review » Needs work

The last submitted patch, 1033818.patch, failed testing.

RobLoach’s picture

Status: Needs work » Needs review
FileSize
17.66 KB

Got it! The problem was WSCCI fallout from the index.php change.

sun’s picture

Status: Needs review » Reviewed & tested by the community

Excellent, thanks!

catch’s picture

Status: Reviewed & tested by the community » Needs work

Sorry folks, all core modules need a hook_help().

RobLoach’s picture

Status: Needs work » Needs review
FileSize
24.98 KB

Thanks! Also noticed the system.api.php hook_xmlrpc() docs could be moved in too.

sun’s picture

Status: Needs review » Reviewed & tested by the community

Awesome!

RobLoach’s picture

#66: 1033818.patch queued for re-testing.

Also, updated the issue summary.

tstoeckler’s picture

+++ b/core/modules/xmlrpc/xmlrpc.module
@@ -0,0 +1,68 @@
+        '@xmlrpcapi' => 'http://drupal.org/node/44895',
+        '@xmlrpc' => url('xmlrpc.php', array('absolute' => TRUE)),

Minor, but the placeholders are defined in the opposite order they are used. Also, maybe a dash or two would make the placeholder names more readable.

+++ b/core/modules/xmlrpc/xmlrpc.module
@@ -0,0 +1,68 @@
+  $items['xmlrpc.php'] = array(

Is there a reason this is 'xmlrpc.php' vs. just 'xmlrpc'? At first I didn't want to mention this as I thought it would be bikeshedding, but then I noticed that if you don't have mod_rewrite, then that path looks like example.com/index.php/xmlrpc.php, which is super weird IMO.

Leaving at RTBC for now.

RobLoach’s picture

FileSize
24.98 KB
883 bytes
  • Switching those declarations to be the right order makes sense to me!
  • Switching to just "xmlrpc" for the menu callback should probably be a follow-up issue. This just tries to keep backwards compatibility with Drupal 7. I agree it's the right path though!
cweagans’s picture

IMO, backwards compatibility shouldn't be a huge concern here - if a site is making use of the XML-RPC server and they're upgrading from D7->D8, they'd likely also update whatever system is using XML-RPC.

Also? Who really doesn't use mod_rewrite ;)

RobLoach’s picture

Created #1627496: Switch XML-RPC callback from "xmlrpc.php" to "xmlrpc" to get that fix in afterwards :-) . I'll stick it in the issue summary.

RobLoach’s picture

Issue summary: View changes

Updated issue summary.

tstoeckler’s picture

RTBC++

catch’s picture

Title: Move xmlrpc includes into xmlrpc.module » Change notification for: Move xmlrpc includes into xmlrpc.module
Priority: Normal » Critical
Status: Reviewed & tested by the community » Active

Thanks! Committed/pushed to 8.x.

Will need a change notification.

RobLoach’s picture

Status: Active » Needs review
cweagans’s picture

Title: Change notification for: Move xmlrpc includes into xmlrpc.module » Move xmlrpc includes into xmlrpc.module
Status: Needs review » Fixed
Issue tags: -Needs change record

Perfect!

cweagans’s picture

Priority: Critical » Normal

Oh yeah, priority should change too.

Status: Fixed » Closed (fixed)
Issue tags: -XML-RPC, -Framework Initiative, -API clean-up

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Updated issue summary.