Authentication settings in the UI

Yes, you can add authentication credentials to the settings of a service, but only in code and not through the UI currently. Something like:

$service->settings['options']['login'] = 'user';
$service->settings['options']['password'] = 'pass';

See http://php.net/manual/soapclient.soapclient.php $options for available options.

We should add a username and password field to the UI to support basic authentication credentials. The options and their names are slightly different for curl (REST services) and PHP SOAP (SOAP services). I think we should define a unified place in $service->settings and then let the specific service types map that themselves.

subscribing to the authentication via UI

Stop subscribing, start following: http://drupal.org/node/1306444

Where should we place this code.

I have to connect to SOAP web service, but i should authenticate somewhere.

I always get this error.

    Warning: file_get_contents(.../Lists.asmx?WSDL): failed to open stream: HTTP request failed! HTTP/1.1 401 Unauthorized in wsclient_soap_wsclient_service_form_validate() (line 101 of.../sites/all/modules/wsclient/wsclient_soap/wsclient_soap.module).
    Error downloading the WSDL file.

How is this for REST?
If its possible, could somebody provide the hook & code for authenticating a REST service on-the-fly when rule is triggered?

Thanks!

the rest basic authentication doesn't work with settings options or user keys - klausi can you please shed some light?

function hook_wsclient_service_load($services) {

  if ($service->type == 'soap') {
    $service->settings['options']['login'] = 'user';
    $service->settings['options']['password'] = 'pass';  
  }
  elseif ($service->type == 'rest') {
    //fails.
    $service->settings['user']['user'] = 'user';
    $service->settings['user']['pass'] = 'pass'; 
  }
}

Any luck with this?

which file do we put the following options:

$service->settings['options']['login'] = 'user';
$service->settings['options']['password'] = 'pass';  

Similar issue here, I have a security certificate from an external site for a SOAP service. Obviously including that in an authentication UI would be trickier, but helpful. Since the certificate would need to live outside of the web directory for security reasons there's still a bit of manual configuration to do most likely.

Does anyone have any thoughts about #7? I'm wondering if I can write a module to hook into the system to provide the certificate, much as #7 is providing the username and password.

Yes, you should write a module and implement that hook:

<?php
// $file: mymodule.module
function mymodule_wsclient_service_load($services) {
  if ($service->type == 'soap') {
    $service->settings['options']['login'] = 'user';
    $service->settings['options']['password'] = 'pass';  
  }
  elseif ($service->type == 'rest') {
    //fails.
    $service->settings['user']['user'] = 'user';
    $service->settings['user']['pass'] = 'pass'; 
  }
}
?>

I'm going to roll wsclient 1.0 soon (finally a stable release) and while this would be a nice feature addition I don't think it should hold up a stable release.

As always: patches welcome!

klausi, could you at least provide a wsclient_examples example that includes basic http authentication? say, with JIRA https://developer.atlassian.com/display/JIRADEV/JIRA+REST+API+Example+-+...?

Here is basic implementation of authentication configuration mechanism: see attached patch. It requires the patch from #2042205: Create plugin for Basic HTTP authentication. With both of them applied, authentication could be done as follows (Recurly service is as an example, edit to meet your use case):

/**
 * Implements hook_wsclient_service_load().
 */
function recurly_wsclient_service_load($services) {
  foreach ($services as $service_id => &$service) {
    if ($service_id !== 'recurly') {
      // Skip all services besides Recurly.
      continue;
    }
    // Authentication parameters.
    $service->settings['authentication']['basic']['username'] = variable_get('recurly_api_key', '');
    $service->settings['authentication']['basic']['password'] = '';
    // Force following redirects.
    $service->settings['curl options'][CURLOPT_FOLLOWLOCATION] = TRUE;
    // Just to do it right.
    // @see Warning section at the page by the link below.
    // @link http://php.net/manual/en/control-structures.foreach.php @endlink
    unset($service);
  }
}

Please note that there is no switch for REST and SOAP service - they are configured identically.
Though it still misses UI, it is quite convinient already and is good starting point, isn't it?

why do we rely on http_client changes? We could just use the cURL options array to pass in any HTTP auth settings?

As I already said in #1: HTTP basic auth already works when you set the credentials in the options array of the service description.

Why do we rely on http_client changes? We could just use the cURL options array to pass in any HTTP auth settings?

You're right, we could - but it is not abstract enough to be convinient. Let me take as an example OAuth plugin from Http Client: http://drupalcode.org/project/http_client.git/blob/refs/heads/7.x-2.x:/i... - all of this stuff could be done as a set of separate settings, but having it as a plugin is much more better, isn't it? Our client has 2 subhandlers: formatter and authentication. We could pack each type of authentication (Basic, OAuth, Digest and so on) into its own plugin. As well as each type of formatter (PHP, url-encoded form, JSON, XML) into its own formatter plugin - see appropriate issue #1280332: Advanced REST service formatters + UI setting. Don't you think that all of this are parts of one task?
I prefer to change issue status back while waiting for your feedback.

That makes sense, but we cannot proceed here until the changes have landed in http_client.

This issue is about making HTTP Basic Auth configurable in the UI - let's stick to that first and implement any http_client improvements later (when they land).

The patch on this issue adds support for oauth2 authentication: #2138617: OAuth2 authentication for http_client

I am now one of the maintainers of the project http_client, and I have applied (committed) the required patches. I also have reviewed the patch proposed in this issue and it seems OK. So, you can proceed with applying/committing it.
Or, would you like to grant me commit access on this project?

I've pushed the *foundation* for the auth support, both htauth from Patchranger here and the OAuth from dashohoxha in as a copmbined patch, as the second one repaired some issues with the first.

Right now this gives a better API for developers calling it by code.

STILL NEEDS UI added to put this to bed.

So I guess this needs work, since there is not patch for the UI yet?

I added a module that adds username / password values to the $services variable for my soap endpoint but still getting "Error parsing the WSDL file: SOAP-ERROR: Parsing WSDL: Couldn't load from..." failed to load external entity. Module newbie here...

I need to get a SOAP webclient operational which connects to a SOAP services that needs login authentication.

So I have been trying to figure out how to do that with the above mentioned info. But unfortunately didn't succeed. Also I have been trying to implement the dev version.

For the moment it's only 1 SOAP service with several functions. Which would be the best way to get this running?

Thanks

To be more specific in what I have tried to do to get the soap incl authorization running but which failed, maybe someone can help me out.
1. I entered the patches from #14 to the modules wsclient.module, wsclient_rest.module and wsclient_soap.module.
2. I entered the hook from #14 in the wsclient.module like this (*** is authorization):

/**
* Implements hook_wsclient_service_load().
*/
function irion_wsclient_service_load($services) {
  foreach ($services as $service_id => &$service) {
    if ($service_id !== 'irion') {
      // Skip all services besides irion.
      continue;
    }
    // Authentication parameters.
    $service->settings['authentication']['basic']['username'] = variable_get('irion_api_key', '***');
    $service->settings['authentication']['basic']['password'] = '***';
    // Force following redirects.
    $service->settings['curl options'][CURLOPT_FOLLOWLOCATION] = TRUE;
    // Just to do it right.
    // @see Warning section at the page by the link below.
    // @link http://php.net/manual/en/control-structures.foreach.php @endlink
    unset($service);
  }
}

~dman added php tags for code highlighting

Testing it shows no authorization header while processing. Could anyone please tell me what I did do wrong?

Thanks

*just* from looking at your code (without reference to the hook) - I'm not sure what purpose your unset() at the end is achieving.
Folling the (helpful) link to docs you provided, I notice that that does an unset at the end (to be tidy) outside of the loop. You do yours inside the loop.
I would have expected that doing that would nullify the $service declaration altogether, right after you updated things.
AFAIK, that last unset() is just garbage collection, and would be dropped anyway when you exit the func. But I may be reading it wrong.
... OK, I see that is based on the eg from Patchranger above. My confusion stands, but it's probably not directly an issue.

apart from that I guess that - if you have already confirmed that your own code is being called at all then what you are doing to the service definition object looks generally correct.
Does your code actually trigger at all? Did you cache-clear after adding the hook to an existing module file?
I often either trace (via xdebug) or just drop a watchdog() in there to be sure the hook hasn't got a typo or need a cache-clear. Obvious stuff, but it's where I have to start when there are no obvious clues or errors.
I would also have devel on and (if not xdebugging) just drop a dpm(get_defined_vars()); at the end of the func. Cheap debug, but it throws up the obvious more than half the time.

dman,
Thanks for your feedback, great !
I am not sure about a few things though, my know-how just runs short...
1. I have entered the hook coding in wsclient.module, is that the correct place?
2. I have changed the code from Patchranger with "irion" in a few places. That is in the function (name), comparison- and variable_get statement. "irion" is the system name for the webservice in Drupal, is that correct? Especially the naming conventions and calling of hooks is unknown to me.

3. Yes, I did a "clear all caches" in Drupal before testing, hope that it will suffice.

4. I will enter a watchdog statement in the hook:

watchdog("My wsclient hook", "The wsclient irion authorization hook was called !", NULL, $severity = WATCHDOG_NOTICE, $link = NULL)

Thanks

Test result:

The test shows in the log:
Plugin Web Service Parser of plugin type feeds:plugins points to nonexistent file sites/all/modules/wsclient_feeds/FeedsWSClientParser.class.php for class handler handler.

In the wsclient test-screen (response):



soapenv:Server
WSDoAllReceiver: Incoming message does not contain required Security header

1. No
2. Mostly. Yeah hook naming conventions are magic, but you guessed right.

The (admittedly lightweight) example from patchranger there makes a few common assumptions on writing module/extensions/hooks inside Drupal that are not spelled out.

* You just writing a function named after the hook (which you did correctly) is not enough to make that hook fire.
* The hook needs to be part of your own module with your own name that contains that hook function.
... that's probably too much to go into in this thread, sorry. Writing your own Drupal7 module

* But in short, You do NOT edit the wsclient files directly!
* you make your own mini-module in a folder inside sites/all/modules called "irion".
* You place your code above (which is actually probably just right) into sites/all/modules/irion/irion.module as code
* You add a sites/all/modules/irion/irion.info file per instructions (3 line text file)
* you enable that module now that it shows up on your sites modules admin page.

- in brief, at that point you have written your first module that implements 1 hook and modifies the behaviour of an existing process!
Importantly - your local code tweak is separate from the provided module code, but they co-operate.
It's not appropriate to go into this side of things deeper in this existing thread - but it is documentation-ish stuff that was assumed (and implicitly understood by devs) in comment #14.

UI and/or docs are still lacking here, so I've gone into a little more detail to fill in the needed HOWTO doc around this hidden feature.

dman,
I'm sorry for the basic characteristics of my problems... I'll do my best to learn quickly :-)
Thanks, wonderful that you wanted to tell me how to do this, it was very helpful and watchdog now tells me the coding was called!, yes!

But unfortunately wonders didn't come for free...
No more "test"-label per operation in the wsclient page now.
And 2 errors in the log:
- Plugin Web Service Parser of plugin type feeds:plugins points to nonexistent file sites/all/modules/wsclient_feeds/FeedsWSClientParser.class.php for class handler handler.
- Plugin Web Service Fetcher of plugin type feeds:plugins points to nonexistent file sites/all/modules/wsclient_feeds/FeedsWSClientFetcher.class.php for class handler handler.

What could be the matter here?

If wsclient testing is now returning results,
if you can now use the tester UI to return any type of successful responses to your queries, and its not complaining about authentication at all any more.

1. yay!
2. this is not a question for this thread. Open a new issue.

if it's wsclient_feeds that is being indicated by this error, go describe what you are doing over in the issue queue there. (that's actually my module, but we need details on what you did to install or delete it. Feeds and feeds_wsclient is a whole big layer around just getting wsclient to work, don't run before you can walk)
It's a different issue, specific to your case now.

dman,

1. I'm a bit confused on what the problem might be. I didn't get any errors from the feeds modules just up to the moment I started testing with these #14 patches in wsclient. And I have made no changes to the feeds installations. So it looks like the wsclient modules are now starting to interfere with the feed modules, but how? When the #14 patches from this thread are removed from the modules, the feed errors are also no longer generated.

2. Testing with the wsclient test module seems to be deactivated now as there is no test label to click on in the webservice page with the webservice operations, so I can't test the webservice operation.

I think it's the patches in the wsclient module, wsclient rest module and wsclient soap module, together with the hook that are interfering with the feed and wsclient test module, but I can't figure out how?

3. Should I install the newest dev version? I think that includes the authorization patches also?

1. deactivate and uninstall wsclient_feeds until you can first get wsclient working right first. I can't guess what the issue there is (you've not described why you are using it), but as you are posting it as errors, it's getting in the way of your debugging.

2. I'm not sure why you would find the test button deactivated - that is not functionality that even is part of the thing. But I do slightly suspect the fact that you unset() the $service in your hook (that only now has started 'working'.

3. Yes. As this issue is marked 7.x-1.x-dev, and the auth support described here was already added to -dev back in #20, then you really should be using the latest code please.

It's really hard to play guessing games without any idea of what you are actually doing, what you have already done, what you are seeing, and what you are trying to do. So it would make a huge difference if you could post the actual web service you are using and the details you are sending it. You are not being at all clear about what you have done so far, and what you expect to see.
I know that often these web services are sort of restricted, but if we could access a safe version of it, then there may be a chance to troubleshoot . Without the ability to test or even imagine what you are doing, support is 20 times harder.
I can only offer to help access good public open data web services. Closed or proprietary web services you have to sort out yourself, because they just cannot be debugged from outside. There is way too much guessing needed.

dman,
You're right and I think possibly too much things cached, interfered and collided, so I did a restore to yesterday's status, now I can start from "scratch".

What we are trying to do?
We want to use the operations at http://services.irion.nl/services/cls?wsdl in our portal. This operation includes intelligent indexing of documents and articles. The documents and articles are imported with the feeds and feeds-tamper module. With the rules and webservice modules, the imported, stored and/or changed document/articles will be presented to the irion webservice for intelligent indexing. These indexes will be stored in a tags field.
For making use of the irion soap operations, an userid/password authorisation needs to be provided in a XML security header, preferably in a scrambled format.

I'll start uninstalling wsclient_feeds and will install and test the dev version, together with the new irion module.

That was a good strategy, the cleaned system now operates as expected, thanks!
Now we have the wsclient dev version and the http_client dev version operational, including our (see #24) irion hook program as in #14 by PatchRanger.
The hook is checked to be called and the $service_id is checked to be "irion" via watchdog. So far so good.
But.. the following error is generated by the remote service:
WSDoAllReceiver: Incoming message does not contain required Security header
I tested it with and without the unset() call.
If I test the same remote service with SoapGui it does operate flawless, so no problem in the remote service.

Any things I overlook to get wsclient-dev to include a Security header in the soap?

At this point, it would seem that what your endpoint calls a "Security header" may not be the same thing as what this thread is about - which is "HTTP basic authentication".
There are innumerable other ways to do authentication. "HTTP basic" is just one of the oldest and cheapest to support.

I could not say with certainty that the message you report "Incoming message does not contain required Security header" is even related to this functionality at all.
Though the fact that you had a username and password to use, does make it a reasonable first guess.

You would have saved us a lot of time if you had some documentation about how to use the web service from the original provider - They should have provided some instructions at least.

A possibility is that it implements security in a different way. Heck, maybe it just expects the cURL header parameters to be capitalized differently or something (I *have* seen a broken service with something like that before now)

So, I searched the web for reports of the error message you are getting and all the results were talking about another type of authentication.
Without going into technical details about "Web Services Security", I can see that it's *not the same as* what we have been doing here.

Come and join me over in #2420779: Support WSS (Web Services Security) authentication

Adding UI for setting authentication username and password. Both htauth and OAuth 2

I tried the patch from #36 by dman, but it didn't work.

First of all, the settings are saved in a flat (sub)array while the code expects it to be in a subarray with the authentication type as the key.
Line 44 of wsclient_rest/wsclient_rest.module:

foreach ($this->service->settings['authentication'] as $auth => $auth_settings) {

Then, if the type is oauth2, then client_id, client_secret and token_endpoint are necessary, instead of username and password. None of these options are available.

The attached patch has a more complete UI settings form.

I tested this using an oauth2-secured web service. So review (and testing) may be needed for basic http authentication.

Seems like the patch from #14 is also no longer needed, since it is not about adding settings to the UI, and because it has been committed to the dev version via another ticket (if I read/understood the notes correctly).

Thanks for the additions of OAuth2 settings. We've not got ability to test that, but the UI form additions look helpful.

Re-roll, plus some fixes for undefined indexes, and we can't have the optional OAuth2 field 'required' inside the hidden fieldset - without some extra work anyway, so I gotta disable that validation for a bit.
Looks like lots of progress though.
While we are here, I will also add the WSS authentication #2420779: Support WSS (Web Services Security) authentication that fit here also.

Re-roll with Minor cleanup for labels and undefined indexes.

Added WSS authentication option to the UI as well

Because of the commit in #1280332: Advanced REST service formatters + UI setting, the patch(es) from this issue did no longer work. This is because this patch adds a form field "settings", and the commit in #1280332: Advanced REST service formatters + UI setting alters the form_state to add the formatters to the service settings. Those alteration are overridden by our settings field in wsclient_service_form_submit() by entity_ui_form_submit_build_entity().

I've attached a new patch, which fixes this. It adds the new formatter setting fields also below our setting field, which also removes the need to alter the form_state in wsclient_rest_main_form_submit().

Since this issue is no longer just about basic http authentication, I've changed the issue title to reflect this.

Thanks for the TESTING - and the explanation.
I see the wsclient_rest_main_form_submit() function that does indeed overwrite our $settings. Looks a little shoehorned in there - I think that the rework you've done on the UI form there is better overall.

I was worried about the size and side-effects of that patch over there, but OTOH, had to get that (and a few other pending ones) it in so we could approach some finality with these other ones.
The only service I've got access to test auth on, was a SOAP one.. and even that I haven't got access to test directly against from here. It's a bit blind.
Patch here *looks* OK, but I wish I could imagine how to really do automated tests here.

I've been testing OAuth2. I googled a bit and found http://www.httpwatch.com/httpgallery/authentication/. This site has an image which is protected with http authentication (see example 10). If you use the url which creates the image (http://www.httpwatch.com/httpgallery/authentication/authenticatedimage/d...), you could test it via wsclient.

Perhaps you'll find a oauth2 playground as well...

You were talking about automated tests. Sorry misread that...

For basic http authentication, I guess you could perhaps create a test url which is protected by http authentication inside the wsclient module, which IMO should be fairly easy to setup, and test that in an automated test.

For OAuth2 it might be a bit more difficult. I have no experience with setting up an OAuth2 server... If I find the time, I could take a stab at it, but should we really wait for this to have this patch in the dev version? Perhaps there should be a separate ticket for the automated tests? Either way, I can confirm OAuth2 (2-legged authentication) works.

Hi, people!
I solved this problem of authorisation with Soap from code.
First, you ned to enable modules: Entity API, Web service client, Web service client SOAP.
I took this method of soap service initialisation from example module and added login and passoword thruogh the "settings"->"options" data hierarchy. It works, gues!!!

      ini_set('display_errors','On');
      ini_set("soap.wsdl_cache_enabled", "0");

      $service = new WSClientServiceDescription();
      $service->name = 'SoapTestService';
      $service->label = 'Soap Test Service';
      $service->settings = array ('options' => array ( 'login' => 'somelogin', 'password' => 'somepass',),); 
      $service->url = 'http://109.195.230.156:8080/L11/ws/ozenki.1cws?wsdl';
      $service->type = 'soap';
      try {
        $service->endpoint()->initializeMetaData();
        $services[$service->name] = $service;
      }
      catch (WSClientException $e) {
          watchdog('wsclient', $e->__toString());
      }                      
      //ок, authorisation is successfull
       $params["Tel"] = "9281705666";
       $params["Pass"] = "000011";
       $params["DataN"] = "2012-01-01";
       $params["DataK"] = "2012-02-28";  
       $Items = $service->GetData($params);  //here we call method of a service

Thanks for the example. Looks like that will be useful to someone

@dman: Are you still waiting for some automated tests?

Well, waiting-for/slowly-creeping-towards-a-way-to-build-them .
There's just so many moving parts to automate before I can demo a real web service transaction in a stand-alone environment that it's all a bit hard for me to scope.
Yet regressions when we change any part of this system are so dangerous and subtle that I really want to be able to have some justified confidence in what it is so far.

Patch #42 deletes the submission line :

@@ -202,7 +211,6 @@ class WSClientRESTEndpoint extends WSClientEndpoint {
         }
       }
     }
-    $form['#submit'][] = 'wsclient_rest_form_submit';
   }
 
   /**

With this your Pad (The path to append to the services base URL) wouldn't be saved and your service-clients won't work.
So, add this line again to your code.

Related to this is a patch to add OAuth v1 support: #2272841: Adding support for OAuth 1.0 authentication

@pbosmans: you're right. Patch fixed.

WIP to add OAuth v1 support using the code from #2272841: Adding support for OAuth 1.0 authentication that needs work.

Some minor improvements to the new fields - their labels are now written in proper English and passed through t().

Additional minor improvements to the form labels.

So submitting the settings form lead to problems - the token_endpoint fields had a validator that would fail if they were left empty, and the form threw an error because wsclient_rest_main_form_submit was removed.

Oh, the patch in #56 also started adding a callback for OAuth, but it definitely needs work.

I removed the OAuth 1 integration as it was starting go down a rabbit hole that I didn't end up needing after all.

This is just a continuation of the UI changes for the existing authentication mechanisms.

We'll just have to add oauth1 support later if needed. Would be a nice to have on #2272841: Adding support for OAuth 1.0 authentication if this goes in first.

OK. Sorry this has been so stagnant.

As I posted long ago, I was worried about applying patches that:
* I can't test if they work as promised.
* I can't test if they break anything for existing uses.

But hey, what's the worst that can happen, eh?

Currently:
* Does not explode for me.
* Appears to offer the desired options.
* Docs and visual inspection imply it should work.
* Reports above confirm it works for some folk.

This quite-nice-looking patch still applies clean (enough) and is also a blocker to #2272841: Adding support for OAuth 1.0 authentication so meh, let's just roll it in and build from there with follow-ups.
I suspect we may see a few floating issues with error-handling or undefined indexes possible, especially for upgraders - but nothing that won't go away with a form re-save. Hopefully.

Not the *worst* that can go wrong, but this patch does change the nesting of the expected $service['authentication'] array settings to allow for multiple auth types keyed by type, instead of one settings array determined by type.
So that's an annoyance of an upgrade path needed there I think. Still, was necessary in order to support that UI, and eliminate the undefined indexes.
Will have to be in release notes at the very least, unless we get around to doing an update hook.