Problem/Motivation

We're in the early stages of upgrading farmOS from D7 to D9. An early step in this process is migrating from the oauth2_server module to using simple_oauth. One of the major differences between the two modules is the Drupal entity used to model the OAuth Client. oauth2_server uses an oauth2_client entity and `simple_oauth` uses a consumer entity. The major difference between these entities is that `consumer`s use the UUID as the OAuth client_id, while oauth2_client has a string field to define the client_id.

This is slightly problematic for farmOS because we need the client_id to be unique across multiple farmOS servers. This stems from the fact that there is no central repository for farmOS 3rd party clients; rather than the 3rd party registering a client on every farmOS server, we require 3rd parties to provide a Drupal module that defines an OAuth Client with a client_id unique to their integration. Installing this module enables users to authorize 3rd parties access to their farm data. While it might be possible to maintain a unique UUID across multiple instances of farmOS, a text string would provide better readability/DX and maintain consistency with our API client libraries. The client libraries default to using the farm client ID, and our 1st party PWA mobile app (which can connect to any farmOS server) uses the farm_client client ID.

Adding this functionality would make it possible to validate clients via other attributes in the future (such as validating clients by DNS name) This video provides a great overview of the distributed & self-hosted OAuth problems and proposes some solutions: https://youtu.be/E4msDjZMRZc?t=3457

Steps to reproduce

It is currently almost possible to implement this functionality without modifying the simple_oauth module. Here are the required steps:

  • Create a custom consumer.client_id string field to save the client ID used for OAuth.
  • Override the simple_oauth.repositories.client service to use a custom ClientRepository class. Modify the getClientEntity method to load consumer entities by client_id rather than UUID.

This would work if the ClientRepository->getClientEntity() method was used consistently. Unfortunately the consumer entities are hard coded to be loaded by the UUID in the simple_oauth Oauth2Token, Oauth2AuthorizeController and Oauth2AuthorizeForm controllers. Since each of these classes has an instance of the Oauth2GrantManager (which has a reference to the ClientRepository class), it should be possible to always use the ClientReposiory->getClientEntity() method for loading consumer entities.

Proposed resolution

Modify all the OAuth Controllers to use the ClientRepository->getClientEntity() method from the saved instance of the Oauth2GrantManager (this would require making the ClientRepository a public property on the Oauth2GrantManager) OR load the simple_oauth.repositories.client service separately within each OAuth controller.

Remaining tasks

This issue describes a problem where fields added via hook_consumers_list_alter() are not shown correctly in the UI.

Also as pointed out in this issue: the ClientRepository class should probably move some of the validation logic from getClientEntity() to the validateClient() method.

API changes

Furthering upon that change to ClientRepository, it would be nice to save the consumer client ID field name within the ClientRepository class eg: $this->consumerClientIdFieldName. This would make it easier to extend the base ClientRepository class and only modify that variable, rather than replacing the entire getClientEntity() method. Better yet, perhaps this variable could be provided via config!

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

paul121 created an issue. See original summary.

paul121’s picture

Issue summary: View changes
paul121’s picture

StatusFileSize
new7.07 KB

This patch implements the bare minimum functionality by loading the ClientRepository via the grantManager property that is accessible to all of the OAuth Controllers. This required making the $clientRepository a public property.

paul121’s picture

StatusFileSize
new7.26 KB

This patch implements the same functionality but by loading the ClientRepository via the Drupal::service('simple_oauth.repositories.client') method within each OAuth controller. It seems like this may be less efficient than going through the grantManager class, but doesn't require making the $clientRepository apublic property.

paul121’s picture

StatusFileSize
new10.63 KB

This patch builds upon the 3167287-3.patch to make the consumer client_id field name configurable via the simple_oauth.settings.consumer_client_id_field_name config value (and defaults to uuid).

This was easier to implement than I thought it would be! Having this available in config is much easier than overriding and extending the simple_oauth.repositories.client service to change the client_id that is used.

paul121’s picture

Note that the Drupal 9.1 tests that are failing are due to this issue: https://www.drupal.org/project/simple_oauth/issues/3167509

paul121’s picture

Looking into it further I realized that the validateClient() method that is documented for league/oauth2-server was only introduced in the most recent version of the library. There is another issue open to consider upgrading the library to 8.0. The main catch is that there are some breaking changes in the library.

Overall the upgrade seems quite sensible, but shouldn't be considered as blocking this feature request.

paul121’s picture

Status: Active » Needs review

The last submitted patch, 3: 3167287-3.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

paul121’s picture

Title: Support using custom fields as the consumer client_id » Always load clients through the ClientRepository service
Version: 8.x-4.x-dev » 5.x-dev
StatusFileSize
new13.36 KB

I've re-rolled this patch to work with the recent 5.x release. The only change this patch makes is ensuring that OAuth Clients are always loaded via the simple_oauth.repositories.client. service. I've injected the service as a dependency in the required controllers; I believe I found everywhere this is needed?

This is the bare minimum required to implement the client_id features discussed above. Without this, code must be changed in multiple places to in order to modify behavior that I believe the ClientRepository service should be solely reliable for. Updating the issue title to better reflect these changes.

bradjones1’s picture

Category: Feature request » Task
m.stenta’s picture

StatusFileSize
new12.06 KB
new1.13 KB

Re-rolled the patch from #10 to work with 5.1.0-beta1. Attached is the new patch and an interdiff.

@paul121 I would love your eyes on this if you have a chance to make sure I didn't miss anything...

The patch in #10 was modifying ClientRepository.php, but it *seems* that those modifications (the end result of them at least) have been incorporated via #3173947: Cannot authorize non-confidential clients and #3230707: 5.x broken on php 8 due to incompatibility with lcobucci/jwt v4 via league/oauth2-server ^8.2 (the two commits made to the repo since 5.0.5).

That was the only conflict that was occurring, so this updated patch makes the *other* three changes, but does not touch ClientRepository.php. Hopefully that's correct and I didn't miss anything important.

m.stenta’s picture

Status: Needs review » Needs work

Hmm well, it looks like our farmOS tests are failing in relation to ClientRepository.php with this patch applied, so I think I probably missed something...

Exception: TypeError: Drupal\simple_oauth\Entities\ClientEntity::__construct(): Argument #1 ($entity) must be of type Drupal\consumers\Entity\Consumer, null given, called in /opt/drupal/web/modules/simple_oauth/src/Repositories/ClientRepository.php on line 34

Setting this to "Needs work"...

paul121’s picture

Interesting. So in farmOS we are overriding the ClientRepository service so that we can modify ClientRepository::getClientEntity() function to load entities by a string instead of UUID. It seems this error happened when testing an incorrect client_id: https://github.com/farmOS/farmOS/blob/059e0af8d978b6dca646be3e1a4e252c75...

One of the latest commits to simple_oauth introduced a public function getClientDrupalEntity(string $client_identifier) helper function that we will probably want to use within farmOS: https://git.drupalcode.org/project/simple_oauth/-/commit/7ed0076c7bd5485...

But I think a bug was introduced with this new helper function. The ClientRepositoryInterfeace::getClientEntity() function should return ClientEntityInterface|null, but the latest change has it *always* returning a ClientEntityInterface, which will error when getClientDrupalEntity() returns NULL. The current annotation for this additional function is incorrect as well.

At the very least I think an empty($client_drupal_entities) check needs to be moved and/or added to the getClientEntity() function.

  /**
   * {@inheritdoc}
   */
  public function getClientEntity($client_identifier) {
    return new ClientEntity($this->getClientDrupalEntity($client_identifier));
  }

  /**
   * Get the client Drupal entity.
   *
   * @param string $client_identifier
   *   Client ID.
   *
   * @return \Drupal\consumers\Entity\Consumer
   *   The loaded drupal consumer (client) entity.
   */
  public function getClientDrupalEntity(string $client_identifier) {
    $client_drupal_entities = $this->entityTypeManager
      ->getStorage('consumer')
      ->loadByProperties(['uuid' => $client_identifier]);

    // Check if the client is registered.
    if (empty($client_drupal_entities)) {
      return NULL;
    }
    return reset($client_drupal_entities);
  }

  /**
   * @{inheritdoc}
   */
  public function validateClient($client_identifier, $client_secret, $grant_type) {
    if (!$client_drupal_entity = $this->getClientDrupalEntity($client_identifier)) {
      return FALSE;
    }
    $secret_field = $client_drupal_entity->get('secret');

    // Determine whether the client is public. Note that if a client secret is
    // provided it should be validated, even if the client is non-confidential.
    // The client_credentials grant is specifically special-cased.
    // @see https://datatracker.ietf.org/doc/html/rfc6749#section-4.4
    if (!$client_drupal_entity->get('confidential')->value &&
      $secret_field->isEmpty() &&
      empty($client_secret) &&
      $grant_type !== 'client_credentials') {
      return TRUE;
    }

    // Check if a secret has been provided for this client and validate it.
    // @see https://datatracker.ietf.org/doc/html/rfc6749#section-3.2.1
    return (!$secret_field->isEmpty())
      // The client secret may be NULL; it fails validation without checking.
      ? $client_secret && $this->passwordChecker->check($client_secret, $client_drupal_entity->get('secret')->value)
      : TRUE;
  }

paul121’s picture

StatusFileSize
new13.85 KB
new1.79 KB

Okay, I was confused why this wasn't being caught by the simple_oauth tests but now it makes sense. Because the ClientRepository service is *not always used to load clients*, there are numerous places where other safety checks are performed. As soon as we move forward with this issue it makes sense that we need to move these safety checks into the ClientRepository service. But I think this is good and further demonstrates the motivation behind this issue!

But I think a bug was introduced with this new helper function. The ClientRepositoryInterfeace::getClientEntity() function should return ClientEntityInterface|null, but the latest change has it *always* returning a ClientEntityInterface, which will error when getClientDrupalEntity() returns NULL. The current annotation for this additional function is incorrect as well.

So... considering this, and looking closer at #10 again, I'd like to propose we use that approach for the ClientRepository::validateClient() *instead* of adding the getClientDrupalEntity() helper function altogether.

I think we've forgotten about the ClientEntityInterface::getDrupalEntity() function that can be used instead.

Adding a patch with this approach.

bradjones1’s picture

Thanks - yes, this kind of clean up will help us make the module more secure. Please mark NR when it's ready for me to take a look.

m.stenta’s picture

Status: Needs work » Needs review

This new patch fixed the error I described in comment #13 above in farmOS tests! Thanks @paul121!

We are getting some other test failures in our API/OAuth tests still, but they may be unrelated to this issue. I will set this back to "Needs Review" so bradjones1 can take a look at the patch while we debug our other failures. :-)

bradjones1’s picture

OK thanks.

I will try to get the tests addressed - might disable the install platform requirements check as a stopgap, even though I don't love it.

If you could post information about the test failures, that would help - I have been validating locally on PHP 8 as I've been rolling the beta, but I agree it would be good to get automated testing working again prior to making the next stable release.

m.stenta’s picture

Oh sorry I wasn't clear @bradjones1 - I was referring to farmOS tests that are failing - not Simple OAuth: https://github.com/mstenta/farmOS/runs/4606036865?check_suite_focus=true

bradjones1’s picture

Thanks. I will try to look at this early next week. Planning to release 5.1.0 this week, then this can go on 5.1.1.

paul121’s picture

Okay, here's to revitalizing this issue. There's a lot of info above: we added new patches in #10 #12 and #15 to keep up with changes to simple_oauth 5.x. Comments above are largely just figuring out what those changes were and how it affected our patches. The original issue description is still quite accurate. But just to reset and highlight some of the motivations for this issue:

From #10 (true of this MR as well)

This is the bare minimum required to implement the client_id features discussed above. Without this, code must be changed in multiple places to in order to modify behavior that I believe the ClientRepository service should be solely reliable for.

From issue description:

This would work if the ClientRepository->getClientEntity() method was used consistently. Unfortunately the consumer entities are hard coded to be loaded by the UUID in the simple_oauth Oauth2Token, Oauth2AuthorizeController and Oauth2AuthorizeForm controllers. Since each of these classes has an instance of the Oauth2GrantManager (which has a reference to the ClientRepository class), it should be possible to always use the ClientReposiory->getClientEntity() method for loading consumer entities.

Thus; the proposed solution needs to do two things:
1) Use dependency injection to ensure the ClientRepository simple_oauth.repositories.client service is consistently used in all of the controller, form and manager classes. (this is the bulk of the changes in the diff)
2) Ensure ClientRepository::getClientEntity() is working correctly (essential - it should be the single method used to get clients). As identified in #14 I believe the new ClientRepository::getClientDrupalEntity() function introduces a bug in the event the Drupal entity is not loaded. Instead of this new "helper" method, we should just use the existing ClientEntityInterface::getDrupalEntity() function to get the Drupal entity. See patch for how this is resolved, it is quite a simple change.

I've opened a new merge request with the patch from #15 rebased onto the latest 5.2.x branch. @bradjones1 if you could please advise on how to run tests for this MR? In the past I was able to run the simple_oauth tests on my local but now they are failing (even without any of these modifications to 5.2.x). These failures seem to be the same failures that we are getting when running the farmOS API tests with patch #15 applied so there must be something related.

Test result (running phpunit in our docker container - yes this might not be the best way to do this, but used to work? For reference, here are the related farmOS test failures: https://www.drupal.org/project/farm/issues/3282186#comment-14536882)

$ phpunit /opt/drupal/web/modules/simple_oauth
PHPUnit 9.5.20 #StandWithUkraine

Warning:       Your XML configuration validates against a deprecated schema.
Suggestion:    Migrate your XML configuration using "--migrate-configuration"!

Testing /opt/drupal/web/modules/simple_oauth
................F....................EEE.........                 49 / 49 (100%)

Time: 04:25.714, Memory: 20.00 MB

There were 3 errors:

1) Drupal\Tests\simple_oauth\Functional\RolesNegotiationFunctionalTest::testRequestWithRoleRemovedFromUser
Trying to access array offset on value of type null

/opt/drupal/web/modules/simple_oauth/tests/src/Functional/RolesNegotiationFunctionalTest.php:160
/opt/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

2) Drupal\Tests\simple_oauth\Functional\RolesNegotiationFunctionalTest::testRequestWithRoleRemovedFromClient
Trying to access array offset on value of type null

/opt/drupal/web/modules/simple_oauth/tests/src/Functional/RolesNegotiationFunctionalTest.php:220
/opt/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

3) Drupal\Tests\simple_oauth\Functional\RolesNegotiationFunctionalTest::testRequestWithMissingScope
Trying to access array offset on value of type null

/opt/drupal/web/modules/simple_oauth/tests/src/Functional/RolesNegotiationFunctionalTest.php:276
/opt/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

--

There was 1 failure:

1) Drupal\Tests\simple_oauth\Functional\PasswordFunctionalTest::testPasswordGrant
Failed asserting that 500 matches expected 200.

/opt/drupal/vendor/phpunit/phpunit/src/Framework/Constraint/Equality/IsEqual.php:96
/opt/drupal/web/modules/simple_oauth/tests/src/Functional/TokenBearerFunctionalTestBase.php:167
/opt/drupal/web/modules/simple_oauth/tests/src/Functional/PasswordFunctionalTest.php:81
/opt/drupal/vendor/phpunit/phpunit/src/Framework/TestResult.php:726

ERRORS!
Tests: 49, Assertions: 366, Errors: 3, Failures: 1.
bradjones1’s picture

Dumb comment bump to see if it triggers MR testing...?

bojan_dev made their first commit to this issue’s fork.

bojan_dev’s picture

For the 5.2.x this change makes sense, but for the 6.0.x I would like to address this in the data model by adding a new BaseFieldDefinition (string): 'client_id' on the consumer entity and replace all uuid loading logic. Will be working on this soon.

bojan_dev’s picture

Version: 5.x-dev » 6.0.x-dev
Status: Needs review » Fixed
paul121’s picture

Status: Fixed » Needs work

Sorry @bojan_dev I wasn't able to review your merge request before it got merged. I'm glad we're switching to using the client_id on the consumer entity as that was the original motivation for this issue. However, I originally opened this issue to address a more generic issue that we should always be using the client repository to load the consumer entity. It wasn't clear from your comment in #27 if this was the approach you would take.

With your change there still are multiple places where the logic is hard-coded to load consumers by their client_id. It's my understanding the client repository service exists as an abstraction to consolidate this logic into a single place (please correct me if this has changed in 6.x). I agree with #16 that consistently using this service will make this more secure and maintainable going forward.

Marking as "Needs work" as I don't think the original issue is "Fixed". IMO we need to update the issue/description as to what was actually changed, mark "Closed (won't fix)", or agree to implement a solution to the original issue.

@bojan_dev I'm happy to provide a MR for this, just want your opinion before I start anything. It would be a great opportunity for myself to look closer at the 6.x code and implementation which I just have not had the time to do.

bojan_dev’s picture

With your change there still are multiple places where the logic is hard-coded to load consumers by their client_id. It's my understanding the client repository service exists as an abstraction to consolidate this logic into a single place (please correct me if this has changed in 6.x). I agree with #16 that consistently using this service will make this more secure and maintainable going forward.

This is a valid point and I agree we should leverage the ClientRepository to centralize the loading of clients/consumers. In 6.0.x the usage of loading consumers by client_id has been reduced, but is still in place for the Oauth2AuthorizeController and Oauth2Token.

So feel free to provide a MR, and it would be great to have more input on the 6.0.x code.

bojan_dev’s picture

Version: 6.0.x-dev » 5.2.x-dev
Status: Needs work » Needs review

I have addressed the fix for #3316519: Token request broken consumer load by client id changed in this MR: https://git.drupalcode.org/project/simple_oauth/-/merge_requests/31, this issue is related and it had already work done by Paul.

A review would be welcome.

paul121’s picture

Status: Needs review » Reviewed & tested by the community

Thanks @bojan_dev this LGTM! I'm not able to test too extensively at the moment locally but did include this patch in our farmOS automated tests and it worked.

  • bojan_dev committed db7ec32 on 5.2.x authored by paul121
    Issue #3167287: Always load clients through the ClientRepository service
    

bojan_dev’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.