Problem/Motivation
We're in the early stages of upgrading farmOS from D7 to D9. An early step in this process is migrating from the oauth2_server module to using simple_oauth. One of the major differences between the two modules is the Drupal entity used to model the OAuth Client. oauth2_server uses an oauth2_client entity and `simple_oauth` uses a consumer entity. The major difference between these entities is that `consumer`s use the UUID as the OAuth client_id, while oauth2_client has a string field to define the client_id.
This is slightly problematic for farmOS because we need the client_id to be unique across multiple farmOS servers. This stems from the fact that there is no central repository for farmOS 3rd party clients; rather than the 3rd party registering a client on every farmOS server, we require 3rd parties to provide a Drupal module that defines an OAuth Client with a client_id unique to their integration. Installing this module enables users to authorize 3rd parties access to their farm data. While it might be possible to maintain a unique UUID across multiple instances of farmOS, a text string would provide better readability/DX and maintain consistency with our API client libraries. The client libraries default to using the farm client ID, and our 1st party PWA mobile app (which can connect to any farmOS server) uses the farm_client client ID.
Adding this functionality would make it possible to validate clients via other attributes in the future (such as validating clients by DNS name) This video provides a great overview of the distributed & self-hosted OAuth problems and proposes some solutions: https://youtu.be/E4msDjZMRZc?t=3457
Steps to reproduce
It is currently almost possible to implement this functionality without modifying the simple_oauth module. Here are the required steps:
- Create a custom
consumer.client_idstring field to save the client ID used for OAuth. - Override the
simple_oauth.repositories.client serviceto use a custom ClientRepository class. Modify thegetClientEntitymethod to load consumer entities byclient_idrather thanUUID.
This would work if the ClientRepository->getClientEntity() method was used consistently. Unfortunately the consumer entities are hard coded to be loaded by the UUID in the simple_oauth Oauth2Token, Oauth2AuthorizeController and Oauth2AuthorizeForm controllers. Since each of these classes has an instance of the Oauth2GrantManager (which has a reference to the ClientRepository class), it should be possible to always use the ClientReposiory->getClientEntity() method for loading consumer entities.
Proposed resolution
Modify all the OAuth Controllers to use the ClientRepository->getClientEntity() method from the saved instance of the Oauth2GrantManager (this would require making the ClientRepository a public property on the Oauth2GrantManager) OR load the simple_oauth.repositories.client service separately within each OAuth controller.
Remaining tasks
This issue describes a problem where fields added via hook_consumers_list_alter() are not shown correctly in the UI.
Also as pointed out in this issue: the ClientRepository class should probably move some of the validation logic from getClientEntity() to the validateClient() method.
API changes
Furthering upon that change to ClientRepository, it would be nice to save the consumer client ID field name within the ClientRepository class eg: $this->consumerClientIdFieldName. This would make it easier to extend the base ClientRepository class and only modify that variable, rather than replacing the entire getClientEntity() method. Better yet, perhaps this variable could be provided via config!
| Comment | File | Size | Author |
|---|---|---|---|
| #15 | interdiff_12-15.txt | 1.79 KB | paul121 |
| #15 | 3167287-15.patch | 13.85 KB | paul121 |
| #12 | interdiff_9-12.txt | 1.13 KB | m.stenta |
| #12 | 3167287-12.patch | 12.06 KB | m.stenta |
| #4 | 3167287-4.patch | 7.26 KB | paul121 |
Issue fork simple_oauth-3167287
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
paul121 commentedComment #3
paul121 commentedThis patch implements the bare minimum functionality by loading the
ClientRepositoryvia thegrantManagerproperty that is accessible to all of the OAuth Controllers. This required making the$clientRepositoryapublicproperty.Comment #4
paul121 commentedThis patch implements the same functionality but by loading the
ClientRepositoryvia theDrupal::service('simple_oauth.repositories.client')method within each OAuth controller. It seems like this may be less efficient than going through thegrantManagerclass, but doesn't require making the$clientRepositoryapublicproperty.Comment #5
paul121 commentedThis patch builds upon the 3167287-3.patch to make the consumer client_id field name configurable via the
simple_oauth.settings.consumer_client_id_field_nameconfig value (and defaults touuid).This was easier to implement than I thought it would be! Having this available in config is much easier than overriding and extending the
simple_oauth.repositories.clientservice to change theclient_idthat is used.Comment #6
paul121 commentedNote that the Drupal 9.1 tests that are failing are due to this issue: https://www.drupal.org/project/simple_oauth/issues/3167509
Comment #7
paul121 commentedLooking into it further I realized that the validateClient() method that is documented for league/oauth2-server was only introduced in the most recent version of the library. There is another issue open to consider upgrading the library to 8.0. The main catch is that there are some breaking changes in the library.
Overall the upgrade seems quite sensible, but shouldn't be considered as blocking this feature request.
Comment #8
paul121 commentedComment #10
paul121 commentedI've re-rolled this patch to work with the recent 5.x release. The only change this patch makes is ensuring that OAuth Clients are always loaded via the
simple_oauth.repositories.client.service. I've injected the service as a dependency in the required controllers; I believe I found everywhere this is needed?This is the bare minimum required to implement the
client_idfeatures discussed above. Without this, code must be changed in multiple places to in order to modify behavior that I believe theClientRepositoryservice should be solely reliable for. Updating the issue title to better reflect these changes.Comment #11
bradjones1Comment #12
m.stentaRe-rolled the patch from #10 to work with 5.1.0-beta1. Attached is the new patch and an interdiff.
@paul121 I would love your eyes on this if you have a chance to make sure I didn't miss anything...
The patch in #10 was modifying
ClientRepository.php, but it *seems* that those modifications (the end result of them at least) have been incorporated via #3173947: Cannot authorize non-confidential clients and #3230707: 5.x broken on php 8 due to incompatibility with lcobucci/jwt v4 via league/oauth2-server ^8.2 (the two commits made to the repo since5.0.5).That was the only conflict that was occurring, so this updated patch makes the *other* three changes, but does not touch
ClientRepository.php. Hopefully that's correct and I didn't miss anything important.Comment #13
m.stentaHmm well, it looks like our farmOS tests are failing in relation to
ClientRepository.phpwith this patch applied, so I think I probably missed something...Setting this to "Needs work"...
Comment #14
paul121 commentedInteresting. So in farmOS we are overriding the
ClientRepositoryservice so that we can modifyClientRepository::getClientEntity()function to load entities by a string instead of UUID. It seems this error happened when testing an incorrect client_id: https://github.com/farmOS/farmOS/blob/059e0af8d978b6dca646be3e1a4e252c75...One of the latest commits to simple_oauth introduced a
public function getClientDrupalEntity(string $client_identifier)helper function that we will probably want to use within farmOS: https://git.drupalcode.org/project/simple_oauth/-/commit/7ed0076c7bd5485...But I think a bug was introduced with this new helper function. The
ClientRepositoryInterfeace::getClientEntity()function should returnClientEntityInterface|null, but the latest change has it *always* returning aClientEntityInterface, which will error whengetClientDrupalEntity()returns NULL. The current annotation for this additional function is incorrect as well.At the very least I think an
empty($client_drupal_entities)check needs to be moved and/or added to thegetClientEntity()function.Comment #15
paul121 commentedOkay, I was confused why this wasn't being caught by the simple_oauth tests but now it makes sense. Because the ClientRepository service is *not always used to load clients*, there are numerous places where other safety checks are performed. As soon as we move forward with this issue it makes sense that we need to move these safety checks into the ClientRepository service. But I think this is good and further demonstrates the motivation behind this issue!
So... considering this, and looking closer at #10 again, I'd like to propose we use that approach for the
ClientRepository::validateClient()*instead* of adding thegetClientDrupalEntity()helper function altogether.I think we've forgotten about the
ClientEntityInterface::getDrupalEntity()function that can be used instead.Adding a patch with this approach.
Comment #16
bradjones1Thanks - yes, this kind of clean up will help us make the module more secure. Please mark NR when it's ready for me to take a look.
Comment #17
m.stentaThis new patch fixed the error I described in comment #13 above in farmOS tests! Thanks @paul121!
We are getting some other test failures in our API/OAuth tests still, but they may be unrelated to this issue. I will set this back to "Needs Review" so bradjones1 can take a look at the patch while we debug our other failures. :-)
Comment #18
bradjones1OK thanks.
I will try to get the tests addressed - might disable the install platform requirements check as a stopgap, even though I don't love it.
If you could post information about the test failures, that would help - I have been validating locally on PHP 8 as I've been rolling the beta, but I agree it would be good to get automated testing working again prior to making the next stable release.
Comment #19
m.stentaOh sorry I wasn't clear @bradjones1 - I was referring to farmOS tests that are failing - not Simple OAuth: https://github.com/mstenta/farmOS/runs/4606036865?check_suite_focus=true
Comment #20
bradjones1Thanks. I will try to look at this early next week. Planning to release 5.1.0 this week, then this can go on 5.1.1.
Comment #24
paul121 commentedOkay, here's to revitalizing this issue. There's a lot of info above: we added new patches in #10 #12 and #15 to keep up with changes to simple_oauth 5.x. Comments above are largely just figuring out what those changes were and how it affected our patches. The original issue description is still quite accurate. But just to reset and highlight some of the motivations for this issue:
From #10 (true of this MR as well)
From issue description:
Thus; the proposed solution needs to do two things:
1) Use dependency injection to ensure the ClientRepository
simple_oauth.repositories.clientservice is consistently used in all of the controller, form and manager classes. (this is the bulk of the changes in the diff)2) Ensure
ClientRepository::getClientEntity()is working correctly (essential - it should be the single method used to get clients). As identified in #14 I believe the newClientRepository::getClientDrupalEntity()function introduces a bug in the event the Drupal entity is not loaded. Instead of this new "helper" method, we should just use the existingClientEntityInterface::getDrupalEntity()function to get the Drupal entity. See patch for how this is resolved, it is quite a simple change.I've opened a new merge request with the patch from #15 rebased onto the latest 5.2.x branch. @bradjones1 if you could please advise on how to run tests for this MR? In the past I was able to run the simple_oauth tests on my local but now they are failing (even without any of these modifications to 5.2.x). These failures seem to be the same failures that we are getting when running the farmOS API tests with patch #15 applied so there must be something related.
Test result (running phpunit in our docker container - yes this might not be the best way to do this, but used to work? For reference, here are the related farmOS test failures: https://www.drupal.org/project/farm/issues/3282186#comment-14536882)
Comment #25
bradjones1Dumb comment bump to see if it triggers MR testing...?
Comment #27
bojan_dev commentedFor the 5.2.x this change makes sense, but for the 6.0.x I would like to address this in the data model by adding a new BaseFieldDefinition (string): 'client_id' on the consumer entity and replace all uuid loading logic. Will be working on this soon.
Comment #29
bojan_dev commentedComment #30
paul121 commentedSorry @bojan_dev I wasn't able to review your merge request before it got merged. I'm glad we're switching to using the
client_idon the consumer entity as that was the original motivation for this issue. However, I originally opened this issue to address a more generic issue that we should always be using the client repository to load the consumer entity. It wasn't clear from your comment in #27 if this was the approach you would take.With your change there still are multiple places where the logic is hard-coded to load consumers by their
client_id. It's my understanding the client repository service exists as an abstraction to consolidate this logic into a single place (please correct me if this has changed in 6.x). I agree with #16 that consistently using this service will make this more secure and maintainable going forward.Marking as "Needs work" as I don't think the original issue is "Fixed". IMO we need to update the issue/description as to what was actually changed, mark "Closed (won't fix)", or agree to implement a solution to the original issue.
@bojan_dev I'm happy to provide a MR for this, just want your opinion before I start anything. It would be a great opportunity for myself to look closer at the 6.x code and implementation which I just have not had the time to do.
Comment #31
bojan_dev commentedThis is a valid point and I agree we should leverage the
ClientRepositoryto centralize the loading of clients/consumers. In 6.0.x the usage of loading consumers byclient_idhas been reduced, but is still in place for theOauth2AuthorizeControllerandOauth2Token.So feel free to provide a MR, and it would be great to have more input on the 6.0.x code.
Comment #32
bojan_dev commentedI have addressed the fix for #3316519: Token request broken consumer load by client id changed in this MR: https://git.drupalcode.org/project/simple_oauth/-/merge_requests/31, this issue is related and it had already work done by Paul.
A review would be welcome.
Comment #33
paul121 commentedThanks @bojan_dev this LGTM! I'm not able to test too extensively at the moment locally but did include this patch in our farmOS automated tests and it worked.
Comment #36
bojan_dev commented