Problem/Motivation

User gets an error when trying to reference an unpublished entity that they have just created and that he has full access to. This basically makes it impossible for a user to submit a set of unpublished interreferencing entities (e.g. artefact/architect/city).

Steps to reproduce:

  1. Create content type A, make sure Published is unchecked in the Publishing options.
  2. Create content type B, add an ER field referencing A, choose Autocomplete widget in Form display.
  3. Create a test user, give permissions to View own unpublished content, to Create A and B, to Edit own A and B.
  4. Log in as that user.
  5. Create a test node of type A, verify that it is Unpublished and the user can see it.
  6. Create a test node of type B, start typing the label of the A node just created, verify that it's not recognised and suggested by the system.
  7. Type the complete label and hit Save, get the "No such entity" error.
  8. Type the entity label followed by its nid in parentheses, still get the "This entity (node: N) cannot be referenced”

This issue also prevents the user w/ appropriate permissions from referencing unpublished entities authored by other users.

Proposed resolution

Allow referencing unpublished entities if the current user has access to them.

To implement that:
- We are adding a flag in selection handler plugin
- Based on the flag values we are checking additional permissions and allowing users to check the node

Solution approach is based on #80 #82 #83

Remaining tasks

- Needs review
- UX review if configured option labels isn't confusing

User interface changes

Additional option is added on the selection handler plugin settings

API changes

none

Data model changes

none

Release notes snippet

Issue fork drupal-2845144

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

ndlarge created an issue. See original summary.

marassa’s picture

Issue summary: View changes
marassa’s picture

Title: User can't reference his own unpublished entity even when he has access to it » User can't reference his own unpublished content even when he has access to it
marassa’s picture

Ok, I did some research and discovered that all my problems conveniently reside in one small file:
/core/modules/node/src/Plugin/EntityReferenceSelection/NodeSelection.php
Apparently, it's by design that functions buildEntityQuery() and validateReferenceableNewEntities() require the referenceable nodes to be published (unless any access control module is installed).
As it basically ruins my whole idea of submit/publish workflow and I am reluctant to install an access control module for the sole reason of overriding a couple of lines in core, I tried to simply remove the offending lines.
Another problem for me was "Create referenced entities if they don't already exist" does not respect published/unpublished default. There's a line in the same module explicitly setting Published to TRUE because "In order to create a referenceable node, it needs to published". Well it doesn't anymore, so I removed this line too.
I have been testing with these changes in core and everything seems to work like a charm: a user is able to submit a whole batch of unpublished interreferencing entities (like artefact, architect, city, pictures).
Now the serious question: could anyone having a better understanding of the Drupal internals (admittedly I know next to nothing about it) please tell me what specific risks I am running after removing that code from the core? I may well underestimate the impact of what I have done.
If there are no risks, the obvious next question is: does this code really have to be there?
I will greatly appreciate any insight.

megan_m’s picture

You can get around this by using an Entity Reference view as the Reference Method and removing the published setting from the view. But, this only works if you don't need users to be able to create new entities if they don't already exist. That checkbox isn't even available if you choose a view as the reference method.

I agree that the user should be able to reference any entities they have access to and create new entities as unpublished if they have access to do that.

marassa’s picture

mmcdermott, thank you for your time and for agreeing with my view of the existing behaviour as wrong!
Unfortunately my users DO need to create new (UNPUBLISHED!) entities if they don't already exist, so...
I did some reading on the subject and realized that I posted to a wrong place - entity_reference.module is retired, and the discussion should probably be under the "entity system" component. Will try to move it there...

marassa’s picture

Component: entity_reference.module » entity system
liam mcdermott’s picture

Version: 8.2.4 » 8.4.x-dev

I can reproduce this on 8.4.x-dev.

please tell me what specific risks I am running after removing that code from the core?

If your site is built on the assumption that everyone can see their own unpublished content then this probably doesn't matter, aside from the fact you'll have to tediously patch your site every time there's an update to core.

Obviously it would be unacceptable for the Drupal project to remove the check, it would go from a bad assumption to a worse one, from: only those with 'bypass node acces' may reference unpublished nodes, to: anyone may reference unpublished nodes and now we have an information disclosure vulnerability.

Unless there's a good reason not to, users should be able to reference entities they have permissions to view.

marassa’s picture

you'll have to tediously patch your site every time there's an update to core

True, unfortunately - that's why I am still on 8.2.6 ;)

anyone may reference unpublished nodes and now we have an information disclosure vulnerability

IMHO, the disclosure is minimal because the most you will see while adding/editing content is the fact that a node with a certain title exists, the rest is still reliably hidden. But I agree there should be a more elegant solution to that, restricting the referenceable nodes to those visible to the current user.

Unless there's a good reason not to, users should be able to reference entities they have permissions to view

That's exactly the point I was making.
Unfortunately my Drupal core skills are only strong enough to identify offending lines and hack them off but not strong enough yet to offer a reasonable and robust replacement. Was hoping someone more experienced is having the same problem and will be able to fix it in a more acceptable way. But so far my workaround works for me, apart from every core update being unreasonable pain ;)

liam mcdermott’s picture

Status: Active » Needs review
StatusFileSize
new2.48 KB

Here's a patch. NodeSelection::buildEntityQuery() now includes unpublished nodes created by the user (if they have the 'view own unpublished content' permission). The NodeSelection::createNewEntity() override was deleted, since reference-able nodes no longer need to be published.

marassa’s picture

Many thanks, Liam!
There's one more check for $node->isPublished() I had to remove in the very last function in the same file - validateReferenceableNewEntities(). I guess your new check should be copied there as well?

liam mcdermott’s picture

Thanks for the review ndlarge.

There's one more check for $node->isPublished() I had to remove in the very last function in the same file - validateReferenceableNewEntities().

Yes that's correct; good catch, updated patch attached. If you see something wrong (like this) feel free to mark the issue 'Needs work'

dawehner’s picture

Issue tags: +Needs tests

This needs some serious test coverage to not break again :)

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

imperator_99’s picture

@megan_m your solution worked perfectly for me and is quite maintainable. Thanks very much!

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

oksana-c’s picture

Title: User can't reference his own unpublished content even when he has access to it » User can't reference unpublished content even when he has access to it
Issue summary: View changes
StatusFileSize
new3.14 KB

Hi all, thank you for catching and documenting this issue.

I'm adding an updated patch that takes into account Content Moderation permission "View any unpublished content". Since this module is now in core, this permission should be considered.

Use case:
REST resource provides POST method for content that contains several entity reference fields.
POST calls are authenticated using a user w/ an API role; most referenced content is created by a user w/ Editor role in unpublished status. API and Editor are two different users.
Given above, each POST request fails w/ an error "This entity (node: N) cannot be referenced".

Unfortunately, due to limited resources outside my influence or control, It is not possible for me to add test coverage for scenarios addressed in this issue at this time. I'll do my best to follow up asap if/when possible.

Status: Needs review » Needs work
oksana-c’s picture

tests are failing - https://www.drupal.org/pift-ci-job/1239157
Existing test coverage needs to be adjusted and new tests need to be added to cover entity references by different authors w/ varied set of permissions.

marassa’s picture

@oksana-c, I think found a typo in your code:

+        return ($node->getOwner() == $this->currentUser->id() || $node->isPublished());

$node->getOwner() is an entity and $this->currentUser->id() is an id, so this condition is never true.
Should be $node->getOwnerId(), like in the original patch, I guess.

oksana-c’s picture

that is a serious typo. Thanks for catching @ndlarge. Attached updated patch.

oksana-c’s picture

StatusFileSize
new3.14 KB
marassa’s picture

@oksana-c, works for me now, thank you! I guess the only thing that's missing is tests.

lahoosascoots’s picture

IMO, this should be configurable. The entity_reference_unpublished module (https://www.drupal.org/project/entity_reference_unpublished) adds this functionality in as a selection plugin which would allow site owners to choose if an entity reference field can reference unpublished content. However it doesn't limit based on current permissions. Maybe we could combine the two way to both respect permissioning and be configurable.

marassa’s picture

IMO, this should be configurable

Frankly, I can't come up with a single reason why a user should NOT be allowed to reference content he has access to. Does anyone have a use case for when the current behavior is reasonable/desirable?
To me the current behavior is a bug.

andyd328’s picture

+1 for making it configurable.

I have a number of clients that hate seeing unpublished entities available in lists and I can understand that they want to quickly select from current entities. When presented with a complete list of all entities they would also end up referencing a similarly named unpublished entity by mistake and call us asking why their site is broken when the referenced entity doesn't show for anonymous users of the site. For another specific case - site owner does not want to see members of a site that have left on bad terms, but the entities need to be retained in the system for reporting, being able to hide unpublished from lists is ideal.

marassa’s picture

@AndyD328 - could you please clarify the exact circumstances under which your clients are currently seeing unpublished entities in lists? Without these patches installed, they shouldn't see any unpublished entities at all. With the patch installed, they will ONLY see unpublished entities if they are explicitly given access to those entities. If they don't need to see those entities why give them access to them?

andyd328’s picture

@marassa - the clients need to be able to view ( and sometimes edit ) unpublished entities, but they shouldn't link to them in entity reference lists - which makes not showing in the list is the right ux decision.
The use case to have access to entities but not show them in entity reference fields does exist, and having it be configurable with a default to hide unpublished - making it a non-breaking change - would be ideal.

NewZeal’s picture

I've set up entity reference views for my reference fields that reference unpublished content as suggested by Megan in comment #5. I can't think of any situation where this isn't all that is required. As Andy says, it may well be that a user can view unpublished content, but that doesn't mean they want to access it wherever they go in the site, certainly not without having the choice.

Are there any situations where an entity reference view doesn't do the trick?

marassa’s picture

@kent@passingphase.nz:

Are there any situations where an entity reference view doesn't do the trick?

Yes, and they are clearly outlined right in comments #5 and #6.
Drupal Autocomplete widget for ER fields has an option of creating new referenced entities if those don't exist. In my case when a user adds a "building" node, they fill in country, city, architect(s) which are all ER fields. If the city/country/architect they have typed don't exist, they are created automagically which doesn't work with a view.
IMO, using a contrib module to bypass a kernel bug is just a clumsy kludge, not a solution in any case, and in some situations like mine it even doesn't work at all.

vinodhini.e’s picture

@oksana-c, I applied patch #19, works for me now, thank you!

vinodhini.e’s picture

Status: Needs work » Needs review

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

orlando.thoeny’s picture

What about introducing a new permission in Core? Setting it to FALSE by default, so the behaviour of existing sites is not affected?
%type_name: Reference any unpublished content or something along those lines.

Of course one could create entity reference views, but if you want this behaviour on every entity reference field, that's not really that practical.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

oriol_e9g’s picture

Status: Needs review » Needs work
StatusFileSize
new3.15 KB

This is only a reroll for Drupal 9.1.x. Still needs work to fix tests.

vsujeetkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new7.58 KB
new3.85 KB

Fixed test, Please review.

marassa’s picture

Status: Needs review » Reviewed & tested by the community

Thanks a ton @vsujeetkumar - works ok for me!

alexpott’s picture

Title: User can't reference unpublished content even when he has access to it » User can't reference unpublished content even when they have access to it
alexpott’s picture

Issue summary: View changes
alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/field/tests/src/Kernel/EntityReference/EntityReferenceItemTest.php
    @@ -455,9 +455,7 @@ public function testAutocreateValidation() {
         $errors = $entity->validate();
    -    $this->assertCount(1, $errors);
    -    $this->assertEqual($errors[0]->getMessage(), new FormattableMarkup('This entity (%type: %label) cannot be referenced.', ['%type' => 'node', '%label' => $title]));
    -    $this->assertEqual($errors[0]->getPropertyPath(), 'field_test_node.0.entity');
    +    $this->assertCount(0, $errors);
     
         // Publish the node and try again.
         $node->setPublished();
    @@ -503,19 +501,13 @@ public function testAutocreateValidation() {
    
    @@ -503,19 +501,13 @@ public function testAutocreateValidation() {
         ]);
     
         $errors = $entity->validate();
    -    $this->assertCount(2, $errors);
    -    $this->assertEqual($errors[0]->getMessage(), new FormattableMarkup('This entity (%type: %label) cannot be referenced.', ['%type' => 'node', '%label' => $unsaved_unpublished_node_title]));
    -    $this->assertEqual($errors[0]->getPropertyPath(), 'field_test_node.0.entity');
    -    $this->assertEqual($errors[1]->getMessage(), new FormattableMarkup('This entity (%type: %label) cannot be referenced.', ['%type' => 'node', '%label' => $saved_unpublished_node->id()]));
    -    $this->assertEqual($errors[1]->getPropertyPath(), 'field_test_node.1.target_id');
    +    $this->assertCount(0, $errors);
     
         // Publish one of the nodes and try again.
         $saved_unpublished_node->setPublished();
         $saved_unpublished_node->save();
         $errors = $entity->validate();
    -    $this->assertCount(1, $errors);
    -    $this->assertEqual($errors[0]->getMessage(), new FormattableMarkup('This entity (%type: %label) cannot be referenced.', ['%type' => 'node', '%label' => $unsaved_unpublished_node_title]));
    -    $this->assertEqual($errors[0]->getPropertyPath(), 'field_test_node.0.entity');
    +    $this->assertCount(0, $errors);
    

    I think we need to maintain these tests. I.e. we need to test when a user doesn't have access.

  2. +++ b/core/modules/node/src/Plugin/EntityReferenceSelection/NodeSelection.php
    @@ -28,23 +28,22 @@ protected function buildEntityQuery($match = NULL, $match_operator = 'CONTAINS')
    -  public function createNewEntity($entity_type_id, $bundle, $label, $uid) {
    -    $node = parent::createNewEntity($entity_type_id, $bundle, $label, $uid);
     
    -    // In order to create a referenceable node, it needs to published.
    -    /** @var \Drupal\node\NodeInterface $node */
    -    $node->setPublished();
    

    I'm not sure that we should change the createNewEntity behaviour. Why do we need to do that? That's a bigger scope of change.

  3. +++ b/core/modules/node/src/Plugin/EntityReferenceSelection/NodeSelection.php
    @@ -28,23 +28,22 @@ protected function buildEntityQuery($match = NULL, $match_operator = 'CONTAINS')
    +    // Permission to "view own unpublished content" allows
    +    // the user to reference any published content or own unpublished content.
    +    // Permission to "view any unpublished content" allows
    +    // the user to reference any unpublished content.
    +    if ($this->currentUser->hasPermission('view own unpublished content') && !$this->currentUser->hasPermission('view any unpublished content')) {
    +      $or = $query->orConditionGroup()
    +        ->condition('status', NodeInterface::PUBLISHED)
    +        ->condition('uid', $this->currentUser->id());
    +      $query->condition($or);
    

    We need to add test coverage of a user with 'view own unpublished content' and one with 'view any unpublished content'. And we need to test that a user with 'view own unpublished content' cannot reference anyone else's unpublished content - which would be a security issue.

marassa’s picture

@alexpott:

I'm not sure that we should change the createNewEntity behaviour. Why do we need to do that?

You are right, we shouldn't! The current behaviour is outright wrong. The current EntityReferenceSelection plugin has to override the standard createNewEntity method as an ugly kludge because "In order to create a referenceable node, it needs to published". This results in any new node created from an ER widget being published regardless of content type settings and publishing workflow in place.
The whole idea of the patch is to remove this arbitrary and unnecessary limitation. Now that a newly created node does not have to be published to be referenceable, we don't need to force it to be published bypassing any settings or publishing workflows.

alexpott’s picture

@marassa yes but this is a change in behaviour that is not necessary in order to make the fix. People might have workflows that rely on node created this way to be published and this issue is changing that without needing to.

marassa’s picture

@alexpott I agree that there is a change in behaviour but the rest of the patch changes the existing behaviour as well. Either we decide that the existing behaviour is wrong in more than one way and fix it or we decide that we don't want to change any existing behaviour ever.
For my particular use case changing one without changing the other is very simply useless.
I just noticed that I simply mentioned this related issue in my comment #4 but failed to attach it as related (was very new to Drupal three years ago), attaching it now.

I can only speak of my use case/workflow but I can't see what workflow would consciously set up a content type as unpublished at creation by default but would expect or tolerate it to be auto-published when it is auto-created from an ER widget on another content type form.

psf_’s picture

In my use case, I have a profile entity that references a content type that I don't want to publish after the user creates it. I want a moderator to publish it after revision.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

tlwatson’s picture

#39 works great for me, thanks!

vsujeetkumar’s picture

Status: Needs work » Needs review
StatusFileSize
new7.61 KB

Re-roll the patch for 9.2.x.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

vselivanov’s picture

StatusFileSize
new7.62 KB

Updated the patch #50 for 9.3.x. Also tested it with Drupal 9.1.10.

dalin’s picture

Issue summary: View changes

My opinion on the "fix it now, later, or not at all" question around entity creation:

If I as a site builder configure an entity bundle to be unpublished by default, but for some weird reason this back corner of Core is not respecting that, then it could almost be classified as an access bypass vulnerability. Good security is not just about what is technically true/false, but also about what the user interface suggests should happen. I suggest that fixing this is more important than maintaining the existing faulty behaviour.

marassa’s picture

@dalin

it could almost be classified as an access bypass vulnerability. Good security is not just about what is technically true/false, but also about what the user interface suggests should happen. I suggest that fixing this is more important than maintaining the existing faulty behaviour

I agree! At least ten people in this thread either agreed it's a bug or provided patches or rerolled patches or reported using patches. I don't know how many more are silently using those patches. Apparently not enough to convince the powers that be to commit it...

dalin’s picture

Issue summary: View changes

@marassa

There are no "powers that be" :-) It's just us working together to come up with a way forward (at least in theory).

Here's what I've added to the issue summary. We need to rally around one of the options, or propose a new one. Secondly to fix the issues with tests, but that's more straightforward.

Remaining tasks

Resolve what to do (if anything) for entities that are created via an entity-reference field. (See comments #43–46)
The gist of the problem is the existing kludge that Core had before this patch misleads site builders and content creators into inadvertently exposing private content. Is it more important to fix that, or is it more important to avoid changing existing behaviour (because some people may be relying on that).

Options include:

  • Do not make any changes to entity creation.
  • Roll a separate issue for this, possibly bump to Drupal 10.
  • Decide that fixing the problem is more important than maintaining existing behaviour.

Secondly, some changes to tests were recommended in comment #43 that are yet to be addressed

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

moshe weitzman’s picture

mass.gov has started successfully using this patch, as our editors were suffering. Seems to me like we should just commit non-BC fix to Drupal 10, and then folks can backport and discuss BC as a separate effort.

vinodhini.e’s picture

gatorjoe’s picture

I confirm that #52 solves my issue of an editor not being able to reference an unpublished entity. Thank you so much!

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

joaopauloscho’s picture

I can also confirm that #52 works for us, thanks!

mitthukumawat made their first commit to this issue’s fork.

alphawebgroup’s picture

StatusFileSize
new7.64 KB

re-rolled for 9.5.x

Status: Needs review » Needs work

The last submitted patch, 63: 2845144-63.patch, failed testing. View results

ravi.shankar’s picture

StatusFileSize
new7.59 KB
new1.51 KB

Tring to fix failed tests of patch #63.

edysmp’s picture

Last patch is not working on 9.4 used #63 instead.

ankithashetty’s picture

Status: Needs work » Needs review
StatusFileSize
new7.59 KB
new6.92 KB

Fixed the test errors in #63. Attached a diff file for reference. Not sure why, but interdiff with #63 didnt work:

interdiff 2845144-63.patch 2845144-67.patch > interdiff_2845144_63-67.txt
interdiff: Whitespace damage detected in input

Thanks!

marassa’s picture

#67 works for me on 9.4.0 - thank you!

gatorjoe’s picture

I upgraded to 9.4 and replaced patch #52 with #67 successfully. Thank you!

mstrelan’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/node/src/Plugin/EntityReferenceSelection/NodeSelection.php
    @@ -53,12 +52,19 @@ public function createNewEntity($entity_type_id, $bundle, $label, $uid) {
    +    // Permission to "view own unpublished content" allows
    +    // the user to reference any published content or own unpublished content.
    +    if ($this->currentUser->hasPermission('view own unpublished content') && !$this->currentUser->hasPermission('view any unpublished content')) {
           $entities = array_filter($entities, function ($node) {
             /** @var \Drupal\node\NodeInterface $node */
    -        return $node->isPublished();
    +        return ($node->getOwnerId() == $this->currentUser->id() || $node->isPublished());
           });
         }
    +
    

    If the node is published we should return TRUE before checking the current user's permissions.

    Also wondering how this would affect anonymous users? Ideally they wouldn't have the "view own unpublished content" permission, but if they did then they would also see all content owned by user 0.

    Also a nit, we should use === here.

  2. +++ b/core/modules/node/src/Plugin/EntityReferenceSelection/NodeSelection.php
    @@ -28,23 +28,22 @@ protected function buildEntityQuery($match = NULL, $match_operator = 'CONTAINS')
    +    // Permission to "view own unpublished content" allows
    +    // the user to reference any published content or own unpublished content.
    +    // Permission to "view any unpublished content" allows
    +    // the user to reference any unpublished content.
    +    if ($this->currentUser->hasPermission('view own unpublished content') && !$this->currentUser->hasPermission('view any unpublished content')) {
    +      $or = $query->orConditionGroup()
    +        ->condition('status', NodeInterface::PUBLISHED)
    +        ->condition('uid', $this->currentUser->id());
    +      $query->condition($or);
    +    }
    

    We should check for user 0 here too.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

phjou’s picture

It seems like the SelectList widget is still not listing the unpublished nodes even if we have the permission when using the patch in #67

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

macsim’s picture

Why don't we add new permissions to handle this use case?

  • "Reference own unpublished [node_type] entity"
  • "Reference own unpublished [vocabulary_name] term"
  • "Reference any unpublished [node_type] entity"
  • "Reference any unpublished [vocabulary_name] term"
  • etc.

Giving the ability to a role to see an unpublish content doesn't mean you want this role to be able to reference unpublished entities.
You might allow an "administrator" role to have those permissions and disallow a "content contributor" from having them.
That would be a more granular (but also more complex) way to handle this.

dalin’s picture

@MacSim,
Sometimes we talk about how Drupal Core should handle 80% of common use cases. What you're describing sounds like an edge case, so contrib is probably the better place for something like that to live.

macsim’s picture

@dalin,

I am sorry but I can't totally agree ; Drupal is also about logic and granularity and a lack of logic/granularity oftenly leads to a new opened issue.
In #55 you said

We need to rally around one of the options, or propose a new one.

That's exactly what I am doing ;)

On this issue, the quick workaround is to give a powerful "Bypass content access control" permission to the roles we want in order to allow people referencing unpublished entities. Since it can have security implications, we're hereby talking about the best way to resolve this issue.

And as usual with an issue there's a "quick and dirty" way to resolve it and the "hard but prettier" way (I am not saying that what was done here is the "quick and dirty" one but I would say - and it's just my own opinion - that it could be prettier).

Using a "view" permission to handle a "reference" action doesn't make sense to me.

Since we already have those permissions model in drupal core to view / edit / delete own/any entities it sounds more logical to me to add a "reference unpublished" permission following the same model rather than cheating with another already existing permission.

Why would we only answer to 80% of common use cases when we can reach a 100% working a bit more?
Why would we leave the last 20% on the road especially when we noticed an issue and are working to resolve it?
Why would we need to develop and maintain a contrib module that people might not found if its name is not explicit enough to handle 20% of use cases that could have been fixed in core directly?

In those 20% of use cases there might be site builders who really need the feature and don't have development skills to build a contrib module.

Anyway, I agree with #73 the patch provided in #67 doesn't work as expected on a core 9.5.9.

bibhor’s picture

I am facing the same issue on core 9.5.10
May I know if there is any solution?

dieterholvoet’s picture

StatusFileSize
new7.55 KB

Rerolled against 11.x.

dww’s picture

Issue tags: +Bug Smash Initiative

Thanks to everyone who has contributed towards fixing this bug! My attempt to summarize:

  1. The existing behavior is definitely unexpected and undesired in many cases.
  2. Some sites might depend on the current behavior.
  3. Drupal Core needs to worry about BC in general, and doesn't even use new major releases for disruptive changes (except dropping deprecated code). Alas, #57 is not really an option.
  4. Therefore, we probably need a setting to control this behavior, default to the expected thing for new installs, provide a post_update to set the legacy behavior for existing sites, etc. See #26, #28.
  5. #43 still needs to be addressed. #43.2 is under debate. I tend to agree that the point of fixing this bug is changing that weird behavior and things that rely on it. We need clarity from the framework / release managers on how we want to scope this. Regardless, #43.1 and #43.3 are still necessary before we can remove "Needs tests" and therefore get this to RTBC.
  6. #70 also needs to be addressed.
  7. #73 also sounds like a bug in the current implementation that better test coverage would reveal.
  8. Re #75 + #77: I'm reluctant to add Yet More Permissions(tm) to solve this. Seems more like a widgetselection plugin setting to me (e.g. a "[x] Include unpublished entities the user has access to view" checkbox or something).
  9. We need to convert this to an MR against 11.x, maybe starting from #38 so that "fixing tests" doesn't necessarily mean removing all the coverage we had. 😅

Tagging to be smashed. I'll ping some folks in #bugsmash to try to get scope decision / approval before we go further.

Thanks again!
-Derek

marassa’s picture

@dww,

I'm reluctant to add Yet More Permissions(tm) to solve this. Seems more like a widgetselection plugin setting to me (e.g. a "[x] Include unpublished entities the user has access to view" checkbox or something).

If we absolutely can't just change the existing [unexpected and undesired] behavior, then I think a widget setting is a very good idea. Because it will allow to override the default behavior on an individual field level, not on a site level. Who knows, maybe someone will need different behavior in different parts of the same site?

larowlan’s picture

A setting is a great idea. It would need to be on the selection handler rather than the widget though.

dww’s picture

Re: #82: Indeed, that's what I had previously edited comment #80 to say, should have called it out explicitly. This needs to be a selection handler setting, not a widget setting. Seems like this has direction to proceed again, if anyone's got time to run with it.

Thanks!
-Derek

prashant.c’s picture

StatusFileSize
new1.14 KB

I've noticed there's been a lot of discussion on this topic, and I'd like to contribute by sharing my findings:

1. I successfully replicated this issue on version 11.x as well.
2. Could we consider making the following change in the code:

-      $query->condition('status', NodeInterface::PUBLISHED);
+      if ($this->currentUser->hasPermission('view own unpublished content')) {
+        $or = $query->orConditionGroup()
+          ->condition('status', NodeInterface::PUBLISHED)
+          ->condition('uid', $this->currentUser->id());
+        $query->condition($or);
+      }
+      else {
+        $query->condition('status', NodeInterface::PUBLISHED);
+      }

It seems to yield the expected results, displaying unpublished content for which the user is the author, while the default remains Published only.

I would appreciate your thoughts and feedback on this.

Thank you.

mlncn’s picture

Priority: Normal » Major

Gently bumping this to major for all the above and for the following scenario, cannot save content with unpublished entity reference:

  1. An editor creates content that references a term or other content.
  2. An editor unpublishes that other content or term.
  3. An editor edits the content and cannot save it while receiving a baffling error: "Leadership (value 3)" and over on that field, "This entity (node: 792) cannot be referenced."

No further explanation to add "because it is not published." No link to the content that needs to be published to be able to save the current piece of content.

Even though the editor may have the permissions to fix the error themselves and try re-saving, they are often (repeatedly, even for experienced, motivated editors) unable to.

That's the experience using the default autoselect widget.

With the Inline Entity Reference it will have something like "Error message: 1 error has been found: Partners" and then repeat in four places on that field "This entity (taxonomy_term: 1582) cannot be referenced." (at the top of the field, for the "Add existing" autocomplete, below the "Add" button, and for the "Create new" button— but not on the offending term, which does not have an ID next to it so it is even harder to identify the culprit.

Oh, and if the widget is a select list dropdown or checkboxes? The editor can save the content no problem— the unpublished reference is simply removed. Boom, data loss!

Two very different scenarios, one leading people to be stuck unable to save content they have edited, the other silently deleting a reference, based merely on the form display.

So yes, it is of major importance that people be able to reference unpublished content, especially that they have access to. I'd like to add messages warning people that they have referenced unpublished content from published content (and Drupal will not display it), but that can and likely should be done in contrib.

mlncn’s picture

Last two patches, reroll of the main approach in #79 and the alternate approach in #84 do not cover taxonomy terms.

lars.stiebenz’s picture

StatusFileSize
new7.52 KB

The patch is an update to the one in #79 against Drupal 10.2.3.

One test class was moved:
Old path: core/modules/system/tests/src/Functional/Entity/EntityReferenceSelection/EntityReferenceSelectionAccessTest.php
New path: core/modules/system/tests/src/Kernel/Entity/EntityReferenceSelection/EntityReferenceSelectionAccessTest.php

mohit_aghera changed the visibility of the branch 2845144-user-cant-reference to hidden.

mohit_aghera’s picture

Issue tags: -Needs tests

Note: This is the different approach compared to all the uploaded patches. So be cautions if you are replacing this with your existing patches. This is based on comms in #80and #82

I've refactored the implementation based on discussion from #80 #82 and #83

Updates on #43
- Fixed point 1 and point 3.
Removed new test for validating both the permissions and removed changes in edit callback.
Reverted un-necessary changes made to the test callbacks.
- Reverted point 2.

I've ignored changes done in-between #50 to #69 since all are removing existing test cases and changing things in node handler.

#70
I'm somewhat confused about content owned by user 0.
I've added the changes and test case changes in commit. I think we can refactor it depending on the feedback.

mohit_aghera’s picture

Status: Needs work » Needs review
mohit_aghera’s picture

Issue summary: View changes
smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs issue summary update, +Needs change record, +Needs usability review

So this seems to be adding a new setting to the form, issue summary should contain a screenshot of the change being added.

Will eventually need a change record.

One of the remaining tasks is to get usability sign off on the labels so tagging for that.

csmdgl’s picture

StatusFileSize
new0 bytes

Rerolled #67 for 10.2.5.

csmdgl’s picture

StatusFileSize
new7.54 KB
new0 bytes

Rerolled #67 for 10.2.5.
Patch attached this time.

tobiasb made their first commit to this issue’s fork.

rkoller’s picture

Usability review

We discussed this issue at #3427743: Drupal Usability Meeting 2024-03-22. The recording of the meeting can be found here: https://youtu.be/0ktdZlth-FA?si=ECoI-aaRngvQuEgB&t=773

For the record, the attendees at the usability meeting were @benjifisher, @rkoller, @simohell, @skaught, and @worldlinemine.

If you want more feedback from the usability team, a good way to reach out is in the #ux channel in Slack.

At first apologies the overdue comment. There was a chain of unfortunate circumstances. In our testing by switching between the applied- and none-applied MR state while having two user accounts switching between two different browser windows got things mixed. That lead into a few misconceptions so that at the end of the meeting we had no clear consensus, several loose ends, and not agreed on who will comment on the issue. After noticing the missing comment, uncovering the misconceptions during renewed testing, it took a while to get that patchwork of information into a coherent and hopefully comprehensive order.

The first misconception, it looked like that the other form display widgets for autocomplete would already provide the option to display unpublished nodes. In consequence we were drawn between the following three options at the end of the meeting:

  1. Take the patch as is, so it adds this new configuration option to the autocomplete widget
  2. Unconditionally change the behavior of the autocomplete widget to make it consistent with other widgets and always give the behavior as if this new option was ticked all the time.
  3. Don't do anything and leave it the way it is

The second misconception, in the context of scenario 1 and permissions, we were under the impression that the user would have the permission to view any unpublished content with any content type and that the label include unpublished entities the user has access to view isn't reflecting that fact. But while writing up this comment, those two misconceptions turned out false assumptions when i've retested the functionality. So after this brief outline of our stumbling blocks along the way a summary of out main takeaways. It has to be noted that the group agreed to revisit the topic at a later point since we haven't got to a clear consensus during the meeting but that was mainly due to our false assumptions.

So it is save to say that the group agreed that the functionality the MR provides is useful and it is a good thing to have the option to enable the feature on a per content type basis. There are two points to note we came across during the meeting that apply in any way:

In regards of permissions @skaught has taken another look after he got word about our misconception in that regard. I'll paste his comment from Slack in here:

#540008: Add a container parameter that can remove the special behavior of UID#1 yes, userone has a service setting now (but, still exists) and is still defaultly engaged.
I think I was thinking around what if userone can add a node any -- even if that node has some custom Node Access Permission overriding it. But then another user edits that later, then this item (being altered thru something in drupal (:) would not fall into the combined logic of permissions the routine is checking for, but now this editor drops an item.

The main usability related points are in regards of situational awareness for the user. Those points could either be covered within this MR or moved into followup issue(s):

  • At the moment you are unable to distinguish between published and unpublished nodes while scrolling the list of available results. Unpublished nodes should be labeled in the list of options in the autocomplete field in some way.
  • In the example given in the steps to reproduce section the author might be unaware publishing a node that is linking a yet to be published still unpublished node. There are several more scenarios publishing or unpublishing a node with linked nodes. It should always be a deliberate informed decision for the persons editing, no matter if a node, or the nodes linked, are getting published or unpublished. Otherwise that might lead into a mess in regards of publishing states. One solution might be simply adding a confirmation step informing the user. In case of the example from the steps to reproduce section the user would have to confirm are you sure you want to publish this node since you are linking a node that is still unpublished. But the different potential scenarios would have to be explored and tested, but those are definitely out of the scope for this issue

I'll leave the needs usability review tag since we plan to revisit and discuss the issue at a later point.

mxr576’s picture

Could we get rid of this whole problem by solving this other task?

#3452449: Grant query level access to own/any unpublished nodes

scott_euser’s picture

That unfortunately wouldn't solve the usability issues flagged in #97, particularly to distinguish between published and unpublished content

parvateesam’s picture

#95 patch is applied for 10.3.0. thanks @csmdgl

abhinand gokhala k’s picture

#95 patch is applied for 10.3. and working, thanks

liberatr’s picture

StatusFileSize
new7.49 KB

#95 wasn't applying for me, attempted a re-roll.

Nevermind. This was an unrelated issue. Only line numbers changed. Usually git doesn't care about that.

ericgsmith made their first commit to this issue’s fork.

ericgsmith’s picture

Found this issue while looking #3110677: [PP-1] Referencing new media requires "administer media" permission which is dealing with the same issue - only for the view own unpublished media permission / MediaSelection handler. I set that issue to postponed as I believe the backwards compatibility handling and UX review happening here are needed and applying the solution from this to the media selection could be done as a follow up to this issue.

Looking at the feedback from the accessibility review

> At the moment you are unable to distinguish between published and unpublished nodes while scrolling the list of available results. Unpublished nodes should be labeled in the list of options in the autocomplete field in some way.

I would suggest it should be a follow up, since this change is not introducing this problem. Seeing unpublished content is currently the case for users who see unpublished content by way of the existing checks so changing the label could be done independent of this work?

I also think the wording is a little misleading - because when "Include unpublished entities the user has access to view" is not checked - we still have the current behaviour where the node grants implementation may still allow the unpublished entity to be returned. Whats more, given the setting is on the base handler then it shows for media and term references as well without actually changing their behaviour - so perhaps #3110677: [PP-1] Referencing new media requires "administer media" permission should be handled here as well? Because what that enables now is for a user to NOT select "Include unpublished entities the user has access to view" but then unpublished entities still show up if the have "administer taxonomy" or "administer media" permissions for the respective references. Likewise, it now allows "Include unpublished entities the user has access to view" to be checked for these reference types - but without actually showing all the entities that user has access to.

ericgsmith’s picture

Issue summary: View changes
StatusFileSize
new26.66 KB
bkosborne’s picture

I agree that the ability to distinguish between published and unpublished content is out of scope for this and should be handled in a follow up. This issue has gone on long enough, adding that requirement will delay it further.

bkosborne’s picture

Title: User can't reference unpublished content even when they have access to it » Users without 'bypass node access' permission can't reference unpublished content even if they have access to it
tonytheferg’s picture

I am finding I can't reference unpublished nodes through the autocomplete form, even as admin with all permissions

ltrain’s picture

StatusFileSize
new7.42 KB

Rerolled #95 for 11.2.0

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

kerasai’s picture

Noting that the patch in #109 applies to 11.3.12.

Have not thoroughly tested, so test of functionality is TBD.

sophie.sk’s picture

Confirming the patch in #109 also applies cleanly on Drupal 11.4.5.

Looks like the merge request still needs to be updated though.

herved made their first commit to this issue’s fork.

herved’s picture

I moved the setting to NodeSelection. It only works there, but the checkbox was shown on every entity type.
There is a @todo to move it to <code>DefaultSelection in #3110677: [PP-1] Referencing new media requires "administer media" permission, once other handlers support it.
That issue could be rescoped to every entity type implementing EntityPublishedInterface in core: media, taxonomy terms, comments, custom blocks,...

I also dropped the post_update in favor of requiredKey: false, which is simpler.
The default already applies when the key is missing, and the update added the key to fields whose handler does not define it, like ViewsSelection, which then fails config schema validation.

herved’s picture

Status: Needs work » Needs review

CR created: https://www.drupal.org/node/3620430

---

Note for anyone using patch #109: it does not have the "uid 0" problem, but it does have the other one:
A user with neither "view own unpublished content" nor "view any unpublished content" gets no status filter at all, so they can reference any unpublished node. That patch has no setting, so this applies to every user on the sites using it.

That is fixed in the MR, in the commit "Only reference the unpublished nodes the user may view", with a test.

usmanjutt84’s picture

StatusFileSize
new57.95 KB
new57.95 KB

Patch #109 is applied with Drupal 11.4.9 successfully. However, as @herved highlighted it has an issue that I can confirm.

Moreover, MR#6831 won't work with that Drupal 11.4 because of https://www.drupal.org/project/drupal/issues/3581349 issue. So, here is the patch that works with Drupal 11.x.

usmanjutt84’s picture