The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.Problem/Motivation
In #3228000: Users deleted via JSON:API DELETE don't follow the site-wide cancel_method in the user settings we now respect site settings when deleting a user. This means for example after deletion a batch process is started which deleted/updates all content of that user. This currently is an 'in request' batch process.
As @alexpott metiones in that issue there is a recommendation about asynchronous processing in json:api. This would mean we could actually run the batches in the background and expose this information in the API.
Process should be something like (using users as example).
- Send DELETE to users endpoint
- Respond with 202 Accepted
- Add Content-Location header: (Content-Location: https://example.com/photos/queue-jobs/5234)
- Return a queue-jobs resource type with status data.
That is the recommendation, but perhaps for Drupal we need to massage that a little. Some generic job resource type could be helpfull in different ways though.
I do think though this would be most usefull to work with true queues and queue workers we now support instead of the archaic batch system.
Proposed resolution
tbd.
Remaining tasks
tbd.
User interface changes
tbd.
API changes
tbd.
Data model changes
tbd.
Release notes snippet
tbd.
Comments
Comment #2
bbralaComment #3
wim leersCould … batch API be refactored on top of JSON:API? 🤓🤞 Now that'd would truly push Drupal forward in an API-First direction!
Comment #4
bbralahehe, interesting thought. Kinda interesting though how you would go about that, that would mean always a jsonapi, and mean disable by default i think since you don't want to expose everything. The web that is forming in my head of the implications is a bit large, but still. Interesting though :D
Comment #5
berdir> - Send DELETE to users endpoint
> - Respond with 202 Accepted
> - Add Content-Location header: (Content-Location: https://example.com/photos/queue-jobs/5234)
> - Return a queue-jobs resource type with status data.
FWIW, that's technically not how it works internally in Drupal, you first do a cancel operation, and only after that you delete.
So a new API that is user specific to cancel a user, which takes care of deleting/reassigning content or whatever, or even gives you a choice if the site is designed that way, so you also need an endpoint that returns the cancel-options I guess. And then you're only allowed to DELETE after you went through the cancel process would probably be closer to that.
Second FWIW, #3068764: Refactor cron queue processing into its own class would be neat to have but is IMHO in no way a blocker. I'm not even sure it's needed at all:
Batches already kind of are an API, there is a JSON endpoint that's used for the javascript batch implementation. And access works based on the id + token, so you set up a batch on the server and then anyone who has that information can run it.
So I guess it's not strictly json:api, but it's also already possible to build this in a decoupled client (you could argue that this is one of the few pieces in drupal that already _are_ decoupled out of the box) you just need to tell the client how to initialize it.
Comment #6
berdirClarification: what I wrote about is batch api, not queue. But batch is what you want for the user cancel context, queue is an asynchronous backend processing task. You can't run anything specific there, you can just run it for N seconds. I don't think this is really about the generic queue api?
Comment #7
bbralaThe reason i wrote it like that is because the related issue this is a split of came from the following problem: when DELETE was sent to an user the user and all its content was deleted because of delete hooks. Now this delete respects the site settings as defined. A request like that COULD also send back any queueing that has been set.
I know the current batch api already has an endpoint that gives some information on how the batch is doing, although that was pretty useless in its current form for the related issue. Also the current batch API is pretty highly coupled with the formapi, which ment calling it was a hassle.
The point here is, can we expose queues/jobs that might be spawned from requests in JSON:API, which is an interesting thing to explore. For that it would be helpfull to have batch api stop doing it's own weird thing and move a bit more towards 2021 :)
Comment #8
berdirnothing in drupal allows you to run "a queue job", you can only run a certain queue. A queue is a fire-and-forge thing. You request it, and eventually, in the more or less near future, a background task will execute it. A poormanscron-like runner that is triggered by the browser is still a background task IMHO in this context.
User cancel could be implemented as a queue, but it will need not be immediate then and you will have no feedback when it is done.
If you need to process a specific thing then it's batch API, and yeah, maybe I was a bit too optimistic (you can probably process a batch, but you can't get the complete information I guess), but it's still a very different use case than having access to the generic queue API, which IMHO has far few use cases, the only thing I can think of is building a decoupled queue UI that shows you many items are in a certain queue and force-process them, like drush offers.
Comment #9
catchLet's take 'queues' out of the issue title.
Berdir's right that we can't just use the queue API as it is. There's no connection between adding items to a queue, and processing those items, and there's no feedback.
We should probably think about a new 'task runner' API, that borrows heavily from the queue API - we might even be able to re-use QueueWorkers, but where the queue item creation API is different. Then once we have that, look at converting existing uses of the batch API over to it.
One issue with queues, and the user cancel case is one of them:
When you cancel a user, you can put just the user ID in a queue - this will be the cheapest thing to do for the original cancel request.
But then the queue worker still has to find all of that user's content and update it, if there's thousands and we're running the queues via http requests, we've still got problems.
There's then broadly three approaches:
1. Hit and hope
2. Self-regenerating queue item that chunks some entities, pauses processing, then restarts again, and keeps doing that until there's nothing left. Can probably be done with RequeueException.
3. Cascading queue which creates thousands of little entity update queue items to be handled by a different queue runner.
If we need a specific job ID that we can poll from the browser until it's finished, then something like #2 might work.
Comment #12
mxr576This idea rings a bell to, there can be some overlap in the concept and the implementation with TUS module and library; and how they solve the large file upload problem
https://www.drupal.org/project/tus