Overview

#3575874: Include Pages in the CLI push/pull commands is adding the ability for the Canvas CLI's push command to push pages, including all of its component instances, from the local json-render representation to the Drupal site. However, it isn't yet able to push component instances with values for image props. That will be done by #3578452: Include pages with images in the CLI push/pull commands which is blocked on this issue.

The use case we're trying to enable is while working in a local IDE, to be able to prompt a coding agent to create or change a page, and for the agent to be able to add new image component instances or change existing image component instances to point to new images that the agent finds on the internet, such as on https://unsplash.com/. The CLI tool can fetch the images and upload them as blobs to Drupal, so that we don't need Drupal to fetch the remote images directly.

The goal of this issue is to provide the back-end API that the CLI can use to upload those images into the Drupal site as media entities.

Proposed resolution

Create a route and controller for a POST method where:

  • The request body contains the image blob, the image filename, and alt text.
  • If all validation passes, the controller generates a new file entity and media entity. The filename created on the server can be different than the filename in the request body, just like when uploading to an image field through Drupal's regular UI.
  • The response returns the media entity's ID and UUID.
  • If it's easy enough to do, it would be nice to also return the inputs_resolved of what a Canvas image prop that uses this new media entity would get, so that the CLI can update its local json-render files accordingly. If that's tricky, then we can open a follow-up to discuss options for how to do that or work around the need for it.
  • Drupal core's FileUploadHandler is used (see comment #2).

We do not yet need other HTTP methods (GET, PATCH, DELETE) or a listing API. Those can be done in other issues if/when they're needed.

Issue fork canvas-3578449

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

effulgentsia created an issue. See original summary.

wim leers’s picture

Title: Add or decide to reuse existing HTTP API to create media entites from images or image URLs » Add or decide to reuse existing HTTP API to create media entities from images (upload blob) or image URLs (fetch remote blob)
Issue tags: +Needs security review
Related issues: +#1927648: Allow creation of file entities from binary data via REST requests, +#2940383: [META] Unify file upload logic of REST and JSON:API

⚠️

This is security territory, which I worked on almost exactly 8 years ago in #1927648: Allow creation of file entities from binary data via REST requests, which then resulted in a generalized solution years later : #2940383: [META] Unify file upload logic of REST and JSON:API (which was later applied to both REST in #3401734: Refactor FileUploadResource to use FileUploadHandler and JSON:API in #3444748: Refactor JSON-API file uploads to use FileUploadHandler.)

If we end up doing something custom (i.e. do not reuse JSON:API), then for the "upload blob" use case, we MUST use FileUploadHandler.

I fear that because we also want to support "fetch remote image blob" use case, we do need something custom 😅

I suspect we'll want this to be custom, to avoid this sequence:

  1. POST file blob ⇒ response with File entity target ID
  2. POST JSON blob containing image media entity details including the File entity target ID

because it is both slower (2 round trips) and brittle+complex (much more state management because double the opportunities for network problems).

wim leers’s picture

Component: CLI Tool » Internal HTTP API
Issue tags: +blocker, +Needs issue summary update

Changing issue queue component per The scope of this issue is just to provide the back-end endpoint for the CLI to use. Adding the command itself to the CLI tool will be done in a separate issue.

(This issue should link to the one it blocks once that exists.)

effulgentsia’s picture

Component: Internal HTTP API » CLI Tool
Related issues: +#3578452: Include pages with images in the CLI push/pull commands

I fear that because we also want to support "fetch remote image blob"

We only need EITHER a way to upload a blob OR a way to send remote URLs so that the server fetches the blob. We can choose whichever is easier/safer here, and the CLI command can be written to use that. In other words, if we prefer either option 1 or option 2 over option 3, then the CLI can fetch the remote images and upload them to Drupal as blobs.

#3578452: Include pages with images in the CLI push/pull commands is the issue for the CLI work.

effulgentsia’s picture

Component: CLI Tool » Internal HTTP API
effulgentsia’s picture

Title: Add or decide to reuse existing HTTP API to create media entities from images (upload blob) or image URLs (fetch remote blob) » Add an unstable (v0) "canvas_external_api" POST route for uploading an image blob and getting back the ID and UUID of the created media entity
Issue summary: View changes
Issue tags: -Needs issue summary update

I discussed the options with @penyaskito, @longwave, and @mglaman, and consensus was on option 2 which also matches the recommendation in comment #2. I updated the issue summary accordingly.

effulgentsia’s picture

Issue summary: View changes
penyaskito’s picture

Assigned: Unassigned » penyaskito

penyaskito’s picture

Status: Active » Needs work
penyaskito’s picture

penyaskito’s picture

penyaskito’s picture

Status: Needs work » Needs review
wim leers’s picture

The scope change in #6 makes it seem like this is related to #3532574: `ContentTemplate` config entities must support exporting images in `StaticPropSource`s that reference media items: `File` content entity dependency, base64-encoded upon export .

However, I think it's actually explicitly out of scope because despite the issue title being very generic, AFAICT it's solely about Canvas WorkBench to be able to set Canvas Page content entities' "image object" props' StaticPropSources? If so: can we update the title+summary to reflect that tight scope? 😊

(It'd be good to know that we don't intend the Canvas UI to start using this.)


@penyaskito did a high-level walkthrough of the MR. Thanks! Overall looks good, some security-related concerns, but definitely on the right track.

penyaskito’s picture

Assigned: penyaskito » mglaman
Status: Needs work » Needs review

I've resolved @Wim's feedback on the MR (not the issue metadata though). @attilatilman made a really good question, which I already had thought about and discarded. Maybe @mglaman can break the tie?

mglaman’s picture

Assigned: mglaman » Unassigned

Replied! I think we're doing the best we can with the current way.

penyaskito’s picture

Assigned: Unassigned » mglaman
mglaman’s picture

Assigned: mglaman » Unassigned

addressed the Q