Overview

#3475363: [exploratory] PoC of Astro island components editable via StackBlitz has a proof-of-concept of editing the source code of JS framework (e.g., Preact, Svelte, etc.) components within the XB UI. It uses StackBlitz for the editor and preview, and a StackBlitz WebContainer for running the npm run build step in the browser. A big advantage of StackBlitz and their WebContainer technology is that since it's able to run Node.js in the browser, it's able to run virtually any JavaScript project's dev server and build step in the browser. However, a disadvantage is that it's a commercial service that requires the end-user to purchase a license if they're using it for commercial purpose.

Therefore, in this issue we want to create a similar PoC, but without StackBlitz. For the code editor, the two leading options are CodeMirror and Monaco, so we'll need to choose which one to try out first (we can always switch to the other one later if we learn that our initial choice wasn't the best one). But then the question is how to compile the JS source code that the user enters into the editor into JS code that can run in a browser, both for showing a preview, and ultimately for the actual site as well. I.e., how to do the build step without Node.js.

We're looking into https://swc.rs/docs/usage/wasm for that. This, however, will likely not support all JS frameworks. Therefore, for this PoC we'll focus on just Preact+Tailwind, with no usage of any other library.

We're also looking into whether we can still wrap the Preact components into Astro islands, similar to #3475363: [exploratory] PoC of Astro island components editable via StackBlitz, for easier integration into XB.

Proposed resolution

User interface changes

CommentFileSizeAuthor
#22 Screenshot 2024-11-13 at 6.42.00 PM.png74.53 KBhooroomoo
#15 Screenshot 2024-11-07 at 6.19.47 PM.png37.4 KBhooroomoo
#14 xb-js-components-tw-sketch.png122.93 KBbalintbrews
#9 xb-js-component-preact-tailwind-poc.gif34.65 MBbalintbrews

Issue fork experience_builder-3483267

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

effulgentsia created an issue. See original summary.

hooroomoo made their first commit to this issue’s fork.

johnwebdev’s picture

johnwebdev commented

Have you considered vscode in web? Ah Monaco is vscode :-)

effulgentsia’s picture

effulgentsia commented

Monaco is a component of VSCode, but VSCode does a lot more than Monaco. https://code.visualstudio.com/docs/editor/vscode-web is an interesting option, but does it work in a cross-origin iframe?

xinran made their first commit to this issue’s fork.

balintbrews made their first commit to this issue’s fork.

effulgentsia’s picture

effulgentsia commented

the two leading options are CodeMirror and Monaco

https://npmtrends.com/@codemirror/view-vs-monaco-editor shows how basically tied they are in terms of npm downloads.

we'll need to choose which one to try out first (we can always switch to the other one later if we learn that our initial choice wasn't the best one)

Monaco has some appeal just because it's backed by Microsoft and it's the "just the editor" portion of VSCode and VSCode is great. However, there's a few downsides:

  • Its version is still 0.x and has been for years. I'm not aware of any announcement as to whether a 1.0 release is even remotely close.
  • https://microsoft.github.io/monaco-editor/ says it's not supported on mobile.
  • https://blog.replit.com/code-editors is a blog post that's a few years old now, so not sure how much of it is still accurate, but it points out some other challenges with Monaco.

I think this makes CodeMirror the better option at least for now, but I'm curious what others think.

hooroomoo opened merge request !384

balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented
StatusFileSize
new xb-js-component-preact-tailwind-poc.gif34.65 MB

This is still very much in progress, but here is a sneak peek into where we are currently.

Compiling Preact and Tailwind in the browser

balintbrews credited f.mazeikis.

balintbrews credited longwave.

balintbrews credited tedbow.

balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented
balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented
StatusFileSize
new xb-js-components-tw-sketch.png122.93 KB

Here is an outline of our approach for handling Tailwind-generated CSS — developed based on several meetings with @effulgentsia, and another meeting with @effulgentsia, @hooroomoo, @tedbow, @longwave, and @f.mazeikis.

Experience Builder's Tailwind CSS support for JavaScript components

JS components vs. Tailwind support sketch

JS components in XB have two target groups:

  1. Marketing teams with sufficient level of design and frontend coding skills to author (and/or use LLMs to generate) basic components, with no or minimal interactivity, mostly consisting of markup.
  2. Developers, who can create advanced, potentially highly interactive components.

80-90% of the JS components will be authored by marketing teams. Therefore we will consider XB as the primary source of truth for JS components. This mostly means that the Tailwind CSS 4 config will be maintained by the Experience Builder module, rather then e.g. residing in an external code repository.

Basic components

Marketers can write Preact components using an in-browser editor. They can also maintain their Tailwind CSS 4 config using Experience Builder. Tailwind CSS 4 is still in alpha, but there are already examples of it being used in production. One of the great new features is CSS-first configuration.

We are already able to build CSS using Tailwind CSS 4 in the browser via a new package that has been authored and published on npm: tailwindcss-in-browser. Using this we will produce the following CSS files using the components' markup and the Tailwind CSS 4 config maintained in XB:

  1. Tailwind Preflight & Tailwind theme CSS;
  2. Individual CSS files with Tailwind utility classes for each component.

Advanced components

Developers can develop components outside of Experience Builder using any workflow that fits them, e.g. keeping their code in a code repository. XB will provide a CLI tool to support the followings:

  • Preview components by building CSS using the Tailwind CSS 4 config, retrieved from XB on-the-fly;
  • Deploy components to XB with their built CSS — an individual CSS file for each.

CSS aggregation

Every JS component, basic or advanced, will end up with their own CSS file. While this will ensure great portability and reduces complexity for the initial implementation, it also results in a great deal of duplication in the CSS code. It was agreed upon that this is acceptable for #3455753: Milestone 0.2.0: Early preview, and can potentially be addressed later in Drupal core, implementing de-duplication as part of the CSS aggregation process.

hooroomoo’s picture

hooroomoo
they/them
Location nyc🗽
commented
Issue summary: View changes
StatusFileSize
new Screenshot 2024-11-07 at 6.19.47 PM.png37.4 KB

Ok my update: passing in the SWC compiled code into an astro island looks like it is working. There is still a lot of cleaning up to do. The editor is not connected to the SDC wrapper/astro island yet, I just used a copy and paste of the compiled SWC code for a simple counter which lives in Counter_SWC.js in the gist linked below.

Currently all the files, including those relevant to the renderer-url attribute of the astro island live in my /sites/default/xb/astro and that is what the code is pointing to but that should change in the future. For now, I created a gist with those files if anyone wants to test it.

https://gist.github.com/harumijang/86ed1c6148690404d02ef322d0eddd37

One thing I am not sure about is in Counter_SWC .js, is where I am supposed to be calling the render function and to what part of the document I should inject it into.

effulgentsia’s picture

effulgentsia commented

One thing I am not sure about is in Counter_SWC .js, is where I am supposed to be calling the render function and to what part of the document I should inject it into.

I don't think you need to call render(). I think you only need to export { Counter as default };.

The JS in the renderer-url is what calls render(). For the @astrojs/preact integration, that's done in https://github.com/withastro/astro/blob/main/packages/integrations/preac.... The JS code in component-url only needs to export the component. There might be some tricky bits to figure out such as if the component code has dependencies on Preact functions, where to get those from, but I think that's the high level outline of it.

hooroomoo’s picture

hooroomoo
they/them
Location nyc🗽
commented

Steps to test SimplePreactCounter to the preview canvas:

  1. Add Astro bundled files located in /ui/components/editor/astro-bundles to /sites/default/files/xb/astro
  2. In the XB UI, go to Library Tab and click Simple Preact Counter and open it in editor.
  3. Click the blue Upload button in the top bar. This button will write the SWC-compiled code to your /sites/default/files/xb/astro with some manual changes with the imports. @see JSComponentUploadController.php for a comment
  4. Refresh XB to close the editor since we don't have a a button to close it yet, and under the Library's component's tab you should see Simple Preact Counter that you can drag into the preview canvas. It should be hydrated but with the current implementation of the preview canvas you can't interact with it but you can go into your console and document.querySelector(xxxxx).click(); to test the counter

Important note, the component currently requires an export default to work with the astro bundles which is why the SimplePreactCounter uses the inline export default function syntax.

The code in JSComponentUploadController.php to handle the SWC-compiled file will need to be updated to handle different imports since imports are currently hard-coded. Based on a conversation with @effulgentsia ideally later on we could just rely on the Astro-bundled Preact packages instead of also bringing in Preact CDN through importmap. Later on we also want to change how either SWC or Astro bundles their JavaScript to be mutually compatible there are some differences in the compiled code between the two and resolving this might make it smoother for the SWC compiled code to use the Astro bundles.

effulgentsia’s picture

effulgentsia commented

We're also looking into whether we can still wrap the Preact components into Astro islands

Yay, #17 proves that we can! Because Astro doesn't insert any special sauce into the JS for the component-url: that's just vanilla Preact so @swc/wasm-web is sufficient for compiling that. Meanwhile, all of Astro's special integration code, from the JS for the renderer-url to its JS for the <astro-island> custom element can be generated statically rather than in-browser. So that's all very promising!

Later on we also want to change how either SWC or Astro bundles their JavaScript to be mutually compatible

I looked a bit into this, and I think all we need for this is to set SWC's jsc.transform.react.runtime configuration to automatic. That compiles to the jsx() function introduced in React 17, which is more efficient at creating vnodes at runtime than the older createElement()/h() function. So that's what Astro's Preact integration (and pretty much everything else in the React ecosystem) uses by default at this point, and SWC's default of classic is pointless for new projects.

Based on a conversation with @effulgentsia ideally later on we could just rely on the Astro-bundled Preact packages instead of also bringing in Preact CDN through importmap.

I think I figured out how we can do this. The challenge here is how to make Astro bundle its JS in such a way that it exports the original, rather than the minified, names of the Preact functions that we want components to be able to import. Normally, Astro takes care of bundling the components, so it can minify the export names of library functions since it can compile the components to import those minified names. In our case, since we're compiling the component code separately from the Astro code, we need any Astro bundles that we want to import from to export un-minified names. Astro uses Vite which uses Rollup, but I couldn't find any Astro/Vite/Rollup configuration to accomplish this. However, I found the following indirect way to accomplish it.

Within the Astro project, we can add a Stub component like this:

const { useState } = await import("preact/hooks")
const { useSignal } = await import("@preact/signals")
const { jsx, jsxs, Fragment } = await import("preact/jsx-runtime")

export default function () {
}

And then add a <Stub client:only="preact"/> usage of it in the index.astro file. Because this Stub component uses dynamic imports, it results in Rollup exporting the useState, useSignal, jsx, jsxs, and Fragment functions, with those names, from the corresponding module bundles. If we want to expose additional functions/hooks for in-browser-editable components to be able to use, we can add those as well to the Stub component, but for now, let's just keep it to these.

With this in place, we can then take the output of SWC's compilation and replace from 'preact/hooks' with from './hooks.module.js', and similarly for signals and jsx-runtime.

effulgentsia’s picture

effulgentsia commented

Just jotting some notes down from a conversation with @balintbrews and @hooroomoo...

When you run astro build, it generates preact.module.js, hooks.module.js, and some other JS files that the in-browser-editable components depend on. So the question is how do we want to get those assets "into Drupal".

I think the most straightforward way would be to just add an astro build step to XB's build step. In other words, XB's current build step is npm run type-check && vite build so conceptually we could expand that to npm run type-check && vite build && astro build.

However, XB is a React project in the ui directory. It would probably be good for the Astro project to be a separate project (meaning, have its own package.json) from the main XB project. This separate project could be either a directory that's a sibling of the ui directory, or a child. For example, astro or ui/astro (or we might want to come up with some other name for it than just calling it astro). In which case, we'd want the build script in ui/package.json to do whatever it does plus then kick off an npm run build command within the astro directory. I'm guessing there's idiomatic conventions for how to structure/implement this type of setup, but I don't know what that is.

effulgentsia’s picture

effulgentsia commented
Title: [exploratory] PoC of Preact+Tailwind components editable via CodeMirror or Monaco » [exploratory] PoC of Preact+Tailwind components editable via CodeMirror

The MR here is currently using CodeMirror and no strong opinions have yet been raised making the case for why Monaco would be better, so retitling to reflect the current state. We're still open to switching to Monaco if a strong case is made for that, but in addition to comment #7, I'd like to point out that:

  • CodeMirror is used as the code editor within the dev tools of Chrome, Safari, and Firefox.
  • The experience described in https://sourcegraph.com/blog/migrating-monaco-codemirror also matches my (much more limited) initial positive impressions of CodeMirror and frustrations with Monaco. There's a lot to like about the user experience of Monaco, so I hope Microsoft evolves it into something that can also be made lighter weight and easier to write plugins for.
balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented

#19:

It would probably be good for the Astro project to be a separate project (meaning, have its own package.json) from the main XB project. This separate project could be either a directory that's a sibling of the ui directory, or a child. For example, astro or ui/astro (or we might want to come up with some other name for it than just calling it astro). In which case, we'd want the build script in ui/package.json to do whatever it does plus then kick off an npm run build command within the astro directory. I'm guessing there's idiomatic conventions for how to structure/implement this type of setup, but I don't know what that is.

Tools I'm thinking of that we can evaluate are Yarn workspaces, Lerna, or Nx (also uses Lerna, probably an overkill for us, but it's an awesome tool).

hooroomoo’s picture

hooroomoo
they/them
Location nyc🗽
commented
Issue summary: View changes
StatusFileSize
new Screenshot 2024-11-13 at 6.42.00 PM.png74.53 KB

Olivero(?) styles bleeding through but yay Preact component using Tailwind css being rendered in the Preview canvas.

balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented

I just learned about @brianperry's module, Islands. We could explore how to leverage it for our hydration logic with Astro.

effulgentsia’s picture

effulgentsia commented

Oh that's neat: thanks, @brianperry, for creating that!!

Looks like that module is using 11ty instead of Astro. Not sure if that really matters for us. We started here with Astro because that's a very popular and well maintained framework, but it's quite possible that 11ty is sufficient for our needs here. @brianperry: what are your thoughts on pros/cons between 11ty islands vs. Astro islands?

brianperry’s picture

brianperry
he/him
commented

Thanks for sharing here @balintbrews. Following along with this issue and other XB work has pretty directly led to experiments like this, so happy to share anything I can.

> @brianperry: what are your thoughts on pros/cons between 11ty islands vs. Astro islands?

@effulgentsia the main reason I started with 11ty's implementation is that Astro didn't seem to have an easy way to use the astro-island element outside of a full astro project (unless I'm missing something). 11ty's implementation is focused on exactly that. The readme calls it "a framework independent partial hydration islands architecture implementation" and it doesn't require 11ty to use.

It's a lot simpler than I expected once I dug into it. Most of the code is the the expected client loading directives, and then it adds some simple utilities for module imports, automatically initializing various frameworks, and replacing fallback content.

Being framework independent also means that pretty much everything the package does is on the client. So it won't help with any of the things the MR on this issue does around compiling. It can work with 11ty's server rendering features, but it could work SSR from other frameworks, or as this module is trying to prove - content server rendered by Drupal.

The other noteworthy difference is that @11ty/is-land doesn't seem to have built in support for React. I don't know if this is directly related to the challenges of using JSX without a compile step, or just that the maintainer isn't a big fan of React based frameworks.

So for what this issue is setting out to do, my gut would be that you're still better off with Astro. The Islands module currently isn't much beyond the provider of a re-named is-land element, along with a naming convention based approach to import maps that will become obsolete once Drupal has formal support. I have issues in the queue to go deeper into dealing with Drupal content, and also things that will require some kind of compilation step - if I come up with anything interesting I'll be sure to share it here and/or in Slack.

hooroomoo opened merge request !425

hooroomoo changed the visibility of the branch blocks-spike to hidden.

wim leers’s picture

wim leers
Location Ghent 🇧🇪🇪🇺
commented
Assigned: Unassigned » balintbrews
Issue tags: +Needs issue summary update

What are the next steps here? Is this ready for review? I asked @f.mazeikis to review the config bits of this MR already, to get this going again.

(Doing this because @effulgentsia told me this is the biggest unknown/highest risk for #3455753: Milestone 0.2.0: Early preview.)

balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented
Assigned: balintbrews » wim leers

There is a lot of code in multiple MRs in this issue which were written to prove concepts and ideas.

A short summary of what we proved:

  1. We can provide a code editor using CodeMirror;
  2. Compile and bundle JavaScript code authored as Preact components using the WebAssembly build of SWC (Speedy Web Compiler);
  3. Used the bundled code with import maps to provide a live preview;
  4. Compiled CSS in the browser using Tailwind CSS v4 (currently at beta.9) that resulted in this library: tailwindcss-in-browser;
  5. Built UI for defining props for components where the default values were also presented in the preview;
  6. Architected a solution leveraging Astro's islands architecture to hydrate the components when rendering them server-side;
  7. Solved overriding the menu block template with a JS component;
  8. Implemented the first version of the config entity which will store the JS components.

I'm sure there is more. With all these challenges our focus was the build a POC, to see what is possible. Because of that, the code we've written is mostly not ready for prime time.

Here is what I would recommend. Now that we have a clear idea of what we would like to build, and we now also have designs (not sure if they're ready to be shared yet), I would propose that we close this issue, use it as a reference moving forward, and create a meta issue that maps out what needs to be done in a series of smaller child issues. That way the progress will be more visible, it's easier to prioritize features, and we can plan it better what we can work on in parallel. I'd be happy to work on the meta issue and build out the plan.

Thoughts, @wim leers, @effulgentsia?

effulgentsia’s picture

effulgentsia commented
Assigned: wim leers » Unassigned
Status: Active » Closed (outdated)
Issue tags: -Needs issue summary update +Needs followup

I agree with all of #29. I'm conflicted about whether to mark this Fixed (as in, we learned what we needed to from the PoC) or Outdated, but choosing the latter to reduce potential confusion in someone thinking that we delivered any usable implementation as of yet.

effulgentsia’s picture

effulgentsia commented
Assigned: Unassigned » balintbrews

@balintbrews: When you create the new meta, please make #3498889: ComponentSource plugin for code components a child of it. I opened it just now realizing that's a piece that wasn't part of the PoC at all. There's probably some other pieces too that'll require entirely new work rather than just polishing what's in this MR.

larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
commented

@effulgentsia

In which case, we'd want the build script in ui/package.json to do whatever it does plus then kick off an npm run build command within the astro directory. I'm guessing there's idiomatic conventions for how to structure/implement this type of setup, but I don't know what that is.

I think npm workspaces might be what you're after it lets you built multiple packages in a single monorepo.

Astro supports rollup plugins and is self described as 'built on top of vite' so if we can unpick what 'astro build' is in vite terms, we might be able to use library mode with vite and pass multiple entry points. Looking at the source code it appears to just map to esbuild https://github.com/withastro/astro/blob/main/scripts/cmd/build.js#L67

balintbrews’s picture

balintbrews
he/him/his
Primary language Hungarian
Location Netherlands 🇪🇺
commented

I think npm workspaces might be what you're after it lets you built multiple packages in a single monorepo.

Adding this to the list I recommended in #21.

Astro supports rollup plugins and is self described as 'built on top of vite' so if we can unpick what 'astro build' is in vite terms, we might be able to use library mode with vite and pass multiple entry points. Looking at the source code it appears to just map to esbuild https://github.com/withastro/astro/blob/main/scripts/cmd/build.js#L67

This will be very relevant in #3500058: Hydration library for code components. I updated that issue summary to reference your comment.

brianperry’s picture

brianperry
he/him
commented

> I think npm workspaces might be what you're after it lets you built multiple packages in a single monorepo.

I talked myself out of adding this comment previously because I know core is standardized on Yarn, but pnpm is extremely good at this as well. https://pnpm.io/workspaces

I'd also add a -1 against Lerna. All I ever hear about Lerna these days is people desperately trying to migrate away from it.