Add Tugboat integration

I guess this changes will be somewhat like: https://www.drupal.org/project/drupal_cms/issues/3471003

I'd be willing to help out with this, though I do not know anything about Experience Builder.

Ideally we would have the node/1 created there by default so that all you would have to do is to log in and navigate to Experience Builder.

Is there a programmatic way you typically do this, or is this usually a manual task (logging in via the browser to create the node)?

I don't see a programmatic way to create that node. I guess may be we do something with drush generate-content

@q0rban: Yay, thank you! 🚀

We currently do this manually when developing XB:

# See it in action + recommended development environment
1. Drupal 11 (preferably a git clone, for git archeology — 10.3 will work too).
2. `composer require drush/drush`
3. `drush si standard`
4. `drush pm:install experience_builder`
5. Browse to `/node/add/article` — you'll see a `🪄 XB Demo ✨` field. Don't touch that — just enter a title for the article and hit save: a component is rendered using the article title 🤓
6. If you're curious: look at the code, step through it with a debugger, and join us!
7. If you want to run *all* tests locally, including the OpenAPI spec one: `composer require league/openapi-psr7-validator webflo/drupal-finder devizzent/cebe-php-openapi --dev`

— /CONTRIBUTING.md

But we also have and end-to-end test that literally performs those steps, added months ago (#3452497: [MR Only] Edit any component prop, powered by a new FieldForComponentSuggester service, which will power the JS UI) in the super early XB days, but still serving its purpose: \Drupal\Tests\experience_builder\Functional\EndToEndDemoIntegrationTest::test()

And the Cypress end-to-end tests (similar to Nightwatch, but better) perform similar steps:

    $field_definitions = \Drupal::service('entity_field.manager')->getFieldDefinitions('node', 'article');
    $image_field_sample_value = ImageItem::generateSampleValue($field_definitions['field_hero']);
    $node = Node::create([
      'type' => 'article',
      'title' => 'XB Needs This For The Time Being',
      'field_hero' => $image_field_sample_value,
    ]);
    $node->save();

— \Drupal\Tests\experience_builder\TestSite\XBTestSetup::setup() (@abhisekmazumdar in #4: look at that 😊)

Proposal: if you can make this work to the point where it installs the Standard install profile, creates the first article node (and perhaps fails for now) and then navigates to /node/1, I'm happy to fill in the few missing pieces. 😊😇

@q0rban, this is your world. How can we leverage this into the config.yml file? I see the WIP MR.

I can only think of a custom drush command, but I'm not sure if that would be overkill for this requirement. Alternatively, may be we could script this out and use it in the config.yml.

I will keep my eye on the MR.

Okay, I think I've got this working with help from @marcoscano, if someone could add him as a contributor to this. I'm not sure how to do that.

The MR looks good:

• The Preview works: "You should see a link under the MR to the preview. Once this is merged in, that link will show on any MRs."
• Username/password: admin/admin
• Node is created with hero image. Which is perfect.

TODO:

Build the React app. Steps mentioned on MR comment.

Thank you @abhisekmazumdar

What version of nodejs should be used for those npm commands?

@q0rban, there is an .nvmrc file in the ui folder to define the required Node.js version: v20 at the time of writing this comment.

Thank you @balintbrews. I pushed up a fix to install nodejs and run the npm commands. Did that fix things? I'm not sure how to test.

On a side-note, this is my first time working with nvm, it's not really tailored for CI environments, is it? I couldn't find docs for using it in CI. Seems like it's more geared for human usage with a logged-in shell. Definitely open to suggestions if I'm doing something silly.

Not sure why we still see

Loading Experience Builder…

for /xb/node/1

Also

#13 implies that the npm run build command is failing (or isn't getting reached). Is there a way to see a log of what Tugboat outputs when it runs the nvm use, npm install, and npm run build commands?

[...] nvm, it's not really tailored for CI environments, is it? I couldn't find docs for using it in CI. Seems like it's more geared for human usage with a logged-in shell.

Whether that's the case, I'm not completely sure. One change that might be useful in .tugboat/config.yml is doing nvm install instead of nvm use, which will not only install the specificed Node.js version, but will also start using it.

With that said, I'm not fully convinced if we even need to use nvm in our CI. We won't be changing Node.js versions very often, so just setting it to the same major version we currently have in our .nvmrc would be sufficient in my opinion. I realize I pointed at the .nvmrc file, but if using nvm introduces challenges, I think it's fine to let that go.

One change that might be useful in .tugboat/config.yml is doing nvm install instead of nvm use, which will not only install the specificed Node.js version, but will also start using it.

nvm install is being run in the init command group: https://git.drupalcode.org/project/experience_builder/-/merge_requests/2...

#13 implies that the npm run build command is failing (or isn't getting reached). Is there a way to see a log of what Tugboat outputs when it runs the nvm use, npm install, and npm run build commands?

Thank you, @effulgentsia. Here is the output of npm run build. Any clues there?

 Run npm install
. /root/.nvm/nvm.sh
cd $TUGBOAT_ROOT/ui
nvm use
npm install
npm run build

Found '/var/lib/tugboat/ui/.nvmrc' with version <20>
Now using node v20.17.0 (npm v10.8.2)
npm warn deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported
npm warn deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
npm warn deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported
npm warn deprecated @humanwhocodes/object-schema@2.0.3: Use @eslint/object-schema instead
npm warn deprecated @humanwhocodes/config-array@0.11.14: Use @eslint/config-array instead
npm warn deprecated @babel/plugin-proposal-optional-chaining@7.21.0: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-optional-chaining instead.
npm warn deprecated @babel/plugin-proposal-numeric-separator@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-numeric-separator instead.
npm warn deprecated @babel/plugin-proposal-nullish-coalescing-operator@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-nullish-coalescing-operator instead.
npm warn deprecated @babel/plugin-proposal-class-properties@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-class-properties instead.
npm warn deprecated @babel/plugin-proposal-private-methods@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-private-methods instead.

added 982 packages, and audited 983 packages in 42s

202 packages are looking for funding
  run `npm fund` for details

2 moderate severity vulnerabilities

To address all issues, run:
  npm audit fix

Run `npm audit` for details.
npm notice
npm notice New patch version of npm available! 10.8.2 -> 10.8.3
npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.8.3
npm notice To update run: npm install -g npm@10.8.3
npm notice

> vite-template-redux@0.0.0 build
> npm run type-check && vite build


> vite-template-redux@0.0.0 type-check
> tsc --noEmit --pretty

vite v5.3.1 building for production...
transforming...
✓ 1041 modules transformed.
rendering chunks...
computing gzip size...
dist/index.html          0.53 kB │ gzip:   0.33 kB
dist/assets/index.css  704.40 kB │ gzip:  85.65 kB
dist/assets/index.js   693.58 kB │ gzip: 227.21 kB

(!) Some chunks are larger than 500 kB after minification. Consider:
- Using dynamic import() to code-split the application
- Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
- Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
✓ built in 7.62s

Hm, that's curious. So according to the end of #18, it's creating dist/assets/index.js and it's in the $TUGBOAT_ROOT/ui directory when it does that, and yet
https://mr253-cojczkjc816gddq8fdjhi8vgvfblp7k3.tugboatqa.com/modules/con... returns a 404. That's a mystery.

Ah, interesting, I think I see what's going on. The module is being pulled into drupal/modules using composer + git, but I am running those npm commands inside of the checkout of the module, not the composer-installed location.

Okay, I think I fixed that issue. I'm still not entirely sure what I'm looking for to see if it's working.

Yes it works 🎉 : https://mr253-cojczkjc816gddq8fdjhi8vgvfblp7k3.tugboatqa.com/xb/node/1

I can see the XB and all its components with default content as well.

#3472299: Update default config to make a fresh install result in an XB UI with an empty canvas will simplify this. And because that's in #3454094: Milestone 0.1.0: Experience Builder Demo, postponing this.

But that means this will only become simpler, so I'm confident we can merge it immediately after that! 😄🚀

Thank you, @q0rban! 👏

Yay, awesome to see this working now!!

The module is being pulled into drupal/modules using composer + git

Is Composer pulling in the version that's in the MR? Or is it only pulling in the branch tip without the changes in the MR? The MR in this issue isn't making any changes to the module code, but if the goal of Tugboat is to test out future MRs that do, we should make sure that actually works. I'm asking because the fact that what Composer puts into drupal/modules is a separate git clone rather than a symlink to what's in TUGBOAT_ROOT seems suspicious.

I didn't review the MR in detail — I assumed that this would just follow the pattern that #3471003: Integrate with Tugboat to allow previewing merge requests used, whose very purpose was literally what @effulgentsia described in #24.

So: can you confirm that, @q0rban? 🤞

The short answer is yes, each new MR would have a preview with the changes that are in the MR applied.

The long answer requires an explanation of some of the steps in the config.yml. These steps come from https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa....

We are using drupal/recommended-project and using composer to require the MR branch into it. The way we are achieving that is as follows:

First, we create a branch that we can reference later. We use the unique Tugboat repo ID as the branch name so that we don't accidently conflict with a real branch that a contributor may have:

        # Check out a branch using the unique Tugboat ID for this repository, to
        # ensure we don't clobber an existing branch.
        git checkout -b $TUGBOAT_REPO_ID

Then, inside the composer root of drupal/recommended-project, we set up a composer repository to point to the git checkout of the MR, and composer require experience_builder using the unique branch name above:

        # We configure the Drupal project to use the checkout of the module as a
        # Composer package repository.
        composer config repositories.tugboat vcs $TUGBOAT_ROOT

        # Now we can require this module, specifying the branch name we created
        # above that uses the $TUGBOAT_REPO_ID environment variable.
        composer require drupal/experience_builder:dev-$TUGBOAT_REPO_ID

I believe using composer instead of a symlink is better, as you can be sure that any composer scripts are run as they would for an end-user. These are also the same steps that other d.o Tugboat integrations are using with success, so using them adheres to the principle of least astonishment for anyone else who may contribute to this config.yml in the future.

I am open to ideas and suggestions for improvement!

#26 makes total sense. Thanks for the explanation! That all looks correct to me.

From https://getcomposer.org/doc/05-repositories.md#path:

The local package will be symlinked if possible, in which case the output in the console will read Symlinking from ../../packages/my-package. If symlinking is not possible the package will be copied.

I'm not clear enough on Composer internals to know why in this case it isn't symlinking, but I guess it doesn't matter: it's getting the correct content which is all that matters.

Oh, maybe because I am using vcs instead of path. I didn't know about that option! That seems like it would simplify things for sure. I'll check it out. Thank you @effulgentsia!

I made the change to path instead of vcs. It required modifying the composer stability to dev. So now the module in Drupal is a symlink to the git checkout. It simplified the config, so thank you for the recommendation.

#3472299: Update default config to make a fresh install result in an XB UI with an empty canvas landed!

Sounds like @effulgentsia is the most qualified person to review this one 😄 But I spotted some problems too. Once my feedback is addressed, I defer to @effulgentsia 🤓

Thank you! Feedback addressed. Over to @effulgentsia.

I'd like @effulgentsia to do a final pass, but LGTM! 😊

Thank you!

This looks great. Thank you, @q0rban, for driving this, and everyone else for your help!

Is it 1) possible, and 2) desirable, to add a link to the XB project page to let people view the current state of XB (branch tip of 0.x) in Tugboat? For example, perhaps a new item #4 under "Track progress"? Are there docs somewhere for how to link to a Tugboat deployment like that?

I opened #3475652: Should we add SDDS to XB's Tugboat? as a followup.

#35: desirable: +1 Possible: I don't know, but I bet @q0rban knows? :D

Back to RTBC to get his input :) Tagging to ensure we don't forget #35.

FYI: the first Tugboat-accelerated review is a fact: #3469672-30: The XB annotations and labels should not change size when zooming! 🚀

What's needed for the "view live preview" Tugboat link to show up under an issue's MR? Does it need the issue status to be a certain value? Or the MR to be a non-draft? Or tests to pass? I'd like to have it in #3475363: [exploratory] PoC of Astro island components editable via StackBlitz but it's not showing up there.

The previews expire after 5 days. To recreate it, you can close the MR and reopen it.

If that still doesn’t fix it, make sure the branch for the MR has the tugboat config in it.

Closing the MR and re-opening it did it. Thanks!

@q0rban Based on #40, it sounds like #35 is not possible?

@q0rban Based on #40, it sounds like #35 is not possible?

It's not something built into the drupal.org integration presently, though I do really like the idea. This took a few hours to cobble together using the Tugboat API, nodejs backend, and react frontend, and does basically what you are asking for, albeit for the 0.x of drupal_cms. We could host something similar for this module, but I'd much rather have something built-in to d.o so that anyone could do it, if we can get the DA's support and some drupal dev help. What do you think?

I like it! :D

Closing this issue since the initial Tugboat config is in.

I'll open another issue for a "Demo 0.x" button based on the discussion from #35.

Follow-up issue: #3479646: Demo Experience Builder button on project landing page