From #3176918-21: [policy, no patch] Publishing / Maintaining JS libraries produced by Drupal that do not have a dependency on Drupal

this isn't really an issue with JavaScript - our problem is really the opposite that it gets stagnant in core.

I would add that when there is a momentum it is very easily stopped (for various reasons) and "simple" tasks end up taking years to complete. The situation with how JS is handled as a whole in core today is not simply frustrating, it is not sustainable.

Problem/Motivation

  • It takes too much effort to get JS changes reviewed and committed
  • The amount of effort necessary to keep momentum going on large JS issues is hard to justify for volunteers

We can see that for the last few years, all "large" JS contribution/feature/refactoring have been from paid/sponsored contributors while other important issues are stalled because they are not the priority of the entities paying contributors.

The maturity of PHP and JS is very different in Drupal and while on the PHP side there has been a reckoning and people grew together over the years to get better it has not been the case for JS. The problem is that now the same expectations of maturity apply to JS without all the growth and people (core js folks is still pretty much the same volunteers for a number of years and 1 or 2 person from Acquia drupal acceleration team).

The following topics are pain points and resolving them would make maintaining JS in Drupal more sustainable and help us make sure it's still possible to contribute meaningfully to core JS as a volunteer:

Topic Issue
JS contribution experience #3179415: Improve contribution workflow, build, and management of JS files
#2984405: Formalize a structure for static assets in modules and themes
#1663622: Change directory structure for JavaScript files
Improving JS while keeping BC #3179174: [policy, no patch] Updating core JavaScript APIs gracefully.
#2898533: Better define the backwards compatibility/API policy for JavaScript
Standalone JS that "Drupal" maintain #3176918: [policy, no patch] Publishing / Maintaining JS libraries produced by Drupal that do not have a dependency on Drupal
Third party libraries #1787222: [meta] Strategy for updating vendor JS libraries within a major stable version
Browser support #3080068: [policy, no patch] Define usage heuristics for browser support

The heart of the issue for me is that we can't seem to keep any momentum on JS work. The topics listed above would help reduce friction for people to contribute and would help getting more contribution in the queue, that doesn't mean we'll get more things committed.

Proposed resolution

We need more committers with a "hands-on" approach on JS issues, there are some very big changes necessary to remove jQuery, open up Drupal to the larger JS community and I don't see how it'll be possible to do that in the current situation.

What would help:

  • Add 1 or 2 core committers dedicated to JS (they need to be "hands on" to some extend, taking part, and initiating topics)
  • Have a much higher commit frequency (help work going on smaller JS issues to get more people or remotivate existing contributors)
  • Reduce expectations for JS patches on the short term (to increase commit rate and have more people involved) and raise expectations progressively (we'll need to define and advertise the different milestones)

Reducing expectations would possibly add more bugs, but higher commit rate can help mitigate that problem, also it would give an opportunity for novice by having smaller high impact bugs that help onboard and hook new people. I think the memories of ViewsUI breaking after JS patches can explain some of the aversion to JS changes, but we need to find a way past this. The situation is different, and what we need to do to keep Drupal relevant requires significant changes.

Comments

nod_ created an issue. See original summary.

nod_’s picture

Issue summary: View changes
nod_’s picture

Another issue is the fact that we have 2 versions of every js file in the source code of Drupal. It keeps tripping up new contributors and discourage others.

Speaking about maturity, we could take inspiration from Neos (CMS based on Typo3 with a ReactJS admin, formerly using emberjs): they have a repository for compiled assets neos-ui-compiled exposed as a composer package and that package is depended on by the package neos-ui with the source of the JS/CSS files. That compiled package is updated automatically and sites are depending on it directly.

All this to say that having 2 versions of js files in the source and expecting contributors to know which ones to update is a significant barrier to contribution (I don't see many people reading the warning on top of compiled js files).

bnjmnm’s picture

All this to say that having 2 versions of js files in the source and expecting contributors to know which ones to update is a significant barrier to contribution (I don't see many people reading the warning on top of compiled js files).

Now that you mention it, I recall experiencing this when I got started on Drupal 8. Some reasons I was thrown off:

  • I definitely ignored the warning, subconciously assuming it was a another perfunctory @file doc providing little information.
  • The complied files weren't obviously compiled. Incoherent code is one of the most obvious cues that a file is compiled and not meant for direct editing. These js files were quite readable.
  • There wasn't the cue of having src/dist directories.
nod_’s picture

Issue summary: View changes

For the record I'm all for writing "modern" js and compile down to an arbitrary version. It's just that we finally introduced it, and left it alone. Things around didn't go the way I though/hoped, and now it's hurting more than it's helping. In any case I reworked the summary and opened #3179415: Improve contribution workflow, build, and management of JS files to discuss the topic and keep this one as a meta.

nod_’s picture

Issue summary: View changes
nod_’s picture

Issue summary: View changes
pasqualle’s picture

I think the Symfony UX initiative is relevant here:
https://symfony.com/blog/new-in-symfony-the-ux-initiative-a-new-javascri...

Also the 2 free videos from SymfonyWorld 2020
https://live.symfony.com/account/replay/videos/2020-world

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

quietone’s picture

Status: Active » Postponed (maintainer needs more info)

There are 3 proposals in the issue summary
1. Adding committers
Core governance allows committers can be added at any time, once they are identified. Currently, the JavaScript APIs are the responsibility of the Frontend Framework Managers. My question here is this point about finding people or about changing governance?

2. Have a much higher commit frequency (help work going on smaller JS issues to get more people or remotivate existing contributors).
This seems to be about issue scope. This needs more detail so that reviewers and maintainers understand how to scope JavaScript issues?

3. Reduce expectations for JS patches on the short term (to increase commit rate and have more people involved) and raise expectations progressively (we'll need to define and advertise the different milestones)
Similar to #2 This needs more detail.

Changing to PMNMI for answers to these questions.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.