Improve usability, accessibility, and scalability of long select lists

This looks sharp. Works as expected without a mouse which is great.

I've only looked at this with VoiceOver, but there are some issues here. I've asked for a blind user to review this and provide better feedback. As a sighted person, my use of a screen reader is pretty basic.

When I first go to the screen I'm simply informed that there is an English pop up button:

I go down and I get all 94 results:

I enter in a key (not that there is any indication to me that this would have any impact - it isn't very discoverable) and I get a summary of results. It would be nice if it were able to read that shortened list of results.

When one option has been selected that isn't english and you go back to edit the dropdown:

When you've selected one result.

I added some elements to my taxonomy, then went http://sb8acaa3c676552c.s3.simplytest.me/node/add/article

I've attached the screenshots below, but not fleshed out the descriptions. The file names kinda illustrate the problems.

It works but also needs work.

Oeeh, nice. Looking forward to the A11y review

I've uploaded so many images I can't find the patch anymore.

I am going to have to write up what it should be and then see what we can do to make it that way. We do need to have some action points.

Accessibility is probably going to be the major hurdle for this patch (and I'm not sure whether it's for Select2 or the implementation). I do hope we can git it in though as it's a nice feature.

I'm going to write out what should happen for accessibility:

A) On Choose a language
1) Rather than saying "English pop up button" it should say "Language selection pop up. English selected."
2) On entering the pop up, the user should be told "94 results are available, use up and down arrow keys to navigate" it should say "To choose a language, enter the first few characters. 94 results are currently available."
3) When a key is pressed the user should be told that the list changed. Rather than "6 results are available, use up and down arrow keys to navigate." -- it would be nice if it was more consistent with autocomplete and would read the list.
4) If there is only one result, it should read the result. "One result is available, press enter to select it." is a pretty useless description. In my example it should read something like "Bosanski selected, press enter to save and continue."

I can't see the source code to see how viable that actually is.

B) With the taxonomy term, it is important to provide more context as this will be very confusing.

1) When you first tab to the form you're told "Tags edit text list 1 item" but really should be told that this is an empty form and that they should start typing to either get a list of previously selected tags or to enter a new tag.

2) On hitting f, I'm told that "4 results are available, use up and down arrow keys to navigate" it would be nice if it were consistent with autocomplete right now and list the selection.

3) If there's already a tag selected, the count is off. "4 results are available" seems wrong given there are 3 in the list:

4) Frog is pronounced when selected:

5) The tags have a nice description field which is available but it isn't read to the screen reader as it's done as a title to the input form. It says "Please enter 1 or more character"

Hi. When D8 go to beta release, I will port Select2 module to D8 and we can use it for tags (and other taxonomy ref) fields, autocomplete fields and other.

get the patch back.

I had to search for a cached version of the page to get the patch:
https://www.drupal.org/files/issues/core-js-select2.patch

The summary says to use it for taxonomy, why don't we use it for multiple selects as well?

I think @nod_ was primarily developing this as a use case to see what problems we'd run into.

There are a few accessibility problems with it at this point. I've tried to articulate how they should be improved.

attiks, at this week I will publish dev release of Select2 module for D8, with this module you can use it for all select form elements, autocomplete fields and define select2 options for custom elements by Forms API.

#13 Cool

Alright, so this greatly impacts i18n - who made a major follow up.

Thanks oushen, any chance you can help out this patch? I'd rather have that in core in 8.1.0 if we can instead of contrib. That way we'd make sure the integration is accessible and that many more other contrib modules can use it (like rules maybe).

+1 to that!

@mgifford @nod:

If "choose language" is not being read it's because the "aria labeledby" attribute is missing I think.

The summary says to use it for taxonomy, why don't we use it for multiple selects as well?

Amen to that

@mgifford

The "please enter one character" issue also looks like aria labeledby is missing

We can add aria-labelledby to the 'Choose language' title in core/lib/Drupal/Core/Installer/Form/SelectLanguageForm.php

I think the "please enter one character" is a vendor issue though that's part of Select2. Not that we can't change that, but it is a change for stuff up-stream.

With any dynamic tool like Select2 WAI-ARIA is going to need to be added to add the required semantics.

Drupal.announce() is also worth looking at https://www.drupal.org/node/2014521

With any dynamic tool like Select2 WAI-ARIA is going to need to be added to add the required semantics.

Yes, but having said that, from my research the library already has more aria roles natively than any other and if we make that more robust it would be more responsible for us to add them upstream submitting pull requests to Select2 on github. That way others can benefit as well.

I'm totally onboard with providing patches to Select2 and pushing them upstream. I think they'd be open and everyone would benefit.

I'm just not a JS person so won't be leading the way in developing the patch.

Can you please update the issue summary so it includes the reasons for adding this, which problems it solves, etc?

Also, this looks like Chosen. Do they really do similar things and if so, how do they differ?

Took a quick look for comparisons of Select2 vs Chosen.

They do pretty similar things from what I can tell.

It is worthwhile to look at Kevin his evaluation at https://docs.google.com/spreadsheets/d/1vifOGL_4wgOl6sy2Os59k24saIHvwFFQ...

This would be really useful for new UI for related help topics #2351991-55: Add a config entity for a configurable, topic-based help system

wrong issue

Comment #6: 3,

I sent a patch to upstream and it's merged now. https://github.com/ivaynberg/select2/pull/2726

This would also finally solve the many huge problems (UX-wise and otherwise) with jQuery UI Autocomplete explained at #675446-271: Use jQuery UI Autocomplete. Great work here :)

@Nod

Given the upstream changes to Select2 for a11y are we unblocked on rolling a patch?

Is this going to have to get bumped to 8.1.x.?

Is there any chance that a patch will get into 8.0.x for this?

I put the templates in the summary, but there are details I'm not sure of.

What is the motivation here?
Why is this a feature request?
Could this qualify for beta under "prioritized change"?

I think it's very important to get this into 8.0.0 for multiple reasons:

• the author seems to be quite interested in accessibility improvements, as shown by #29 and also from the 4.0 roadmap
• it greatly improves usability for long selects lists and multiple-selects
  #1271622-29: Seek a better autocomplete replacement for core (jQuery TokenInput / Select2 / Typeahead.js by Twitter)
  #1271622-44: Seek a better autocomplete replacement for core (jQuery TokenInput / Select2 / Typeahead.js by Twitter)
• my personal favorite and the reason I just got involved in this issue is that it finally provides Entity reference with a proper solution for the long-standing problem with core's autocomplete (and the current jQuery UI implementation for that matter) that it cannot make a distinction between what is displayed to the user vs. what gets sent to the backend (see #1411304: Autocomplete hide id options for use with unique labels and #1802916: Add option to autocomplete widget to hide entity ID for real-world user frustration with the current "entity label (entity_id)" solution provided by Entity reference)

Also note that Entity reference is very likely to replace the taxonomy term reference field in D8 so the last point outlined above is going to get in everyone's face soon enough.

I started with @nod_'s patch referenced in #10 but I'm not sure that creating one-off libraries for every select element that wants to use Select2 is a good long term strategy, and we should rather aim for a much better DX by integrating it right into Form API's select element via an optional #select2 property.

Haven't touched the autocomplete bits yet but this patch shows how I envision the Form API integration. I also added the #select2 option in two more places, the timezone selectors, just to demonstrate how easy it should be for everyone to use it.

+++ b/core/misc/drupal.js
@@ -86,12 +86,12 @@ if (window.jQuery) {
+        //try {
...
+        //}
+        //catch (e) {
+          //errors.push({ behavior: i, error: e });
+        //}

Comment out code.

+++ b/core/lib/Drupal/Core/Render/Element/FormElement.php
@@ -128,4 +129,43 @@ public static function processAutocomplete(&$element, FormStateInterface $form_s
+  public static function processSelect2(&$element, FormStateInterface $form_state, &$complete_form) {

Why on \Drupal\Core\Render\Element\FormElement and not on \Drupal\Core\Render\Element\Select?

AFAIK there is no other element that could use it?

Why on \Drupal\Core\Render\Element\FormElement and not on \Drupal\Core\Render\Element\Select?

AFAIK there is no other element that could use it?

Text elements that might have a different purpose than autocomplete can use it too, so why not give them the possibility? :)

Here's the complete integration with Drupal's autocomplete requirements, with proper markup escaping and all that.

The only thing left that still needs work is to provide support for our "term with, comma" syntax, which is not supported by Select2 at all. Every time this question comes up on github or other support forums, people just say to use "|" as a delimiter, but I don't think we have that luxury of choice in Drupal so we'll need to provide our own tokenizer function. I looked a bit at that code but I hope @nod_ can help out here since he's, well.. more familiar with JS :)

Fixed that last test. Let's see what @nod_ thinks about #41.

Can we have a core only patch *pretty please*?

It's in #41?

pardon my ignorance patch looks good to me.

Let's get the beta evaluation updated to be more than boilerplate? :) It would be really valuable to summarize in that table why this is major, what the improvements for accessibility and such are, and what the disruption would be within core, for contrib, etc. :)

The diffstat for the core-only patch in #41 is nice.

+++ b/core/modules/views/src/Tests/ViewsTaxonomyAutocompleteTest.php
@@ -75,8 +75,8 @@ public function testTaxonomyAutocomplete() {
-      'value' => $label,
-      'label' => String::checkPlain($label),
+      'id' => $label,
+      'text' => String::checkPlain($label),

E.g. this change in the array structure is a (reasonably small) disruption for contrib. Etc.

Improved the entire IS a bit, I hope it's more clear now :)

#48 does this mean that the keys need to be changed depending on the fact that you use select2?

Ex: if I remove the "'#select2' => TRUE" from a select, will this break the code?

@attiks, the keys have to be changed everywhere. But that's only about the autocomplete API, not about using Select2 in regular select elements.

I changed them because we also did it when we switched to jQuery UI autocomplete in #675446: Use jQuery UI Autocomplete, so I thought we want to follow the Select2 defaults. I don't have any strong opinion about it though :)

I thought about it some more and I agree that the value => label combination makes more sense than id => text, so let's drop the slight API change and just keep the huge UX improvement :)

I WANT this patch so I tested it but found some problems.

- tags with spaces don't get recognised

- When editing the same node nothing gets recognised

- Why is timezone a select 2 field and country not?

- Can there be multiple authors? That's confusing. After the first author the autocomplete doesn't work anyway.

@aspilicious, thanks for the review!

- tags with spaces don't get recognised

That's already mentioned in #41 :)

- Why is timezone a select 2 field and country not?

Yeah, the country selector is long enough to need the Select2 widget. Added.

- Can there be multiple authors? That's confusing. After the first author the autocomplete doesn't work anyway.

Fixed.

@nod_, I was looking into the tokenizer thing as well and I got it mostly working, here's my WIP diff.

Quick update for the review in #57:

1. Dependencies are not right, the misc file should be in drupal.autocomplete. Not here since it'd make select2 depend on core/drupal and core/drupalSettings.

   Not sure that's correct, that code is just registering Select2 into our library system, and it's only dependency is on jQuery. Since we've now integrated Select2 into Form API, this is not tied to autocomplete anymore.

2. Fixed.

3. We might want to expose that in drupalSettings for people to be able to change it globally, not just with data attributes.

   I don't know what's the "right way" to do this. Do we usually expose stuff in drupalSettings from inside a js file?

4. Fixed as discussed in IRC.
5. Oops :)

Finally understood what @nod_ meant with the separate library. Fixed that and incorporated the tokenizer from #60 into the main patch.

Tag autocompletion still has issues. I made two tags:

"Some tag", "a new tag"

When editing the node you get 1 merged tag: "some tag, a new tag". The tag list in the backend looks ok, so it's a js thingie.

/me giggles at @nod_ :D

The fix from #63 fixes the problem mentioned by @aspilicious, but there's still one small issue: you can never remove a tag which contains a comma :) It is removed visually from the input element but when you save the node, it's still there, sometimes even duplicated.

This looks like a great direction.

Looking at the library and the patch, what approach should be used to bring consistency to non-Select2 select lists and Select2-enabled select lists? @aspilicious's image in #55 shows the differences pretty clearly. I'm guessing the default styling for Select2 would be left in place (considering there are no "skins") and the theme would take on the responsibility of styling them individually (as Seven theme styles select lists already).

I'm not sure that the Seven theme styles select lists already. At least on FIrefox/Chrome on Kubuntu, the select lists look like the OS ones, and I don't see any select-specific code in firebug.

My proposal for styling is to use the Seven style guide, so even if Select2-enabled select elements won't look like the regular ones, at least they'll be consistent with the rest of the theme. We already have a dedicated issue for this: #2318665: Implement the dropdown menu component styling into the autocomplete widget, but I'm not opposed to doing that work here.

Here's a reroll for current HEAD + the small fix from #63 (Drupal only patch for review is in #61).

@nod_, do you have some time to look into that small (last) issue with the new tokenizer found in #64?

I'm not sure that the Seven theme styles select lists already.

It does, but only on Webkit because cross browser support for styling the native
element is poor. #2207391: Style select elements in Webkit only

I have to say this is looking very good. I recommend we stick with the default library styling, and work on the Seven styling in #2318665: Implement the dropdown menu component styling into the autocomplete widget

Yes, I saw the new version of the library and that it contains quite a few things that break our current integration, but that doesn't mean we should drop this issue, I hope?

the data sent to the server will be different, unless we serialize the values from select2 in the input on submit.

This doesn't look like an unsolvable problem :)

hmm, I don't understand. How would you add a new value that doesn't exist as an option in the select dropdown? Or is that now outside of the use case of Select2?

@LewisNyman, a select dropdown is needed just as the initial element, adding new values / tagging works just like before: https://select2.github.io/examples.html#tags

Indeed, looks like we can work with Select2 version 4 just fine; we'll just have to change #type => input to #type => select.

I think a select element feels like a much better JS fallback than a blank text input.

#74: except that you can type in a blank input[type=text], but the <select> Select2 uses would be empty in our case, since free tagging in Drupal allows for thousands of terms to be created, not all of which can be listed. Disable JS and look at https://select2.github.io/examples.html#data-ajax for a more accurate fallback UX.
And that is actually a great point: the no-JS experience. Do we even still care about that? E.g. dropbuttons are also completely broken without JS.

I just spent a day trying to upgrade the D7 Select2 module to support the new 4.0.0 version of the library. It *really* doesn't like working with input fields. Dropping support for (hidden) inputs is actually listed as a feature of the new version: https://select2.github.io/announcements-4.0.html#hidden-input

In trying to use it on input textfields, Select2 simply has no support whatsoever now. It actually tries to embed <option> tags inside of the textfield <input> element. The only hope we'd have is doing as Wim suggested and replace the textfields with select elements.

I don't know if this is going to be a suitable replacement for autocomplete text fields. A select list isn't going to be the right element when dealing with large lists, which is pretty much any situation we have autocomplete currently: author name, term tags, the search on api.drupal.org, etc.

The whole situation is disappointing. The "best" library for autocomplete basically doesn't want to deal with autocomplete and wants to focus exclusively on replacing select list elements. I suppose it shouldn't be too surprising, considering their past claims that autocomplete/type-ahead is outside the scope of their goals: https://github.com/select2/select2/issues/17.

We have a site with more than 6000 tags and counting (since september 2014). We can't use select fields for those in combination with select2 because it's nearly impossible to render.
Making tags a select field would introduce severe performance issues.

As I do see the advantage for 95% of the use cases I would like to have a fallback to the current behaviour. But that would be hard because one widget is a textfield and the other a select widget :s

Damn...

Can't we fund a JS genius to make a drupalSelect fork of select2? XD

@#77,

You won't tagging 6k tags in ONE node ??

Technically, we can rendering select list in client side (converting from input).

see patch in #66, it still making a lot of changes to JS/PHP code. It's a sign that D8 still having bad Autocomplete API architecture in CORE. I think we need to shape it better API to adopt different JS libs for future developers (without CORE JS & PHP patching).

#78 We have similar (and bigger) vocabularies as @aspilicious mentioned in #77, outputting a select list is not going to be an option. Most of the entities are tagged with multiple terms.

Dream scenario:

Have a widget that renders the right HTML depending on the amount of data and attaches the right libraries.

For example (boundaries to be defined in config):
10 or less options and without tagging: output select, no select2
200 or less options and without tagging: output select, use select 2
200 or less options and with tagging: output text, use select 2
201 or more options and without tagging: output empty select, use select 2 + ajax
201 or more options and with tagging: output input, use select 2 + ajax

Really nice to have: Add support for hierarchical vocabularies as well, so site user can select parent term as well and they are rendered as nested.

Whoa, hold on there.. isn't #72 clear enough in this regard? :)

#73 is wrong, there's no change needed on the PHP side (at least for the element declaration), we just need to convert the text input into a select before Select2 is initialized on that element.

#80 I'm probably missing something, but what is the policy for non js browsers, 'cos if we output a select for free tagging it will not work without javascript. Unless we are converting the input to a select using javascript?

Unless we are converting the input to a select using javascript?

Bingo! :) Just as @nod_ said in #69 when the discussion about the new version of Select2 started.

#82 I've updated the IS, but by reading the comments the API change is no longer needed?

I'm a JS noob, so I can't say anything useful, but I think when it's done we need to manual test this with 1000+ tags to see if it has any effect. Just to be sure. I'm always ready for doing that when needed.

It will gain me lots of credits with the UX designers at work ;)

Sorry, yeah I kicked off the discussion in the wrong direction there. We can convert the textfield to a select *with JS*, then run Select2 on the new select list field we just created. However, that means we're going to have to adopt a lot of custom code to deal with converting a textfield to a select list. You'd need to account for single/multiple values, parsing the input field, converting it to a select list with the entered options selected. Then run Select2 on the resulting new select field and load the results via AJAX like we had been doing before. Before the values are submitted to the server, you have to convert these values *back* into a input so it would all come in as a single value in POST. A select list will send multiple values as an array.

Rather than attempt to revert the select back to input, we'd probably find it easier to maintain the original input on the page, and update it as the select list is updated.

So in summary, to use Select2:

- PHP outputs a textfield onto the page.
- Hide the original textfield and create a select list "copy" of the field with JS
- Apply Select2 to the select list, load results via AJAX
- On value change, update the original textfield.
- On POST, the server gets the value from the original field. The select element we could leave with no name attribute, so it would be nothing but a client-side widget for manipulating the hidden field.

So basically we'd re-implement the functionality that was removed from Select2 to support hidden fields.

I don't know... it's possible but doesn't seem right.

#76/#77: did you look at how https://select2.github.io/examples.html#data-ajax (that specific linked example) works? It works with an empty <select>, for which <option>s are generated by Select2.

#86: that sounds very doable? But I agree that it's very unfortunate that the input[type=text]-based autocomplete use case is no longer directly supported by Select2. I really wonder why the author did that? After all, what Select2 presents UI-wise resembles a input[type=text] much more cosely than a <select>, so even in that regards it's a puzzling change. The removal of a input[type=hidden] dependency makes sense to me, the removal of input[type=text] not so much.
Does anybody of us know the Select2 author? Or has any of us followed its development closely? If so, then perhaps we can start a conversation with the author? One of us should reach out to the author IMO; Drupal 8 would be one of the most prominent users of Select2, I imagine, so perhaps he'd be willing to add this back for us. Or perhaps he'll manage to convince us of his reasoning :)

I'm the maintainer of Select2, and I'm interested in working together to fix the current lack-of-text-input issue.

I removed support for hidden inputs (<input type="hidden">) because there was no non-js fallback. Non-js users would be presented with absolutely nothing, and most likely any validation would be failing (pretty silently) because of that. All in all, it wouldn't be a great user experience, and it was something I wanted to avoid. Now, I didn't realize that people were using a text input as the fallback until about a week ago, and because of that much of the code is expecting a <select>, which is great because we don't have to special case that much.

The problem with the old implementation largely revolved around initSelection, a badly named function that handled the job of converting the value attribute to a set of data objects. I'm interested in working together to figure out what would need to be done to have Select2 function as an autocomplete for Drupal, but my knowledge of how the Drupal internals work is very limited.

We are tracking this issue at https://github.com/select2/select2/issues/3022 as well.

That's awesome news! Thank you!

Basically, all there is to know is this:

1. Drupal has supported "free tagging" (also known as folksonomies) for … 10 years or so. You can type any tag you want, and type a comma to indicate a next one.
2. To help users find/select existing tags, there is an autocomplete, which takes the space-trimmed string after the last comma — if any — and asks the server for autocomplete matches. (It's never been quite as awesome as Select 2 though :)) i.e. foo, bar, baz -> "baz" is being autocompleted.
3. To type tags containing commas, we wrap them in double quotes. i.e. Some Person, "Leers, Wim", Another Person has 3 tags: "Some Person", "Leers, Wim" and "Another Person".
   (This behavior would probably need to be implemented on the Drupal side, which is exactly what @amateescu did in #66, until @nod_ remarked in #69 that Select2 version 4 was about to be released, and this issue kind of got stuck on what to do now that Select2 no longer natively supports input[type=text]. In other words: you probably wouldn't even have to care, as long as we can still make it work just like we did for Select2 version 3.)
4. Most importantly to you: what is described above, including the examples, are exactly what is sent to the server (commas, quotes, spaces and all). Basically, our UI has always been input[type=text] with some UI additions for autocompletion that helped the user get data in there in input[type=]text] faster and more easily — hence degrading "gracefully" to a very unhelpful, autocompleteless, mindblowingly simple UI: the plain input[type=text].

Also maintainer of Search Autocomplete interested in helping if needed : #2442699: UX improvment on autocompletion

Now there's a release candidate for Select2 4.0, do you think now is a good time to create a new patch for this issue?

Indeed, more than a month ago, the solution on the Select2 side was merged: https://github.com/select2/select2/issues/3022#event-243644862, https://github.com/select2/select2/pull/3051.

Select2 4.0.0 was released about 2 week ago. Looks like this issue back on. ;)

Tagging frontend for sprint.

Should we bump this to 8.1?

@mgifford

Given other priorities, perhaps. Although given the fact that it is unquestionably an a11y improvement, maybe not.

@tkoleary I don't see the accessibility improvement. At least not when looking at the examples here:
https://select2.github.io/examples.html

It is a simple tool, but there are a lot of basic problems listed here:
http://wave.webaim.org/report#/https://select2.github.io/examples.html

I don't know how many are just part of the example code though.

I haven't had time to really look at it though. I do know that jQuery UI's autocomplete has had quite a lot of accessibility testing.

Can you point me to a comparison of the two for accessibility? Or even the documentation on the a11y work that's gone into Select2?

Last info I looked at was this issue https://github.com/select2/select2/issues/2499 that both you and I commented on which is now closed fixed. Since that time new a11y issues have been opened in 4.0, but (based on my own unscientific testing) 2499 puts Select2 well ahead of jquery autocomplete.

Here are the open a11y issues in select2 4.0

https://github.com/select2/select2/pull/2875 this one was opened after 2499 and was merged in on 12/09/2014
https://github.com/select2/select2/issues/3238 closed fixed
https://github.com/select2/select2/issues/3144 this one is still open as an enhancement but no longer marked a bug

I could not find other issues. We have an individual working with us here in Boston who may be able to test 4.0.

I definitely think there has been progress with Select2 on accessibility, but I haven't had time to test it and it isn't clear to me that others have either. One of the last comments in #2499 is "Unfortunately I have no idea how to thoroughly test it as I only have limited knowledge of how Orca works, but perhaps someone else can help with testing."

That doesn't give me a lot of confidence.

I don't know how testing has been done or by whom. Select2 seems to have quite a few powerful options. I don't know if any of those have been tested.

There's some good addition of ARIA, but I still don't see how it's significantly better than jQuery UI for accessibility.

Some initial testing by Tyler Littlefield here in OCTO shows that in fact there is still a good bit of work to do. I plan on creating some new issues in the Select 2 queue on github.

The WP folks did a nice review of Select2 because they were considering it for WP Core - https://make.wordpress.org/accessibility/2015/09/07/accessibility-userte...

It would be great if we could work with them a bit more on addressing these issues.

Also worth noting https://github.com/select2/select2/issues/3744

Rerolling.

Apparently there's a competitor to select2 that pays homage to select2, but tries to be better/simpler: https://github.com/arendjr/selectivity

Selectize.js also claims to be have a few advantages to Chosen, Select2 and Tags Input. and http://selectize.github.io/selectize.js/ and could be worth considering.

@wim leers

Apparently there's a competitor to select2 that pays homage to select2, but tries to be better/simpler: https://github.com/arendjr/selectivity

It does look simpler. Also bad on A11y but Select2 has not made much progress either, same for selectize. Just tested all three with screen reader.

attempt to reroll

@vprocessor

Looks awesome! Despite failing tests it seems to work fine when I spin it up on simplytest.me.

One thing I noticed, the descriptive text would need to change (or just go away) in several places eg.

Enter a comma-separated list. For example: Amsterdam, Mexico City, "Cleveland, Ohio"

But that's a good problem to have.

Updated all files with newer version of D8. Let's see if test passes.

Rerolled, was just a simple conflict on core/misc/autocomplete.js

It looks like the re-roll was botched at some point, and a missing file (or several!) was causing the test failures. Here's a new re-roll against 8.3.x, using Select2 4.0.3.

I apologize for the colossal patch size; the overwhelming majority of it is the Select2 library.

EDIT: Now that core has functional JavaScript testing, it seems like we might want a functional test of this widget against, say, a taxonomy term reference field. Do folks agree?

Note that #2788209: Entity reference field "Autocomplete matching: Starts with" option does not work added a (small) javascript test for the autocomplete widget, so we could expand it here if needed.

But IIRC, this issue was blocked on the fact that Select2 is not as accessible as we need for core, see comments from #103 to #107.

Looks like the WordPress people were interested in perhaps devoting some brainpower to fixing these issues; I can't tell if anything came of that, but maybe we could do the same? I really like this library and would quite like to see it in core.

+++ b/core/assets/vendor/select2/css/select2.css
@@ -0,0 +1,484 @@
+    .select2-container .select2-selection--single .select2-selection__rendered {

Guessing this was copied from select 2 css and altered and that's where the indented rules come from. We should follow drupal CSS coding style here with no indents.

At first glance I thought it was Sass. :)

All good work here.

I think we need some final calls:
A1. [Frontend] Final analysis of the Performance. Select2 isn't a small kid thought.

If we can't pass A1. It's time focus on contributed and pick another one instead of dead loop :)

A2. [Accessibility] Final analysis of select2 itself (without Drupal)

A3. Basic usage. Testing in cross browsers (Checking the supports table in Select2 is not enough. In Drupal, we have very diff requirements.)

B. [JavaScript] Possible way to improve the Accessibility problem in Select2 Upstream & Drupal ourselves.

With my last patch to Select2 upstream ( https://www.drupal.org/node/2346973#comment-9255223 ), I think there's limitation to patch everything in Select2 upstream. Select2 also has itself design principal.

I could resolve some dependency issues. Looks like there is no Select2 object in version 4.

+1 This will be very very nice if we do have it in Drupal core and out of the box.

I think this needs escalation to product, with a clear answer on 1.

Assigning to nod_ to provide some guidance.

I am looking forward to start using this feature, is there any update on the status of this issue?

Most conflicts are easy enough to fix, however core/misc/autocomplete.js has been changed by #2823589: Improve IME handling on Autocomplete so I'm stepping away from this reroll in fear of messing things up :)

Another choice:
https://semantic-ui.com/modules/dropdown.html

I suspect a lot of conflicts are not showing up because the patch hasn't been updated to allow for the fact that a lot of Entity Reference code has moved out of that module.

More generally:
Everyone wants this. But it's not showing any signs of going anywhere.
There's 4 tough hurdles to climb:

1. Get Select2 working as a Drupal element and entity reference widget
2. Get it meeting Drupal accessibility etc. standards
3. Switch it in place of jquery autocomplete in the default entity reference widgets
4. Deal with the consequences of adopting it in Drupal's admin UIs

What chance of us achieving all this in one 15k-lines patch in 8.x?
How about we split this into 3 steps:

1. Get select2 working in contrib, getting it feature-complete as an optional element and widget
2. Move it into core as an experimental module, working on accessibility etc.
3. Create an issue to deprecate the current jquery autocomplete entity reference widgets in favour of the select 2 ones
4. Create an issue to enable it globally in the admin UI replacing jquery autocomplete and deal with whatever unforeseen consequences that has

#140:
> How about we split this into 3 steps: ... (4 steps)

YES sounds very reasonable.

Re rolled. Not find all the changes. Not able to form interdiff.

I've just come across this while looking for a D8 version of the Select2 module.
I'm working on and already created a basic version of the module which creates a Select2 widget and a FAPI element ( which is what I needed initially.)

Anyone else working on this? Can we combine efforts and get at least a dev release ready asap?

See https://github.com/ueberbit/drupal-select2

Has anyone checked out selectwoo?
https://github.com/woocommerce/selectwoo

There's also someone from waggl.com willing to fund some accessibility work on select2:
https://github.com/select2/select2/issues/3744#issuecomment-314813909

@mgifford,

Good news! that is a @automattic supported project. and other supports from waggl.com

@drewB I'd do it with PRs if I thought they would get merged in, but I doubt they will.

I have same concern as @claudiulodro in #131. I believe we need a smiliar fork or custom extension as woocommerce/selectwoo in druapl also.

1. and recently, when I worked on a custom project, I found the responsive doesn't work very well in select2 in many cases. e.g. rendered as a close Modal markup, Tabs. When it's shown, `width` didn't set correctly. ( https://github.com/select2/select2/issues/3278 )

2. down arrow on focus doesn't work. ( https://github.com/select2/select2/issues/4514 )

Thanks @droplet I do hope we can get this addressed in the 8.x cycle. Seems like there are a lot of folks who want a solution but there hasn't been someone yet to organize it. Maybe folks will rally behind selectwoo.

Thanks for posting those other accessibility bugs in select2.

and there's a selectWoo issue:
https://github.com/woocommerce/woocommerce/issues/16003

jfc.csantos seems to have created https://github.com/Bynder/d8-select2

Hi all!

I asked to contribute my version to the official select2 project but the maintainer didn't wan't to do it, so I'll just go ahead and create this one...it's holding my company back on releasing a few necessary features.

As soon as we can get my changes to the select2 project I'm more than happy to remove this one.

Cheers

@jfs.csantos,

AFAIK, after some point of 2016 Mid ~ 2017, you able to create "Yet another Select 2" in d.org freely.

With current movement, I think it's a bit hard to get Select2 into CORE.

@jfs.csantos (https://github.com/Bynder/d8-select2) & @webflo (https://github.com/ueberbit/drupal-select2)

I think it would make a lot of sense to release one of your module on drupal.org as project to
- make it easier to find it
- provide it via drupal.org composer
- combine into one project and use benefits from both, because drupal-select2 supports views filters for example while d8-select2 supports remote data?

The best would be to release it as D8 version of one of these projects:
https://www.drupal.org/project/select2
or
https://www.drupal.org/project/select2widget
or
http://hemantnegi.github.io/jquery.sumoselect/sumoselect_demo.html
or
https://www.drupal.org/project/selectize
instead of creating a further one??

At the end a Drupal core integration and replacement of the current solution would be perfect, but I think that will take some time...

Hi there @Anybody! Thanks for the input, but I already tried to move in that direction...asked the maintainer of Select2 to upload an initial version of the D8 module, he said he didn't want to because in would be "releasing an official version in a few months".

Also considered releasing the module in Packagist or in drupal.org as a new project, but in the end they were temporary and not very good solutions.

So, in the end, because of my company's need for this to be released, we ended up just adding it as a submodule of the Bynder module. When/if we can actually get some cooperation from the Select2 project maintainer I will gladly contribute with this code.

If you guys still find the module useful, I can still put it up on Packagist.

Cheers

@all: This info and discussion belongs here: #2695535: [select2] Select 2

This is also the community place to discuss and decide the release policy.

OffTopic: following this issue and got alert for new comment at dashboard as "1 new" but how come #154 changed it's comment time? Its not a big issue but would get messy afterwards.

attaching screenshot for ref..

@Anybody
I'm really sorry... but I've just checked SumoSelect on github - https://github.com/HemantNegi/jquery.sumoselect
it contains pretty strange code inside (bad maintainability by my opinion), not sure if it will be acceptable by community
please check - https://github.com/HemantNegi/jquery.sumoselect/blob/master/jquery.sumos...

btw, why not Selectr?
https://github.com/Mobius1/Selectr

Thank you @alphawebgroup https://github.com/Mobius1/Selectr should also be considered. I also just had a look at sumoselect and share your oppinon. Did you have a a look at Selectrs code already?

The problem we see with Selectr is that it changes its height (1 line becomes multiline if many items are selected). This man be a problem for views filter layouts or on mobile devices and other cases. jquery.sumoselect offers the possibility to use "x selected".

See my quote here: https://www.drupal.org/project/contrib_tracker/issues/2695535#comment-12...

Perhaps https://github.com/Mobius1/Selectr (discussion for responsive / touch UI: https://github.com/Mobius1/Selectr/issues/14) might be a better alternative for Drupal (Core) but it also doesn't behave well in responsiveness. The box simply grows line by line... best would be a selective combination from Select2 / selectr and jquery_ui_multiselect_widget by usecase... Chosen is even older...

I created a feature request in Selectr for that but I don't think that will happen: https://github.com/Mobius1/Selectr/issues/56

The problem we see with Selectr is that it changes its height (1 line becomes multiline if many items are selected). This man be a problem for views filter layouts or on mobile devices and other cases. jquery.sumoselect offers the possibility to use "x selected".

I agree. This will make consistant (inline-)form styling nearly impossible. Those "Pill Style"-selects are just usefull for simple vertical forms.

I needed this so contributed the module here if anyone else need it for D8: Select2 Boxes.

If there’s still no viable, actively maintained jQuery widget, it seems we should really team up with WordPress and other consumers to maintain our own fork either of Select2/SelectWoo or another alternative.

I might be wrong, but I guess it’s not going to be more workload than putting more and more efforts into our own, outdated and limited autocomplete widget, especially taking into account all the effort site builders put into introducing their own custom implementations.

Either way it doesn’t seem realistic this would ever be implemented into Core within the D8 life cycle. May I push it to D9, while different routes are weighed in contrib? I’d love to see a major improvement, but fail to see it materializing... :/

I think this could easily be made available in core in D8, even if the default implementation didn't change.

In #140 I proposed a 4 stage plan:

1. Get select2 working in contrib, getting it feature-complete as an optional element and widget
2. Move it into core as an experimental module, working on accessibility etc.
3. Create an issue to deprecate the current jquery autocomplete entity reference widgets in favour of the select 2 ones
4. Create an issue to enable it globally in the admin UI replacing jquery autocomplete and deal with whatever unforeseen consequences that has

Now point 1 has been done, thanks to @chr.fritsch.

Therefore I've opened #2987622: Move Select2 to core as an experimental module in the ideas queue. I'm also suggesting we close this issue despite the heroic work here, because it's no longer Drupal practice to make large changes in core using single monolithic issues. If this going to happen, I imagine it will likely be via the experimental modules route.

This is not outdated, there are patches & valuable discussions.
I see no reason in experimental module while contrib has only first release & that is just a frontend library

Choices.js is a new candidate which is now used by Webform: https://www.drupal.org/node/3046239

As the other libraries didn't match the requirements yet, this could be interesting.

The problem we see with Selectr is that it changes its height (1 line becomes multiline if many items are selected). This man be a problem for views filter layouts or on mobile devices and other cases. jquery.sumoselect offers the possibility to use "x selected".

I agree. This will make consistant (inline-)form styling nearly impossible. Those "Pill Style"-selects are just usefull for simple vertical forms.

The markup of choices.js is pretty good, should be easy to alter the style for horizontal forms by removing the pill style and do something like:

.choices__list--multiple{
white-space:nowrap;
overflow:hidden;
text-overflow: ellipse;
}

Maybe the author of Choices.js is willing to add this as a clean option.

So +1

Based on some quick research, this library seems like a pretty accessible option: https://github.com/alphagov/accessible-autocomplete. It's a React library, but we could use it as a comparison.

Thank you @lauriii for pointing out this very important reference. I've updated the issue summary to list some of the acceptance criteria and the mentioned libraries to evaluate. I'm unsure if we should remove the initial text from the issue summary, which only concerned select2 as option, but isn't really a good choice in several points. Or should we close this as duplicate and create a cleaned up issue based on this discussion?

I went ahead and prototyped how this might work with choicesjs to see what kind of unexpected surprises might show up.

The biggest of the surprises is that the best way to implement asynchronous autocomplete is to use choicesjs's 'search' event, and that event is only triggered on select elements. To get a feel for choicesjs, I added JS that converts all of the autocomplete <input> elements to <select>. This got things far enough along to make it possible to evaluate the use of choicesjs on entity reference fields. However, since the form element is different, it's not currently possible to save values (or populate existing ones).

I think this can be worked around if choices still seems like the best solution, whether they be in Drupal or the js library. This patch at makes it possible to evaluate how it may work in Drupal, which can help us gather evidence to determine if it's worth the implementation hurdles.

Two patches here, the first is an improvement to the choicesjs prototype from #181, the assets are now added as part of drupal.autocomplete instead of jquery.ui.autocomplete.

The other patch is a prototype of using the Accessible Autocomplete library. This probably isn't a good candidate, unfortunately. Among other things, it does not support multiple values, and it only works with string arrays (no ability to have discrete values and labels).

I think the prototype is still very useful to have as reference when evaluating other libraries for accessibility support. There's also a good writeup on some of the research and testing that went into developing the library, which can help inform the evaluation process for this issue.

I had a look at Selectr, wondering if the concerns in #161 were surmountable. I didn't get too far in the evaluation after discovering that (similar to choices in #181), it requires a select element, and would not be able to work on existing autocomplete <input> elements.

If its decided that changing the form elements is an acceptable refactoring, it may still be worth looking into Selectr and submitting a PR for the concerns in #161

Currently working on a prototype for a custom lib based on React Select. Based on preliminary testing done with @lauriii, React Select does not have any of the major accessibility/functionality limitations we've run into with other libraries. So far, any issues we've run into are ones that can be addressed via implementation and not a limitation of the library itself.

#171: nice, your comment is exactly 1 year and 1 day since the last one 😆

Based on the prototyping work done by @bnjmnm this seems to be the current state:

@bnjmnm I think it'd be handy if you expand this initial table (I can give you the source HTML so you don't need to recreate it) and add it to the issue summary, that would make it very clear why which JS libraries are eliminated or still being considered.

My gut reaction here is that I am surprised we're not continuing to look at Select2 first, because as this issue's older comments show, we have successfully worked with the author of that library (a long time ago!) to get accessibility improvements committed. Accessibility is notoriously hard to achieve for user experiences like this, and having years of testing and feedback is a huge advantage.

Select2 still seems to be actively maintained: the last commit 11 hours ago, and it happens to be an accessibility improvement: https://github.com/select2/select2/commit/e5131d0cc8dffd98ba36a68f3d027b...! Look at the complexity of that commit message! 🤯

React-select had its last commit >1.5 month ago and it was just updating a dependency: https://github.com/JedWatson/react-select/commit/c22a4b20aef2161ebaa3a03....
https://github.com/JedWatson/react-select has 1160 open issues and 1250 closed ones. https://github.com/select2/select2 has 40 open issues and 4230 closed ones. Notably, https://github.com/JedWatson/react-select/issues/1084 has been open for over three years. https://github.com/JedWatson/react-select/issues/3353 means not even the label for the form input is being read by screen readers.

That's a lot of signals indicating that Select2 is more mature, which is important for an accessibility-critical UI component.

@WimLeers very good point about Select2 and that's definitely next on the list. The thought was to start with standalone libraries, or at the very least see what can be done without jQuery. Full (not at all surprising) disclosure: react-select definitely does not achieve the "standalone" part of that approach at all, and has the hearty node_modules directly to prove it 🙂

Also worth noting: One of the steps required to get the react-select prototype working made it apparent that Choices and Selectr may still be viable options even though they require a <select> element - the library doesn't have to directly manipulate the entity reference text input - the input can be made hidden and updated via the library-provided select input (similar to how Media Library keeps track of selections)

For select2 we have a stable D8 module in contrib. See https://drupal.org/project/select2. Currently, around 2000 sites are using it.

There were also some discussions around this in #2987622: Move Select2 to core as an experimental module

Here are patches for three prototypes

• Select2
• Choices (there was a very simple way to get around the <select> only limitation, the same approach was used for Select2) (also want to point out the lib isn't minified in this prototype)
• React-select (working with the prototype has made me less enthusiastic about this one)(also want to point out the lib IS

minified and it's still pretty huge.

Will follow up with another comment with some of my findings for each one, but as that may not happen immediately I wanted to make the prototype patches available to others so they can evaluate.

In each case, i just worked with taxonomy fields in the Article content type, so that's going to be the best place to evaluate these patches.

Findings for Choices JS

(some of the evaluation criteria is not included/complete as the review stopped after identifying numerous difficult-to-address accessibility problems - this can always be revisited if our other options have a similar extent of accessibility issues)

Will also add these findings to the issue summary after allowing a bit of time for feedback/corrections.

I think we should at least consider switching to a select element if that improves everything else. At the very worst this could be added as new elements/widgets deprecating the current one but even that might not be necessary depending on how we do it. i.e. if our options include both updating our existing widget and a complete visual update we should look at both. We might decide it's not desirable in the end, but doesn't seem worth ruling out up front. The big thing though is retaining AJAX autocomplete for scalability.

#188: exciting to have 3 prototypes!

#189: Wow, love the deep analysis! Clearly you have a lot of accessibility knowledge 🤩👏 It sounds like some of those would be pretty easy to fix upstream, but others seem to perhaps be deeply embedded in the architecture.

Looking forward to an analysis similar to that of Choices in #189, but then for Select2.

#190: I was gonna say "but wouldn't that be too disruptive?" — then I realized that it's not #type => textfield with #autocomplete_route_name => …, but we're talking about #type => entity_autocomplete. So it seems we're fine there. Except … aren't we trying to remove the jQuery UI autocomplete altogether, which would require this to work for both cases?

"The big thing though is retaining AJAX autocomplete for scalability."
This is very important for a lot of projects.
I know that select2 implementations of a normal select can go wrong with +1000 elements.

Findings for Select2/SelectWoo

Selectwoo describes itself as "a fork of select2 with improved accessibility", which was started when Select2 didn't have much activity on github. Select2 activity has since resumed, so any accessibility issues I identified on one I tried with the other. Assume the problems happen on both unless stated otherwise.

Similar to the ChoicesJS review, theres probably more to learn but I stopped after once the accessibility problem list had several potential dealbreakers. Note that the maintainers of Select2 seem quite open to addressing accessibility issues so it shouldn't be ruled out based on these findings alone several accessibility issues have been addressed within the past few weeks and several more are in progress.