Composer dist problem - with chosen composer.json (false github auth problem)

+++ b/composer.json
@@ -18,12 +18,10 @@
-          "composer/installers": "~1.0"

Without this line, where does this file get placed? We need to require composer/installers for it to end up in the /libraries directory I think.

Apologies if i'm hijacking this issue, but i think we're all going for working installation with composer!

I'm installing Chosen with composer almost a month after chosen 2.0 stable was released:

composer require drupal/chosen

Using version ^2.0 for drupal/chosen 
  - Installing drupal/chosen (2.0.0-beta2)

Checked packagist.drupal-composer.org, and sure enough Chosen is there with a beta 3 release and stable release. Hmm.

Updated my composer.json to ask for "drupal/chosen": "2.0.0", no substitutions.

composer update

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for drupal/chosen 2.0.0 -> satisfiable by drupal/chosen[2.0.0].
    - drupal/chosen 2.0.0 requires harvesthq/chosen ~1.5 -> no matching package found.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://getcomposer.org/doc/04-schema.md#minimum-stability> for more details.

Trying to install harvesthq/chosen with composer require and no minimum stability confirms that the package does not exist at all.

The problem is that packagist.drupal-composer.org has been told chosen requires harvesthq/chosen and it can't find it, so it won't even download the latest releases of Chosen and run the composer.json, even if the github sourcing would work fine at this point.

Seems the easiest fix is to require drmonty/chosen which packagist.drupal-composer.org will be able to find, on packagist.

@jhedstrom: Not removing that when declaring the harvesthq/chosen repository in the root composer.json actually has a funny side-effect where Composer complains that "Package harvesthq/chosen-1.5.1.0 must have a source or dist specified" (except, it shouldn't look for that but rather 1.5.1).

It might work if you move installers to the module requirements instead of the repository requirements. Nonetheless I find it superfluous since anyone who wants to use the module's composer definition would normally have composer/installers as a general dependency (and it gets placed in the right place that way).

+1 For using a dependency which is available via Packagist to avoid the workaround.

So I changed the dependency to drmonty/chosen, and we can probably bump the version to 1.6.1 as well.

With a bit more version bumps.

Committed for testing purposes.

Edited: Actually it works fine now, but the library and module is not placed at the right place. Even though the installers is still defined in the json.

So currently the module ends up under the modules folder, even if there is a contrib folder inside the module folder, and the js ends up in the vendor.

@nagy.balint

I was checking the dev version of the module and I confirmed that the Chosen's composer dependencies get located under the vendor folder.
I've been taking a look and according to comment #29 in this discussion: https://www.drupal.org/node/2671754, seems that it may be possible that we are missing the installer paths in the composer.json.

They may be already in the global composer from drupal, but they need to be added again in the module's composer.json. So something like this (got form this gist,https://gist.github.com/mortenson/a5390d99013b5b8c0254081e89bb4d47 ):

"extra": {
        "installer-paths": {
            "libraries/{$name}": [
                "type:drupal-library"
            ],
            "modules/contrib/{$name}": [
                "type:drupal-module"
            ]
        }
    },

I couldn't try it, as this gets triggered on installing the chosen module, and I'm not sure if triggering the installation from the global composer will change something...

Either this, or an other option is that we should either ask the user to add the chosen module in their global composer.json, so it gets located in the proper path. So it would be adding the library directly in the global composer.json.

Hope this comment helps!

@sturgelose

Thank you for your comment!

I think I also came across this solution before, but my problem was that if we define the drupal-library and drupal-module types in the module, then they are hard wired in our configuration to go in that directory, even if for the site those are defined differently.

And the reason why I was confused is that in https://github.com/composer/installers these drupal-* types are defined, and we specify this as a dependency, then why it wont take these types from the installers module and use it like it was configured for the site.

Of course its possible that even hard wiring is much better than the current situation where it puts the module and library in a completely wrong place, so maybe until we figure this out, we should add these extra parameters.

@nagy.balint

Just tested right now. Seems that if you add the library dependency in the root composer.json, and works as expected: Chosen is in the /libraries folder! Then this runs pretty smooth and the module finds the library as expected.

Solution
I added the repository reference to chosen (copied from Chosen's composer.json). The paths were already preconfigured by Drupal.

Comment
It's no the best case, as you have to add the dependency manually, but it's just the first time, and later on it's just a `composer install` and composer will do magic.
Also, this relies on the paths set up by the user in the root composer.json, so as you said in your previous comment, we should allow the user to set where to save the libraries.

The next code should be added to root composer.json just before doing the `composer require drupal/chosen:2.x-dev`. (copied from you composer.json). It fetches from the github repository.

"chosen": {
            "type": "package",
            "package": {
                "name": "drmonty/chosen",
                "version": "1.6.1",
                "type": "drupal-library",
                "dist": {
                    "url": "https://github.com/harvesthq/chosen/releases/download/v1.6.1/chosen_v1.6.1.zip",
                    "type": "zip"
                },
                "require": {
                    "composer/installers": "~1.0"
                }
            }
        }

Another option maybe (need to test). Would be adding this to the root composer.json:
This selects specifically the chosen library from packagist and would add it in the proper folder.
Because drmonty/chosen package in packagist doesn't have the drupal-library type, it goes to the vendor. Another option would be contacting drmonty and asking him to add the drupal-library tag in his repository :)

{
    "extra": {
        "installer-paths": {
            "web/libraries": ["drmonty/chosen"]
        }
    }
}

The composer/installers part, well, the default one is the one that reads from the installer-paths property and will put it in the proper place. This is exactly how Drupal is setting the custom paths for all the modules and it's already required by drupal (at least 8.2.x), so, requiring it again in our composer.json won't do a lot...

So does that mean that our type definition is basically ignored?

Because we do define drupal-library, but if drmonty would need to add something in the repository then that means that composer does not care about this part of our definition.

So maybe we need to make a new packagist repository that has the proper tags?

If however this is not possible to fix inside chosen module, then i dont see why we have a composer.json the first place, if all we do is suggesting to copy something into the root composer.json, then it could be part of the README, and then we dont need the file at all.

Checking here, I don't see at all the option "type" to set it as "drupal-library".

https://getcomposer.org/doc/05-repositories.md#package-2

So I think, the custom "drupal-library" type is not being set :( and Composer doesn't complain of any parameter that is not using.

So, yes, we can try to make a new packagist repo with proper tags, or we can just contact drmonty to ask to add the tags, which would simplify everything.

The composer.json would be useful if we could fetch a package with the "drupal-library" tag already set and would be placed in the proper folder...

But here https://github.com/composer/installers its there.

Drupal
drupal-core
drupal-module
drupal-theme
drupal-library
drupal-profile
drupal-drush

And we require "composer/installers"

Yes, but I think that this has to be required by the main framework, not by this module :/ In this case the main framework is Drupal...

Its also in the main composer.json

{
    "name": "drupal/drupal",
    "description": "Drupal is an open source content management platform powering millions of websites and applications.",
    "type": "project",
    "license": "GPL-2.0+",
    "require": {
        "composer/installers": "^1.0.21",
        "wikimedia/composer-merge-plugin": "~1.3"
    },

Ok, summarize a bit of what I understand of this installers:

That should be included in the composer.json of the framework, and yes, it's in the Drupal's composer.json. Then, that works to add the module in the proper path, but only for modules, that are installed with the proper label.

The module that we reference in the Chosen's composer.json, is referenced via zip file from github, and that reference to the repository doesn't have any way to add a `drupal-library` tag, (comment #21), because the composer.json structure doesn't define the `"type": "drupal-library",` parameter inside a zip referenced file. You can add the type in the module, but not to a module that you are fetching from a zip, repository or somewhere else. Thus, Drupal, doesn't find the "drupal-library" tag in the zip repository and doesn't put it in the proper place.

Then, if you add the "composer/installers" inside the composer.json of Chosen, the only thing it would do is install this in an inner path inside of the chosen's module root, and still, because the referenced repository doesn't have the "drupal-library" part it won't go to the proper place. `composer/installers` it's only supposed to be added in the base framework (Drupal, Wordpress, Laravel, whatever), and yes it's already added, but still, because there is no label, it won't work.

So, summary, there is no way to add a "type" into a zip referenced repository or something directly fetched from github, and thus put it in the proper place. The only way to add this type is to add it in the composer/packagist configuration, that is what I said before to contact the owner (in this case drmonty) to add the type. Or the workaround with the composer.json in drupal, instead of setting in by label, set it directly by package name, so it goes to the proper place. If you set the path in the composer.json in Chosen module that will do it depending on Chosen's root path, and that can be in different places and paths.

Okay so if that is the case, then we have no way of fixing this json file in chosen module unless we somehow can contact the owner of the package or if we publish our own version of chosen on packagist.

It seems that the package is submitted by https://github.com/mrohnstock/chosen so maybe we could open an issue there.

Thanks to @Sturgelose's clues in #19, I was able to get this working the drupal-composer template by adding in composer.json:

to the repositories key;

        {
            "type": "package",
            "package": {
                "name": "drmonty/chosen",
                "version": "1.6.1",
                "type": "drupal-library",
                "dist": {
                    "url": "https://github.com/harvesthq/chosen/releases/download/v1.6.1/chosen_v1.6.1.zip",
                    "type": "zip"
                },
                "require": {
                    "composer/installers": "~1.0"
                }
            }
        }

to the installer-paths key:

            "web/libraries/{$name}": [
                "type:drupal-library"
            ]

However, as I read the code the path that chosen seeks is still specific to web/libraries/chosen, so I can't use the web/libraries/contrib/chosen pattern that would perhaps be best practice.

I just wanted to manage the dependency myself, as I am not sure https://github.com/RobLoach/component-installer is the preferred standard?

Additionally I didn't want the require.js + jquery dependencies installed in a components folder. So I removed those sections from the composer.json file.

I added this in my repository package section:

        {
            "type"    : "package",
            "package" : {
                "name"    : "harvesthq/chosen",
                "version" : "v1.5.0",
                "type"    : "drupal-library",
                "dist"    : {
                    "url"  : "https://github.com/harvesthq/chosen/archive/v1.5.0.tar.gz",
                    "type" : "tar"
                },
                "source"  : {
                    "url"       : "https://github.com/harvesthq/chosen.git",
                    "type"      : "git",
                    "reference" : "v1.5.0"
                }
            }
        },

and included with the following:

        "harvesthq/chosen": "1.5.0"

I think there is some misunderstanding about how Composer can be used to download the Chosen library.

The current composer.json adds a requirement to drmonty/chosen. That package is a shim repository for the actual Chosen library. To download the library the package depends on RobLoach/component-installer which in itself is kind of package manager for JavaScript and CSS within Composer.

By default the Components installer places downloaded library code in /components. This is not where the Chosen module is expects it to be. Is is possible to alter the default installation directory for the Components installer but that has to be specified in the root composer.json and thus goes against the require-and-forget goal.

The use of RobLoach/component-installer is also why require.js and jQuery is downloaded as mentioned by #28. They are both unnecessary for our usage.

Going forward I suggest the following:

1. Remove the drmonty/chosen dependency from the Chosen composer.json. In my opinion the Components installer adds needless complexity.
2. Update the README to explain that users have to add a package definition to their composer.json in the format suggested by #19. Composer does not load package definitions in required projects so adding the package definition to the Chosen composer.json should not work.

+1 for removing drmonty/chosen from the Composer requirements.
The components installer is not a standard used in Drupal country.

+1.

By default the Components installer places downloaded library code in /components. This is not where the Chosen module is expects it to be. Is is possible to alter the default installation directory for the Components installer but that has to be specified in the root composer.json and thus goes against the require-and-forget goal.

If I'm having to edit my composer.json anyway, it's better that I have to add something that is related to what I want ("A dependency on chosen") than something apparently unrelated (" an alternative path for the components installer"). This is also more maintainable, in that if I come back and look at my composer file there is more chance of me remembering what that line is doing there (particularly important as it is difficult to put comments in composer.json files).

The real problem here is the documentation in the README file, see #2836244: Update documentation for installing via composer

Closing it as duplicate of #2836244: Update documentation for installing via composer