Solarium has a permissive license which can be found here: https://github.com/solariumphp/solarium/blob/master/COPYING

I think it might make it much easier for people to install search_api_solr if they can download the module and be done with it. This also allows us to only update the library version at our leisure.

Thoughts?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Nick_vh created an issue. See original summary.

borisson_’s picture

The library is currently locked on version 3.4.1 so that's not really an argument.
Also, there are other modules (and drupal core) that require developers to run composer install before the library is usable.

Getting #2494073: Prevent modules which have unmet Composer dependencies from being installed into drupal core should be a good thing to improve the DX, so we don't have to.

Berdir’s picture

Yes, this will become common. Commerce will require it, Swiftmailer does, Payment/Currency too.

Core just removed the vendor directory too.

It's also not just about the license of the library you are using, drupal.org simply doesn't allow it, even if it is GPL. (pretty sure that what 7.x-1.x is doing is also not allowed actually).

I'd rather do the opposite and throw out support for having a local vendor folder. People will have to learn how to use composer/composer_manager properly. And it adds an event listener that has to run early on every request.

borisson_’s picture

I'd rather do the opposite and throw out support for having a local vendor folder

I think this is a great idea. It'll also remove the really weird bootstrap file that we currently use in the tests.

borisson_’s picture

Status: Active » Needs review
FileSize
3.96 KB
Nick_vh’s picture

Not so quick. Drupal 8.1.x might have removed it from the source but it is available when downloading the packaged version. Until we can do this with contributed modules I am very against removal of the fact to have support for local composer packages. I understand we're in php revolution but it is by far a standard that people understand fully.

We also do not depend on composer manager and if we want to remove the support for installations without composer manager then we need to adjust accordingly and make composer manager a dependency.

borisson_’s picture

Not so quick. Drupal 8.1.x might have removed it from the source but it is available when downloading the packaged version. Until we can do this with contributed modules I am very against removal of the fact to have support for local composer packages. I understand we're in php revolution but it is by far a standard that people understand fully.

We also do not depend on composer manager and if we want to remove the support for installations without composer manager then we need to adjust accordingly and make composer manager a dependency.

This is not possible for contributed modules because the vendor module should live in the root and that's not in the download of a contrib module.
I understand composer is a new thing for a lot of people, but since we're not the only module to include composer-libraries (see #3) we shouldn't do something different to other what other modules to make sure the learned skills from other modules can be transferred to solr.

We shouldn't depend on composer_manager either, there's multiple ways to composer, some of them are without composer_manager, see https://bojanz.wordpress.com/2015/09/18/d8-composer-definitive-intro/ for more.

Berdir’s picture

We will also need to update travis.yml so it uses a different method to download the library. Running composer require + composer update --lock in the drupal root should work.

@Nick

If people can figure out how to run composer in the module folder, then they can figure out composer_manager too. Including the code simply isn't an option, d.o doesn't allow it with the exception of core.

And as @borrisson_ said, a dependency on composer_manager isn't required, there are other ways too. We probably should update our install instructions to explain or point to an explanation on ways to do this.

borisson_’s picture

borisson_’s picture

FileSize
1.63 KB
5.59 KB

Changed the currently shipped docs.

drunken monkey’s picture

Status: Needs review » Needs work
+++ b/INSTALL.txt
@@ -71,6 +71,11 @@ Afterwards, go to [4] in your web browser to ensure Solr is running correctly.
+Before enabling this module, you should first install it's composer dependencies,

It's "its", not "it's".

Otherwise, this looks great, would be good to go in my opinion.
In the module's D7 version we've fused INSTALL.txt and README.txt, to avoid such duplication of content, but that's another issue.

borisson_’s picture

Status: Needs work » Needs review
FileSize
540 bytes
5.59 KB

Fixed #11.

bojanz’s picture

+1 if that means anything. This is the way things work now when you need a library.

Once https://github.com/wikimedia/composer-merge-plugin/pull/102 lands and we update merge plugin in core, we'll be able to tell people to just do "composer require drupal/search_api_solr", deprecating Composer Manager, it's been an awkward experience.

Xano’s picture

Any contributed module that includes Composer packages in its repository will inevitably clash with other modules having the same packages in their dependencies list. As long as the required versions are compatible, this will likely not cause real issues, but the moment the requirements include different version constraints, users will experience more severe problems without the root cause being clear. When using Composer to resolve dependencies, users will see useful error messages about these conflicts, telling them exactly what's wrong.

Anonymous’s picture

With

Include Solarium in the repository
https://www.drupal.org/node/2661698

underway, with comment there

> People will have to learn how to use composer/composer_manager properly

but composer not currently production-recommended over drush, what is the _current_ proper way for installing solarium library dependency?

E.g., given

	cat ~/d8_site.make
		core: 8.x
		api: 2
		projects:
		  drupal:
		    version: 8.0.3
		  devel:
		    version: 1.x-dev
		  key_value:
		    version: '1.0'
		  multiversion:
		    version: 1.0-alpha5
		  relaxed:
		    version: 1.0-alpha5
		  search_api:
		    version: 1.0-alpha12
		  search_api_solr:
		    version: 1.x-dev

	drush make -y ~/d8_site.make d8

Should the library be subsequently installed as

	composer require solarium/solarium 
		Using version ^3.5 for solarium/solarium
		./composer.json has been updated
		Loading composer repositories with package information
		Updating dependencies (including require-dev)
		  - Installing solarium/solarium (3.5.1)
		    Loading from cache

		> Drupal\Core\Composer\Composer::vendorTestCodeCleanup
		Writing lock file
		Generating autoload files
		> Drupal\Core\Composer\Composer::preAutoloadDump
		> Drupal\Core\Composer\Composer::ensureHtaccess

or otherwise integrated into the makefile.yaml?

And, if it's now `composer` for production-release, how should installed solarium library be updated?

EXEC of

	cd <drupal_root>/vendor/solarium/solarium
	composer update

pulls in a lot of other deps/updates

	Loading composer repositories with package information
	Updating dependencies (including require-dev)
	  - Installing symfony/yaml (v2.8.2)
	    Downloading: 100%         

	  - Installing phpunit/php-text-template (1.2.1)
	    Loading from cache

	  - Installing phpunit/phpunit-mock-objects (1.2.3)
	    Downloading: 100%         

	  - Installing phpunit/php-timer (1.0.7)
	    Loading from cache

	  - Installing phpunit/php-token-stream (1.2.2)
	    Downloading: 100%         

	  - Installing phpunit/php-file-iterator (1.4.1)
	    Loading from cache

	  - Installing phpunit/php-code-coverage (1.2.18)
	    Downloading: 100%         

	  - Installing phpunit/phpunit (3.7.38)
	    Downloading: 100%         

	  - Installing squizlabs/php_codesniffer (1.5.6)
	    Downloading: 100%         

	  - Installing zendframework/zendframework1 (1.12.17)
	    Downloading: 100%         

	  - Installing symfony/stopwatch (v3.0.2)
	    Downloading: 100%         

	  - Installing symfony/polyfill-mbstring (v1.1.0)
	    Loading from cache

	  - Installing symfony/console (v3.0.2)
	    Downloading: 100%         

	  - Installing symfony/filesystem (v3.0.2)
	    Downloading: 100%         

	  - Installing symfony/config (v3.0.2)
	    Downloading: 100%         

	  - Installing psr/log (1.0.0)
	    Loading from cache

	  - Installing symfony/event-dispatcher (v2.8.2)
	    Downloading: 100%         

	  - Installing guzzle/guzzle (v3.9.3)
	    Loading from cache

	  - Installing satooshi/php-coveralls (v0.7.1)
	    Downloading: 100%         

	phpunit/php-code-coverage suggests installing ext-xdebug (>=2.0.5)
	phpunit/phpunit suggests installing phpunit/php-invoker (~1.1)
	symfony/console suggests installing symfony/process ()
	symfony/event-dispatcher suggests installing symfony/dependency-injection ()
	symfony/event-dispatcher suggests installing symfony/http-kernel ()
	guzzle/guzzle suggests installing guzzlehttp/guzzle (Guzzle 5 has moved to a new package name. The package you have installed, Guzzle 3, is deprecated.)
	satooshi/php-coveralls suggests installing symfony/http-kernel (Allows Symfony integration)
	Writing lock file
	Generating autoload files

and it's unclear whether that's recommended when composer is not (yet) being used to manage all of the drupal install.

Nick_vh’s picture

IRC discussion regarding this issue:

[10:35am] Nick_vh: can we at least block the search_api_solr issue till that story is resolved
[10:35am] Nick_vh: so we can see what to do then?
[10:36am] Nick_vh: berdir: ^^
[10:36am] berdir: It's not really related to that. Fact is, you already have to run composer to get the library (as I said, committing it really is not an option)
[10:36am] berdir: and if you can use composer to do that, you can also figure out composer_manager
[10:36am] Nick_vh: berdir: that is why I initially wanted to include it in the module
[10:36am] Nick_vh: for the newbies
[10:36am] Nick_vh: but understandably, this brings other complications
[10:37am] berdir: IMHO, installing solr is a lot harder than doing htis
[10:37am] Nick_vh: not if you use hosted solr
[10:37am] Nick_vh: berdir: trust me, there are a ton of people who never did composer
[10:37am] Nick_vh: and a simple dpm to tell them they need to go to the d.o link and follow instructions
[10:38am] Nick_vh: isn’t too much to ask I think
[10:38am] Nick_vh: documentation has never hurt anyone
[10:38am] berdir: sorry, missed part of what you said
[10:39am] berdir: sure, I'm not against adding documentation
[10:39am] berdir: I also have no problem with waiting to remove it until the core issue about the composer dependency thing is fixed
[10:39am] Nick_vh: it’s either that or include that ourselves if I can make the call
[10:39am] berdir: or we can do that already now as our own install requirements check and then remove it when core is fixed
[10:40am] Nick_vh: berdir: right - that’s what I proposed in irc to borisson_ earlier
[10:40am] berdir: ok
[10:40am] Nick_vh: we should’ve added the irc log to the issue I guess
[10:40am] berdir: that works for me. I don't think I can find time to do it myself, but +1 to that and happy to commit it if someone does it

So please either block this issue on the core issue #2494073: Prevent modules which have unmet Composer dependencies from being installed or make a patch that includes that documentation if the requirements of the search_api_solr fail (eg, solarium is not found).

Either one of those two will result in a "Fixed" state.

Nick_vh’s picture

Nick_vh’s picture

Status: Needs review » Needs work
bojanz’s picture

The core issue is just a generic way of doing:

/**
 * Implements hook_requirements().
 */
function address_requirements($phase) {
  $requirements = [];
  if ($phase == 'install') {
    if (!class_exists('\CommerceGuys\Addressing\Repository\AddressFormatRepository')) {
      $requirements['addressing_library'] = [
        'description' => t('Address requires the commerceguys/addressing library.'),
        'severity' => REQUIREMENT_ERROR,
      ];
    }

  return $requirements;
}

A patch in here could easily do something similar.

Anonymous’s picture

borisson_’s picture

It looks like mollux already did that in https://github.com/mollux/search_api_solr/commit/7ade94b4f58590a0cdadf2a...

So to keep better attribution, we could commit this and merge that pull request. If you want I can copy that in my patch as well though.

Berdir’s picture

I'm OK with a separate issue/pull request for that. However, I would prefer to have this as a PR on https://github.com/amateescu/search_api_solr so we are sure that at least the test environment set up still works on travis. See #8, I think that wasn't addressed yet?

borisson_’s picture

I created a pull request, I think we can update the .travis.yml in another issue #2662612: Update travis.yml to support PHP 7

mkalkbrenner’s picture

Status: Needs work » Reviewed & tested by the community

I came across a similar issue in search_api_solr_multilingual where I add another dependency managed via composer.
I implemented and documented it the same way as the current suggested patch. If I understand the discussion correctly we already have a consensus on the latest patch, right?
I tested the installation process from scratch and therefor I mark this as RTBC.

mkalkbrenner’s picture

Title: Include Solarium in the repository » Keep Solarium managed via composer and improve documentation
Category: Plan » Task
Berdir’s picture

Berdir’s picture

Oh, forget that, looks like it was already addressed there.

mkalkbrenner’s picture

Last call ;-)

I will commit this if nobody votes against this patch anymore.

Berdir’s picture

This being the PR referenced above?

Fine with me. We should open a follow-up to add a requirements hook, see #20 + #22, though.

mkalkbrenner’s picture

Sure, we can open a follow-up. But this follow up should be obsolete due to #2494073: Prevent modules which have unmet Composer dependencies from being installed.

Berdir’s picture

It will be when that gets committed yes. No idea when that will happen, no activity on that issue for quite a while now. Which is why we said we want to have something in the module for now.

mkalkbrenner’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

geerlingguy’s picture

Just a note, as someone who's more sympathetic to @Nick_vh's comments through this issue (and on IRC): As a long-time Drupalist and someone who regularly uses Composer outside of Drupal work, the cognitive dissonance I get from trying to get all the dependencies working in a typical D8 site is alarming...

When I was testing this module in a few different scenarios, I had different results each time, and it took me maybe 30 min to an hour to finally get the *@$( dependencies straight with different versions (e.g. latest alpha as of this writing, and dev release).

  • Prior to D8, process was download, or drush dl the module, install, and done.
  • D8 current alpha release (prior to this issue): process was download, notice there's a composer.json file in the module after noticing fatal exceptions when installing and using the module, then cd into this module's folder, and then run composer install, then run drush cr, then go back to configuring the module.
  • D8 current dev release (after this issue): Now I have to also download and install Composer Manager, read through this docs page to figure out how to use Composer Manager, or adapt my Composer workflow to the new status quo (as mentioned by Berdir earlier in this thread), and all this after reading through this docs page, which doesn't really offer a "Do this.", more of a "here are ways to do this", listing just one option...

My point isn't necessarily that we're doing things wrong—it's that @Nick_vh has an EXTREMELY valid point here:

[10:36am] berdir: and if you can use composer to do that, you can also figure out composer_manager
[10:36am] Nick_vh: berdir: that is why I initially wanted to include it in the module
[10:36am] Nick_vh: for the newbies
[10:36am] Nick_vh: but understandably, this brings other complications
[10:37am] berdir: IMHO, installing solr is a lot harder than doing htis
[10:37am] Nick_vh: not if you use hosted solr
[10:37am] Nick_vh: berdir: trust me, there are a ton of people who never did composer

To be honest, just composer use itself is enough that many people I support for Hosted Solr (and other activities) are going to get their minds blown... but we can work through that. The main thing is we need to give very reliable, step-by-step documentation at every place where people might encounter these issues.

#2494073: Prevent modules which have unmet Composer dependencies from being installed is hugely important, imo, and I think it should be a blocker to the next alpha/beta of this module, since many people who click the download link on Drupal.org will get hit by this. And maybe adding a step-by-step install guide on the project page too. (And the same goes for any other Drupal module that is going to rely on dependencies that will normally be managed by Composer Manager).

bojanz’s picture

Thanks to the fixes that are a part of Drupal 8.0.6 and 8.1-RC1, we can stop recommending Composer Manager (as long as we require those minimal core versions).

The current Commerce instructions:

Run these commands in the root of your website:
1) composer config repositories.drupal composer https://packagist.drupal-composer.org
2) composer require "drupal/commerce 8.2.x-dev"

That's it, two steps. All dependent modules and libraries downloaded.
The first step will be gone once Drupal Packagist is deployed on drupal.org infrastructure.
The Commerce project page has a note that says "The tarballs are for informative purposes only, Composer must be used for installation."

geerlingguy’s picture

The Commerce project page has a note that says "The tarballs are for informative purposes only, Composer must be used for installation."

@Nick - do you think we could get something similar added to the project page for Drupal 8 install instructions?

geerlingguy’s picture

@bojanz - Thanks for your follow-up too; for Search API Solr, all you need to do to install with composer (and not Composer Manager) for an existing Drupal 8 codebase is:

$ composer config repositories.drupal composer https://packagist.drupal-composer.org
$ composer require "drupal/search_api_solr 8.1.x-dev"

Worked great, and this is much more intuitive for someone familiar with Composer than using Composer Manager.

Could we get this kind of instruction added on the project page or in the install/readme?

ibakayoko’s picture

#39 works for me.

Thank you for the solution.

The modules are not added into the contrib folder. That is the only issue.

Xano’s picture

Use https://github.com/composer/installers to install Drupal extensions in the correct folders.

bojanz’s picture

@ibakayoo
All you need is an installer path in your composer.json.
See the bottom of https://github.com/drupal-composer/drupal-project/blob/8.x/composer.json for an example

ressa’s picture

I still use drush and not Composer to download modules. For a quick-start with the latest (alpha) version, the installation commands could be like this:

$ drush dl search_api_solr search_api
$ composer config repositories.drupal composer https://packagist.drupal-composer.org
$ composer require "drupal/search_api_solr 8.1"
$ drush en search_api_solr_defaults -y
bojanz’s picture

So you download search_api_solr with Drush, then redownload it with Composer? Your first Drush line ends up having no effect.

ressa’s picture

I had the same experience like @ibakayoko: "The modules are not added into the contrib folder. That is the only issue.".
I just tested again, and it was because Composer couldn't find the module with a tag like "drupal/search_api_solr 8.1":

- The requested package drupal/search_api_solr 8.1 exists as drupal/search_api_solr[7.1.0, 7.1.0-beta1, 7.1.0-beta2, 7.1.0-beta3, 7.1.0-beta4, 7.1.0-rc1, 7.1.0-rc2, 7.1.0-rc3, 7.1.0-rc4, 7.1.0-rc5, 7.1.1, 7.1.10, 7.1.11, 7.1.2, 7.1.3, 7.1.4, 7.1.5, 7.1.6, 7.1.7, 7.1.8, 7.1.9, 8.1.0-alpha1, 8.1.0-alpha2, 8.1.0-alpha3, 8.1.0-alpha4, 8.1.0-alpha5, 8.1.0-alpha6, dev-7.x-1.x, 7.1.x-dev, dev-8.x-1.x, 8.1.x-dev] but these are rejected by your constraint.

So these commands work and downloads both search_api_solr and search_api. So yeah, strike the first line :-)

$ composer config repositories.drupal composer https://packages.drupal.org/8
$ composer require "drupal/search_api_solr 1.0.0-alpha6"
$ drush en search_api_solr_defaults -y

NOTE: updated commands to use https://packages.drupal.org/8 and the new package syntax.

Search API Solr fulltext search View at http://example.com/solr-search/content

letuptit’s picture