Directory structure is fail, part 1: Composer

The purported advantage of PSR-0 is "you can autoload classes without having to think really hard about it". Merging code from multiple 3rd party sources would be highly problematic. Either way, this is not a Drupal issue. If you feel strongly about it, file a PR upstream with Composer.

Composer does not mix Symfony's component structures together because if it did, then some of the components might run into conflicting files. This isn't a broken system, but a fix to get around potential issues. It uses vendor/package so that you could have your own class-loader if you wanted at chx/class-loader, or drupal/class-loader. We don't need to worry about where it installs the component though, because autoload_namespaces.php handles that for us.

If we're that desperate to mix them together, one thing we could do is bring in the entire Symfony Framework by consuming symfony/symfony, and then remove the parts we don't need in core/vendor/.gitignore. Sticking the following in core/composer.json and running drush dl composer && drush composer install will do just that:

{
  "name": "drupal/drupal",
  "description": "Drupal is an open source content management platform powering millions of websites and applications.",
  "license": "GPL-2.0+",
  "require": {
    "symfony/symfony": "2.1.*",
    "twig/twig": "1.8.*",
    "doctrine/common": "2.3.*",
    "guzzle/http": "3.*",
    "kriswallsmith/assetic": "1.1.*"
  },
  "minimum-stability": "alpha"
}

The above would cause other adverse issues though. The real question we have to ask ourselves is why this is an issue for us. Once we determine that, then we can ask the right questions, and probably get the right answers.

Nevertheless, here, too, this is not critical. Having code harder to debug does not warrant the critical priority. It a major pain, though.

I'd love to help you out, chx. I've been using Netbeans lately. It allows you to click on a class name, and load up the file where that class is defined. xdebug also loads up the currently executing file, which is mighty handy. If I remember, you're in vim, correct? There are a number of project plugins for vim that will help you do that sort of stuff. I'll ask around what some of my other team members use.

@chx I completely understand where you're coming from. The directory structure is different than what one would expect. There are a few things to help you out though. If you tell Composer to bring in symfony/symfony instead of the individual components, it will bring in the whole thing and it might help you grok it a bit. I needed a few hours to set Netbeans up correctly to provide the Intellisense data correctly. If you give a bit of information about what tools you use, we might be able to help you out.

Thanks a lot,
Rob

I agree that this barely makes sense. Let's just pull the whole symfony/symfony. The piece-wise approach that we currently have doesn't make sense anymore. Drupal is a Symfony application, let's accept that.

Just a note, I'm much against pulling in the whole thing, as it's rather large and we don't need it all. Was just suggesting that as something to help developer needs. Not something to go into Drupal's git itself.

As a side note, Netbeans 7.3b2 is supposed to be (Symfony 2, Doctrine 2)-ready. I am still using 7.2, though.

http://netbeans.org/community/releases/73/

We're still using less than half of the components in the Symfony suite. I'm not sure that pulling in, but not using, the entire Forms system is a wise idea. Ibid for the entire Config and Translation systems. If we're not using them, it's going to confuse people who expect us to be using those megabytes worth of code we're making them download.

Yes that's potentially as confusing as weird file locations. The .gitignore option is interesting (could even be used to remove sub-components we're not using), but that's going to create a mis-match with composer so it feels like a hack.

Downgrading to normal since there's no indication the fix is better than the original issue here.

Sounds like you have two issues:

1. PSR-0 is simply about registering a namespace to a certain path. It keeps the full namespace in the path for simplicity and ease of switching class loaders if needed. For example, we're not using Composer's Class Loader, we're currently using Symfony's. Due to the simplicity of PSR-0, we could easily switch to any PSR-0 class loader if we wanted.
2. Installation to vendor/$vendor/$package... The packages we're consuming from Symfony all share different namespaces. Yes, they all share the "Symfony" namespace, but they're all different packages. Composer won't mix different packages together in order to avoid file conflicts and namespace collision.

It is possible to alter the installation path of certain packages with Composer Installers:

{
    "name": "drupal/drupal",
    "require": {
        "composer/installers": "*",
        // ...
    },
    "extra": {
        "installer-paths": {
            "symfony/": ["symfony/classloader", "symfony/yaml"]
        }
    }
}

This will install ClassLoader and YAML into the same place, but will complicate the situation way more than it needs to be. My recommendation, chx, would be to make your IDE open classes for you when you double click on them. The Composer team is very open to suggestions and improvements as well. If you open up an issue in Composer, I'm sure they'd love to talk with you about its design.

Thanks,
Rob

Rob, in #17 what complicates the situation? I am not as offended by the current file layout as chx apparently is but I agree that having all of our Symfony components in one directory tree would be cleaner. From your code sample it looks like it would be one extra line to update when we add a new Symfony component, which is not a routine action at this point.

in #17 what complicates the situation?

Just tried it out and it appears that only works for packages that make use of composer/installers itself. In my eyes though, it's just adding another layer of complexity that doesn't need to be there. Drupal itself does not need to depend on Composer Installers, so there isn't a real reason for us to add it. I'm all for making things easier for developers though!

How is it an issue at all what the directory structure is of our dependencies? That's composer's problem, not ours. The autoloader works, so can we move on?

@msonnabaum, chx's problem is that when debugging (which involves all the code executed in those dependencies if that happens to be where you're trying to debug) it's harder to find where things are with the current folder layout.

@RobLoach I'm not sure what #20 means, does that mean it 'just works' if we set this up, or that it doesn't?

@catch sure, but that's not our issue, it's composer's. If we really want to consider this a problem, then an issue should be filed with them, not us.

Also, this has been discussed before with plenty of good reasons why it is the way it is: https://github.com/composer/composer/issues/1428

Sounds like composer installers already fixes it though?

Also @RobLoach I figured out what #20 means, it wouldn't particularly bother me to add that dependency if there's consensus this would be worth having, rather that than a complex .gitignore file or similar.

Just wanted to point out that there are a number of contrib modules trying to figure out some of this as well, for D7 and D8.

It would be supercool-ideal if there were one vendor directory that both core and contrib could use: #1433256: Proposal: Composer as a Library Manager

This comment gives a mini rundown of some of the contrib projects using composer.

I think the "Avoid namespace clashes" argument is a bit pointless. Because if you have the following:

vendorA/package/Some/Thing/Bla.php
vendorB/package/Some/Thing/Bla.php

Then you *have* a conflict because these two classes have the same namespace and things are going to get really weird anyway :) And if those classes are not actually the same, this will probably be worse than an error

However, the real problem are not the PSR-0 files but the other stuff that we're pulling in composer. If you look at vendor/twig/twig, or vendor/doctrine/common, there are bin/doc/ext/lib/src folders with documentation, shell scripts, license files, tests and so on. And these would actually easily conflict with each other. So we could only explicitly change e.g. symfony/*, where we know that we only pull in PSR-0 files.

As I suggested in #4...

Possible solution

1. Replace all symfony/* dependencies in composer.json with one symfony/symfony dependency
2. Add all Symfony components that we don't need in core/.gitignore
3. Run a composer update
4. Watch as all Symfony vendor paths are moved into one directory (core/vendor/symfony/symfony/src/Symfony/*)
5. Realize this is actually a hacked solution as Drupal actually does not depend on all of Symfony, just a few of its components

The majority of your problem seems to surround PSR-0. PSR-4 will help fix that, which has a trackable issue, and even a PR from donquixote. Not really something we should fix in Drupal itself.

Unless I'm mistaken, the new PSR-4 support in Composer should allow us to put the individual Symfony components into custom specified directories, so that

Symfony\Component\HttpKernel\Kernel

would map to:

/core/vendor/symfony/http-kernel/Kernel.php

or alternatively perhaps even:

/core/vendor/Symfony/Component/HttpKernel/Kernel.php

Am I mistaken?

/core/vendor/symfony/http-kernel/Kernel.php

Please don't do this, we use camel case everywhere else, let's not start using hyphens and lower case in directory paths.

PSR-4 isn't about the client code controlling where files go. It's about letting the provider better control their internal organization. Unless Symfony decides to use PSR-4 for its component repos it has no impact here.

The "symfony/http-kernel" part of the path in #31 is the Composer vendor and package name. Those are all lower-case by convention, but are NOT part of the class name.

Composer manages vendor/ according to the configuration in each package's composer.json.

If it's a problem that symfony has a weird directory structure, then hash it out upstream in symfony-land.

They worked it out so that we can use whichever component we like without having to use the whole symfony stack. They did it by implementing the directory structure this issue finds problematic. It's a good solution. Thus, this is a feature and not a bug.

Do we have enough information? Can we close this issue?

I don't think so. Many people above have pointed out that the current situation is confusing.

I agree that this barely makes sense. Let's just pull the whole symfony/symfony. The piece-wise approach that we currently have doesn't make sense anymore. Drupal is a Symfony application, let's accept that.

+1 on this solution.

It sounds like the main objection to it was this:

We're still using less than half of the components in the Symfony suite. I'm not sure that pulling in, but not using, the entire Forms system is a wise idea. Ibid for the entire Config and Translation systems. If we're not using them, it's going to confuse people who expect us to be using those megabytes worth of code we're making them download.

However, consider that since Drupal 7 we've bundled all of jQuery UI with Drupal core, even though core itself only uses a tiny bit of it. I haven't heard of anyone being confused by that. It can even be a benefit; if you want to use part of the library that Drupal core doesn't happen to use, it's already there waiting for you and you don't have to go figure out how to include it.

Here's a nice quote on that subject, from the original issue #315035-86: Add jQuery UI elements to core (from @webchick, of course):

But, you don't want to tell people "Our content management framework ships with jQuery UI 1.7.2" without giving them ALL of jQuery UI 1.7.2 at their disposal. Otherwise you have to say "Our content management framework ships with select, arbitray parts of jQuery UI 1.7.2 that we needed for core's specific purposes. If you want anything else than that, you're on your own."

:)

Only difference I see is that Symfony is much bigger than jQuery UI, and also probably much more likely to have numerous security issues down the line (including in components that Drupal core doesn't use).... But overall it still makes sense to me.

Since this issue was last active, I have built a full-stack Symfony application. To say that Drupal 8 is "a Symfony application" at this point is simply not true. It's a sibling of Symfony fullstack, which many of the same conventions but some different ones, too. It also is a sibling of Silex, and in fact we have some conventions and naming borrowed from there as well (eg, "Providers" for the classes that provide dynamic service definitions).

Really, there's nothing here for Drupal to do. If you want one of our upstream vendors to use PSR-4 with a shorter directory tree, that issue should be filed with the upstream vendor. Without horribly bastardizing Composer I don't think there's anything we even could do here, and I still believe this is a non-issue in practice. This is the new PHP world order. It has more directories than the old one did. So it goes. :-)

OK, let's banish the phrase"Drupal is a Symfony application" :) But the proposed fix is still the same, and doesn't involve touching Composer at all.

Pulling in all of symfony/symfony would still cause more confusion rather than less, per comment #14. Symfony was designed so that individual components could be used individually. We're using maybe a third of the components that exist. Pulling in the other 2/3s just to move a directory around is "killing a mosquito with a mack truck" levels of excessive.

I think we're all-in with Composer at this point, but linking to the meta so we can revisit this if necessary.