Problem/Motivation

The core/vendor directory is for third-party vendor code kept in its original shipped form. Since its contents are auto-generated using Composer and it contains only PHP code, some Drupal core would like for it to only contain that PHP; additionally, they'd like the ability to make the entire folder inaccessible from the webroot.

However, there are 3rd party vendor could that must be web accessible, such as jQuery, Normalize.css, etc. And we are frequently seeing patches that try to update that code to Drupal coding standards.

We need to separate those 3rd party asset files from the rest of Drupal's code base.

Proposed resolution

Move 3rd party vendor code from core/misc to core/assets/vendor.

The rationale for using "core/assets/vendor" is currently summarized in comment #24.

Remaining tasks

Move the reset of core/misc into various core/assets/css, core/assets/js, and core/assets/images folders.

User interface changes

No user interfaces are changed.

API changes

No API changes are made as the paths are handled by system_library_info().

Files: 
CommentFileSizeAuthor
#32 1473066-32-misc-vendor.patch138.91 KBechoz
PASSED: [[SimpleTest]]: [MySQL] 58,454 pass(es).
[ View ]
#29 1473066-29-misc-vendor.patch138.91 KBJohnAlbin
PASSED: [[SimpleTest]]: [MySQL] 58,931 pass(es).
[ View ]
#29 1473066-interdiff-26-29.txt1.33 KBJohnAlbin
#26 1473066-26-misc-vendor.patch139.3 KBJohnAlbin
PASSED: [[SimpleTest]]: [MySQL] 58,183 pass(es).
[ View ]
#24 1473066-misc-vendor.patch138.96 KBJohnAlbin
PASSED: [[SimpleTest]]: [MySQL] 58,187 pass(es).
[ View ]

Comments

sun’s picture

One thing is clear:

If we're going to do this, then we will remove the additional lib/ directory "separator" in modules, too.

#1290658: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch] decided that we have to have a lib/ directory to separate the namespaced PHP class code from any other code.

--
Personally, I don't think it's a good idea to intermix PHP code with front-end assets within /core/vendor.

OTOH, I'd love to see that lib/ in modules to go away, so I'd be fine either way, but definitely not both at the same time.

RobLoach’s picture

  • lib/ is for the given project's PSR-0 PHP classes
  • vendor/ is for third party code (not limited to PHP)

They're two different things. Let's keep the lib discussion over at #1290658: Move all module-provided classes to PHP namespaces (PSR-0 or similar), and autoload them without the registry [policy, no patch] and #1431804: Bikeshed 'lib' vs. 'libraries' directory. This is to move jQuery UI into vendor since it is third party code. See Crell's post about it.

pounard’s picture

I agree with sun, mixing frontend code and PHP code sounds bad.

nod_’s picture

Priority:Normal» Major
Status:Postponed» Active

Please deceide if we can use core/vendor/ for JS, otherwise close this and it will leave in core/js/vendor/ #1663622: Change directory structure for JavaScript files.

We're not mixing front-end and back-end code, we're grouping third party dependencies, that's different.

nod_’s picture

Priority:Major» Normal

meh

pounard’s picture

@nod_ vendor/ is a PHP library dir, ideally it would be outside the webroot. I like to consider that all public assets should live somewhere in sites/ instead, away from PHP code.

RobLoach’s picture

@pounard Web assets can be hosted in vendor packages in Resources/public directories. I wrote up how we could accomplish something like this at https://github.com/mattfarina/Lootils/issues/1 .

In the mean time, #1663622: Change directory structure for JavaScript files seems like a good first step.

Effectively, we'd end up with a hook_library_info() 2.0 which would handle all the integration for us.

pounard’s picture

Title:Move jQuery UI into core/vendor/jquery/jquery-ui» [Meta] Move third-party assets to core/vendor packages
Status:Active» Postponed

The Resources/public dir sounds like a Symfony convention where to place public assets into components or bundles. We're not speaking about bundles but about self-contained JS scripts. Putting PHP and public JS assets side by side is missing the point, they should not live together. Looking at some other CMS (other technos but still) they actually copy all the assets on module enabling into a common public folder, which makes much more sense since it frees the code of any contraints of where to live (webroot or not), and it makes all the public assets for the client live in a self-contained minimal public webroot folder. I think third party JS libraries must not be mixed with PHP code in the vendor dir, we could use another folder somewhere else to put those, it would make much more sense.

Damien Tournoud’s picture

The way I see it, jquery, jquery-ui, jquery-form, etc. are all part of the core javascript framework and owned by the system module (which implement hook_library() for them). As a consequence they should live under core/modules/system (in a public or assets folder).

nod_’s picture

Status:Postponed» Active

That's what Rob proposed in the other issue (that can probably be closed), seems like we agree about that.

As for the name seems like symfony uses public so, let's go with that?

nod_’s picture

Title:[Meta] Move third-party assets to core/vendor packages» [Meta] Move third-party assets out of core/misc folder
jhodgdon’s picture

#1663622: Change directory structure for JavaScript files seems to be leaning in the direction of lumping Drupal-sourced and third-party code together under modules/system.

I think that is a bad idea, because of the reasons stated in the issue summary above:

What does matter is that its the untouched third-party vendor code (see README.txt). With jQuery UI in core/misc/ui, we are inclined to switch around the data structure, or edit its files.

RobLoach’s picture

Status:Active» Postponed

Needs the build system up first. I'll hopefully have some time very soon to work that out.

webchick’s picture

I'm not sure why this is postponed, but it's exceedingly dumb that we moosh together our JS and vendor JS in the same folder. IMO all third-party code should be in core/vendor. It makes it much easier to exclude it from searches and find/replace stuff.

jhodgdon’s picture

Status:Postponed» Active

+1000000 on #14. PLEASE.

nod_’s picture

Was postponed since we wanted to avoid the tedious folder-bikeshed before feature freeze since that's not holding anything back.

RobLoach’s picture

I sent a pull request up to components/jquery to add Composer support. This is the shim/built repository that's used by the Bower Package Manager. Would be nice to use the same repository as the JavaScript peoples. Then, whenever they put out an update, we'd also get it.

"require": {
    "components/jquery": "1.8.*"
}

We could do the same for components/jqueryui, and any other packages we need.

jhodgdon’s picture

Ummm. How is #17 related to this issue? This issue is about making *directory structure* for the third-party JS libraries, not about how to maintain them...

RobLoach’s picture

@jhodgdon That would give us the directory structure in core/vendor, as webchick pointed out in #14.

nod_’s picture

Priority:Normal» Major

Ok I'm ready to deal with it. What RobLoach says. We put our deps in composer.json and let composer handle it, would make updating libraries easier since that'll happen often in D8.

As for the system module I'm pretty sure we want a system.library.php or whatever with only system_library_info(), the function is currently 1012 lines long and sucks to read.

On this topic, I'd be open to generate some of the library definition with composer and put that somewhere kind of like what's done for autoload_classmap.php.

Anyway, it's time to get all this js out of core/misc.

jhodgdon’s picture

The issue summary needs an update. What is the plan here?

webchick’s picture

Priority:Major» Normal

This is just annoying, it doesn't actually break anything. Demoting back to normal.

nod_’s picture

Most of the action is happening in #1663622: Change directory structure for JavaScript files. I guess we can leave that one open to deal with non-third party scripts that are in core/misc.

JohnAlbin’s picture

Status:Active» Needs review
Issue tags:-Needs issue summary update
StatusFileSize
new138.96 KB
PASSED: [[SimpleTest]]: [MySQL] 58,187 pass(es).
[ View ]

Given that some people want to move the PHP files in core/vendor out of a webroot (seems reasonable to me), let's put 3rd party assets that need to be web accessible in a directory besides core/vendor.

Over in #1663622: Change directory structure for JavaScript files, we had a round-and-round discussion which I'll summarize here:

  1. Assets that need to be web accessible shouldn't be mixed with 3rd party PHP code in core/vendor
  2. We can't use core/libraries (and core/libraries/vendor) because we already have a similarly-named core/lib directory and it is way too late to change core/lib to core/src or whatever.
  3. We could use core/js, core/css, and core/images directories and also use core/js/vendor, core/css/vendor, and core/images/vendor directories. But that's a mess.
  4. We could use core/js, core/css, and core/images directories and also use core/assets-vendor or core/vendor-assets directories. And when people see core/vendor-assets or core/assets-vendor sitting next to core/vendor and look inside to see normalize and jquery, etc., they'll think “wtf? why is this separated from core/vendor?".
  5. We could use core/assets/vendor, core/assets/js, core/assets/css, and core/assets/images. And since I don't see a better option…

Here's the patch that implements option #5.

Looking at the number of minus signs in .jshintignore makes me happy.

JohnAlbin’s picture

Issue summary:View changes

Updated issue summary.

JohnAlbin’s picture

I should note that the directory names I used under core/assets/vendor were either taken from the Bower registry, which the majority of the code was already registered with. The 1 or 2 libraries that weren't registered with Bower all had GitHub repositories, so I took the project part of their URL as the directory name. In other words, I didn't come up with the names of those directories arbitrarily; they are the names chosen by the webdev community at-large.

JohnAlbin’s picture

StatusFileSize
new139.3 KB
PASSED: [[SimpleTest]]: [MySQL] 58,183 pass(es).
[ View ]

Whoops. Missed one reference to a moved misc/ file in ckeditor_library_info(). New patch with 1 line change from previous patch.

JohnAlbin’s picture

Issue summary:View changes

Updated issue summary.

nod_’s picture

Status:Needs review» Reviewed & tested by the community

+1 +1 +1 +1…

Tested the patch, couldn't break it and i see no references to core/misc for vendor libs.

Wim Leers’s picture

Status:Reviewed & tested by the community» Needs work

+1 of course :)

But:

@@ -1495,13 +1495,16 @@ function system_library_info() {
   // Farbtastic.
   $libraries['jquery.farbtastic'] = array(
     'title' => 'Farbtastic',
-    'website' => 'http://code.google.com/p/farbtastic/',
-    'version' => '1.2',
+    'website' => 'https://github.com/mattfarina/farbtastic',
+    'version' => '1.3u',

I'm pretty sure this is unrelated?

JohnAlbin’s picture

Status:Needs work» Needs review
StatusFileSize
new1.33 KB
new138.91 KB
PASSED: [[SimpleTest]]: [MySQL] 58,931 pass(es).
[ View ]

I'm pretty sure this is unrelated?

Good catch! I was playing around with using Bower to package the libraries and the GitHub version of Farbtastic doesn't have a 1.2 tag. I've reverted that change in the latest patch.

Incidentally, the change right after that I purposefully put in; that's the new jQuery dependency for Farbtastic. The dependency exists (you can see the jQuery variable in the last line of the farbastic.js) but was undeclared in the info hook.

Wim Leers’s picture

Status:Needs review» Reviewed & tested by the community

Incidentally, the change right after that I purposefully put in; that's the new jQuery dependency for Farbtastic. The dependency exists (you can see the jQuery variable in the last line of the farbastic.js) but was undeclared in the info hook.

Indeed, I saw that, and that's good :)

Back to RTBC!

alexpott’s picture

Status:Reviewed & tested by the community» Needs work
Issue tags:+Needs reroll

Needs a reroll

curl https://drupal.org/files/1473066-29-misc-vendor.patch | git a
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  138k  100  138k    0     0  17481      0  0:00:08  0:00:08 --:--:-- 24005
error: patch failed: core/modules/file/lib/Drupal/file/Tests/DownloadTest.php:43
error: core/modules/file/lib/Drupal/file/Tests/DownloadTest.php: patch does not apply
echoz’s picture

Status:Needs work» Needs review
StatusFileSize
new138.91 KB
PASSED: [[SimpleTest]]: [MySQL] 58,454 pass(es).
[ View ]

Re-roll!

JohnAlbin’s picture

Status:Needs review» Reviewed & tested by the community

Confirmed the patch in #32 is a proper re-roll.

JohnAlbin’s picture

Issue tags:-Needs reroll

Removing "needs reroll" tag.

catch’s picture

Status:Reviewed & tested by the community» Fixed

Drives me mad having everything in /misc, committed/pushed to 8.x.

JohnAlbin’s picture

Drives me mad having everything in /misc, committed/pushed to 8.x.

*Fist pump*

Ok! We'll start working on patches for the rest of core/misc. :-D

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary:View changes

Added link to rationale of using core/assets/vendor