The TWG was brought in to consult on a project application review (#2191991) and the policy on 3rd party libraries was at the heart of the issue. The policy seems to be fairly clear that any 3rd party libraries originally authored by the module maintainer are exempt but that has been misunderstood on a number of applications. For this reason I think we should, at the very least, clean up the language to be clearer that the code does not have to start on drupal.org to be allowed.
During the conversation about the TWG reading of this policy, it was observed that this policy seems contradictory to the standard applied to Drupal core. This policy identifies 3 reasons to grant an exception but none apply to jquery, jquery ui, nor composer components (like symfony) committed directly to drupal core in drupal 6, 7, and 8 respectively. This lead the TWG to discuss the merits of this policy and whether it should be relaxed.
The reasons in favor of enforcing this policy appear to be as follows:
- the added time between vulnerability discovery, upstream module maintainer update, and downstream drupal module maintainer update could be a frequent security problem
- it makes it too easy for the module maintainer to fork the code by modifying the committed library without patching upstream enabling and encouraging drift
- it's a bit of an antipattern because many modules may want to use the same library and without adding the libraries in a shared way with libraries or similar modules could conflict causing fatal errors in the case of php or duplication or undetected version conflicts in the case of javascript
- it's aesthetically displeasing to duplicate code you didn't write into your repository - it just "feels" wrong and package managers like composer recommend against it
So, should we loosen the current policy? If so we could mitigate some of the above concerns with the following requirements or recommendations in tne updated policy:
- external libraries must have explicit GPL2+ compatible licenses that allow for redistribution (or the committer must have the authority to relicense it (as the sole author)
- external libraries should only be committed to a project repo when it seems unlikely that another module will need to leverage the same library
- external libraries committed to a project repo must be updated in a reasonable amount of time with security updates
- external libraries committed to a project repo should not be modified without submitting a patch to the upstream project and should only be directly modified downstream when absolutely necessary
I personally (not speaking for TWG) would advocate for loosening the policy leaving in language encouraging developers to leverage libraries API but then leaving it to the individual to make the decision. It seems unfair to make the decision in favor of convenience for drupal core but force conceptual purity on contrib.
One of the main reasons cited for including the libraries in project repositories is for ease of installation, particularly on some hosting where users can use the GUI to install modules but may not be able to sftp to upload modules (and where they are not using a proper VCS). I think in an ideal world the libraries module would be able to install external dependencies in the same way the module GUI can or at the least D.O would have a build system similar to that used for install profiles to run make files for module builds so that downloadable zips could include the external dependencies (perhaps optionally). Either of these options would require a lot of engineering and to add the functionality to d.o we would probably need support from the DA.
Comments
Comment #1
kreynen commentedThese arguments are largely the same ones used to make this case repeatedly since the Drupal project started. It isn't just core that has been treated differently. There has been an "inner circle" that has either been given permissions through informal IRC conversations or just asserted that they had the right to commit 3rd party libraries. The LWG has drafted several licensing and git access policy changes to address the 3rd party code issues as well as establish a clear process and criteria for approving or rejecting these requests. We are currently waiting for feedback from the DA. Once they've had a chance to review the changes, the documents will be posted for public comment.
I think you'll be happy with the changes. They should make the project review process much easier. I think they also reinforce many of the other changes being discussed in #2453587: [policy, no patch] Changes to the project application review process.
We really want to remove the two-tiered system for licensing and 3rd party code as well. I often refer to this as the JQuery Update test. That project has always included 3rd party code that has never met the requirements for an exception. If we are going to allow JQuery Update to be hosted on and distributed from Drupal.org, then we need policies that would allow that project.
I think this is really a duplicate of #1856762: Revisit and Redefine Drupal's Policy on hosting 3rd party files and/or files that is not strictly GPL, but I'm going to leave it open. Hopefully we can close a lot of these when the new git and licensing policies are made public and/or adopted.
Comment #2
gisleThere was recently another issue #1850966: Clarify what "3rd party" means about libraries written by the project owner that was referred to the LWG. I think this issue is just a duplicate of that. It is now closed, since nobody objected to the interpretation given during the discussion period.
Also, judging from comment #18 in #2191991: [D7] UFP Identity module, in the final outcome there was no contest to that interpretation given in the events that led up to this issue.
Hence, the right of an author to host a library he/she has written him/herself on Drupal.org under GPLv2 and later is not contested and is already part of the present Git Repo policy. For the record: The LWG do not propose to introduce restrictions on this.
I think this can be closed as a duplicate of #1850966: Clarify what "3rd party" means (or at least moved on to RTBC), but I'll leave it to kreynen or tizzo to do so.
Comment #3
geerlingguy commentedFYI, I posted some notes from an informal gathering with kreynen, Dries, webchick, myself, Crell, and a few others at DrupalCon LA over at #2307487-6: Warn distributions using non-whitelisted licenses that entries will be removed. It was great to have an IRL discussion to get things moving!
Comment #4
wmnnd commentedDear LWG members,
I am the author of a little module that is very similar to jquery_update in that it exchanges the jquery.form JQuery plugin which is shipped with Drupal 7 with a newer version of that script. (By the way: jquery_update also updates jquery.form but to an older version than my module)
In accordance with the 3rd-party guidelines the reviewers rejected my module to be hosted on drupal.org. However, just like with jquery_update, a major point of my module is to make using this newer library more convenient. Requiring the users to install the library separately, would certainly have a major impact on the usefulness of my module. Last but not least, we are talking about one single JavaScript file, not a larger body of code that would indeed profit from a separation (I'm thinking of WYSIWYG modules or other more complex distributions of code).
This conflict highlights, in my opinion, the problem described earlier by kreynen:
I would love to hear your opinion on this issue; you can find the relevant part of the discussion about my module here: https://www.drupal.org/node/2402147#comment-9957529
Comment #5
gisleJust to give you an update on this: The LWG has prepared a set of policy document proposals regarding external libraries. These documents are currently being reviewed by the Drupal association. As soon as the review has been completed, I hope that these documents shall be made public, and that we have a real public discussion about this, hopefully leading up to a revised policy on third party libraries.
I see no point in hiding my own opinions about this: I think we should have equal policies for everybody. That means that we either unpublish JQuery Update (and probably a dozen of other projects that gained an exception through all sorts of informal means - such as private irc chats), or we amend the policy so that the process for being granted an exception is transparent, fair and predictable. (Please note that this is my opinion - I do not speak for the LWG.)
From the thread you're linking to, there is this statement:
And I am afraid that is the way things currently are. The LWG has been asked to not grant any exceptions to the Git Repo policy until a process and timeline for doing so has been approved by Drupal Association and the community. This is being worked on, but these things take time.
To get your application through the queue (IMHO a very worthwhile project), I'll suggest that you comply with the current policies (i.e. remove the external library and provide instructions for a two-step install).
If policies regarding libraries are relaxed at some point in the future, you can always create a new version with a more friendly install procedure.
Comment #6
lookitscook commentedAny update on this? We have a D8 admin theme and install profile based on MIT-licensed code (https://almsaeedstudio.com/preview and other libraries). I'd love to publish this project, but am not going to fight through a long complex submissions and exception request process just to share it. We're perfectly happy using it on our own and distributing outside the drupal.org ecosystem.
It's frankly crippling to not be able to include 3rd part libraries, and doesn't seem to make any sense if their licensing permits inclusion in GPL projects. It feels like we're being forced to set up a secondary project tracking/distribution system (Github, primarily) because of seemingly arbitrary Drupal politics, which really holds back any rapid progress and genuine openness in the community.
Seems like after over 7 years of discussion (https://www.drupal.org/node/418844, maybe longer?) and no change, it makes sense as a developer to simply stop using Drupal.org.
Comment #7
kreynen commentedGPLv2 code that uses an MIT dependency is not something that is being questioned. Technically you were supposed to have an exception to commit that to Drupal's git repository, but the LWG stopped enforcing 3rd party code "violations" for GPLv2+ compatible code > 6 months ago while the policy changes were being reviewed.
The LWG progress on making these changes stalled as some members left the LWG, but I recently took over as the LWG chair to get things moving again.
So go ahead and commit it. If you apply for a whitelisting entry at https://www.drupal.org/project/drupalorg_whitelist so the library can be added using a .make (in a distribution or a module), that is normally reviewed and approved in < 7 days.
Everyone on the LWG agrees and we have been volunteering our time to collect this and want licensing reviews to be one of the reasons people concern about licensing trust code from Drupal.org more than GitHub. The LWG works to make sure that anything you've downloaded from Drupal.org is GPLv2 compatible with the option of distributing your improvements as GPLv2 or GPLv3. A D8 theme that includes an MIT library is no different than the 3rd party code included in JQuery Update.
Comment #8
lookitscook commentedThanks so much for the thoughtful response. I know you, and many others, are working extremely hard on this. Was a bit frustrated last week not seeing an "official" stance re.
Felt as if the genuine desire to contribute was being (officially) stifled by (inconsistent) red tape. It's great to hear that's on hold and actively under review.
Comment #9
gisleThe Drupal Git Repository Usage policy was changed about three months ago.
The very hard restrictions on including 3rd party assets has been lifted, and been replaced with the much more reasonable:
I've also updated Policy on 3rd party libraries on Drupal.org (and some other documentation about this) to reflect the new policy.
I believe we now can close this as "Fixed".