Problem/Motivation
This issue is a blocker for #1612910-384: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)
Core needs to be able to determine, internally, if dependencies are met when installing a module. Modules may need to rely on a particular bug being fixed in another module, or within core itself in order to function properly. If a module cannot say "This module requires drupal 8.5.3 or higher" due to something that got fixed in that version, then we'll end up with broken sites because incompatible code was allowed to satisfy a dependency. This issue enables core to be able to support semantic versioning internally, as a feature of the platform.
The module handler includes a parseDependency() method that converts a module version string into an array format expected by the closely related drupal_check_incompatibility(), which is a global function but depends on this expected format.
Proposed resolution
- Add a new
Dependencyclass toDrupal\Core\Extension. - Introduce a new Version component to core. The component consists of one class:
Constraint
Both classes are immutable value objects.
Dependency represents a dependency and can be created from an array or a string such as drupal:node (>8.x-1.1).
Constraint represents a Drupal version constraint string and is created from a string like >8.x-1.1.
drupal_check_incompatibility() is replaced by Dependency::isCompatible().
Module extension objects have a public requires property. This used a have a array of dependency arrays - now they are Dependency objects.
Remaining tasks
Allow this code to understand semver? - this is followup.
User interface changes
N/A.
API changes
See proposed resolution and https://www.drupal.org/node/2756875
Data model changes
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #153 | 2677532-153.patch | 66.95 KB | eric_a |
| #153 | interdiff.txt | 1.54 KB | eric_a |
| #148 | 2677532-148.patch | 65.41 KB | alexpott |
| #148 | 143-148-interdiff.txt | 1.39 KB | alexpott |
| #143 | 2677532-143.patch | 63.94 KB | andypost |
Comments
Comment #2
dawehnerDo you really believe this is major?
On the first read of the issue I thought, why does this belong to the ModuleHandler, this feels much more like a install level code, so should rather belong into the ModuleInstaller, but there is indeed parseDependencies, so meh. When you look onto #2208429: Extension System, Part III: ExtensionList, ModuleExtensionList and ProfileExtensionList though you could argue that this maybe also belongs onto the extension list ... especially, because it is potential shared code between themes and modules.
To sum things up ideally this doesn't belong onto the ModuleHandler, because the module handler should be about runtime level stuff (hooks), but meh, I just hope we can move the code to the right place, once we have the extension list.
Comment #3
xjm@dawehner LOL no, that was an accident.
Maybe we want to move the other method from ModuleHandler too?
TBH I just filed this because I was annoyed I couldn't use the data provider from the
parseDependencies()test to also test this function. :)Comment #4
dawehnerHa: annoyance driven development, this is quite of a thing.
Ideally yes, but well, ModuleExtensionList doesnt' yet exist, maybe we could create it?
Comment #6
dawehnerWorking on moving this to a VersionChecker component.
Comment #7
dawehnerIt is certainly not the case that I really enjoy this API.:
Comment #8
dawehnerComment #9
dawehnerFor some reason "do not test" was selected by default.
Comment #10
alexpottI like this patch.
Don't need this :)
Needs a class comment.
Deprecate for 9 too? And given there are only two usages I think we could do the replacement with
VersionChecker::parseDependency()in this issue.Comment #11
alexpottDoesn't hurt to have a short comment like the provider below.
Comment #12
dawehnerOne thing I'm wondering is: should this actually be a normal class, that can be extended for example.
Just in general this API will never be called on runtime, so worrying about microperformance doesn't really work.
Comment #13
dawehnerComment #14
dawehner.
Comment #15
dawehnerLet's replace the two usages in core.
Comment #16
dawehnerSome more places.
Comment #17
MixologicIm not sure what the difference would be, but I would say that this API does get called at runtime by modules on drupal.org, namely the project_dependency and project_composer modules, (parseDependency moreso than the drupal_check_incompatibility stuff).
In any case, moving this to its own versionchecker class++
Minor nit: Should be 8.2.0
Also, should this need to happen first before we make a patch for : #2641658: Module version dependency in .info.yml is ineffective for patch releases ?
Comment #18
dawehnerWell we could, at least adding something like that to the module handler is kinda wrong at that point in time.
Comment #19
dawehnerThank you @Mixologic for the review!
I also added a change record
Comment #20
MixologicThinking hypothetically here, if this were a "normal class", and drupal.org was upgraded to drupal 8, and we had need of a parser that handled all of drupal's possible version numbering schemes, past or present, would that allow us to exend this? Or is it extendable in its current state?
Comment #21
dawehnerAt least the parseDependency method is quite helpful and generic. Whether its actually useful for you, I'm not sure about that, but for now this method doesn't have any restrictions in that area. On the other hand, all that code already exists in the wild as well as functions you can just copy.
Are you looking for some composer library or so?
Comment #22
MixologicMostly my musings were supposed to provide some use cases to help answer the question you were asking earlier in #12
I havent looked closely enough at everything to know the answer, just throwing some ideas out to see if that would be helpful.
Comment #23
dawehnerWell, you can do the same with static methods, its just more convenient when you have a class.
Comment #25
mile23Reroll.
Also, project_composer basically has re-implemented all this logic in what might be a better way. See #2641658-21: Module version dependency in .info.yml is ineffective for patch releases
So moving this to
Componentwould be a good way to go regardless, and then as a follow-up fold in the improvements from project_composer.Comment #26
dawehner@Mile23
So do you want to move this to
Component? I certainly like that, even i'm not sure this version limitation is maybe just specific to Drupal itself?Comment #27
mile23You know all those tests we made to catch errors in moving something to
Component? I think I hit all of them. :-)I also ran into trouble with
InstallerRedirectTraitTestbut I'm not sure it's related to the changes here.Here's a patch for the testbot to chew on while I run the tests without the changes and see if it is related.
Anyway: The substantive change here is that
VersionCheckeris now underComponents.Comment #28
mile23InstallerRedirectTrait fixed here: #2845662: InstallerRedirectTraitTest fails under PHP 7.0.0, passes PHP 5.6.10
Comment #32
dawehnerI guess we need to update this number @_
Comment #33
mile23Hmm. That doesn't look like a Component. :-)
Comment #34
mile23We could make a lot of changes to this code and turn it into a component that allows other projects to resolve Drupal extension dependencies. It might be nice to have such a thing, because the composer facade could use it, or an external CLI like drush might. (OTOH, the composer facade already knows how to do this, so we might import it into core. See #2641658-21: Module version dependency in .info.yml is ineffective for patch releases)
But for consistency's sake we should probably leave this Core and not move it to Component. That way core can move forward on #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning and others.
This patch is a re-roll of #25 with CS changes and updates from #32. #27 only moves the code to the component namespace, without behavioral changes.
Comment #35
mile23I guess it might be nice to include the patches.
Comment #37
mile23I couldn't repro the
Drupal\Tests\aggregator\Functional\DeleteFeedItemTestfail, butVersionCheckerTestneeded to usePHPUnit\Framework\TestCaseinstead of the underscore version.Comment #38
dawehnerShould we also add
@trigger_errorstatements here?Comment #39
mile23Added @trigger_error(), linked to CR, updated tests so they don't explode on @trigger_error().
Comment #40
markhalliwellAccording to the CR, this is just a 1:1 map of existing methods:
Before:
After:
Now that we're finally separating this out into its own class, can we finally also take the opportunity to update things here?
drupal_check_incompatibility()has never really made much sense and is usually creates a semantically backwards "flow" whenever its used in code (e.g. returns NULL if compatible). Plus its still using "arrays".... ugh.I think it would make more sense to create a standalone
Dependencyclass rather than a "version checker". This ensures that we have a standardized object to work with. I also think that we should create a separateVersionclass since that has entirely separate logic/assumptions that go with it.Then we can compare both versions and dependencies separately or together as needed:
So with the above "after" example, this would turn into:
Of course, if places where the machine name of the dependency is also needed, it could be done in the following way:
Comment #41
mile23@markcarver: I think that's the right instinct. If the eventual goal is to use semantic versioning on extensions (which it seems to be) then we should eventually normalize on composer/semver. We could make a shim to convert Drupalisms to semver-useable data structures for comparison.
That should be a follow-up, so it's easier to test the new behavior without having to include an .inc file.
And yah: *arrays.* :-)
Comment #42
markhalliwellThat's all well and good, but I'm just saying that I don't think it behooves us to simply create a 1:1 map of procedural code to simply stick into an arbitrary
VersionCheckerclass (maybe its just the name... IDK).In my mind, if we're going to create classes and move procedural code into them, then let's do it right from the get-go.
Even if
LegacyVersion(or something similar) is the first class that is created here, that's still better than returning arrays.All
Dependencyshould really really care about is matching against a class that implementsVersionInterface.The follow-up you mention could definitely be
SemanticVersion, for example, if/once we decide to go that route.Comment #43
MixologicWe do things in small chunks because that stands a much better chance of being understood, reviewed, and accepted, one small step at at time. If you go for the gold, and try and 'do everything right', you may land in a place of perfection, but it'll never make it in, regardless of how correct or just it might be, simply due to the limited time and attention span folks have for ensuring that things work as intended without any unintended consequences.
Given that we now have semver in core, we can actually discard the "do it right from the get-go" mindset that used to be required because you only had one shot at the next massive release. Now we can work on stuff, and it actually goes in, and it actually sees the light of day within six months.
So, Im +1 on the first step, and then +1 on the followup, which, your suggestions are totally sane, and reasonable, and the right way to go. We just shouldnt sprint the whole mile its gonna take to get there.
Comment #44
markhalliwellAgain... not really point I'm trying to make here.
The "small chunks" argument really only flies or makes sense when it involves extremely complex sub-systems. These are literally just two, relatively small, procedural functions (in separate places). Please don't patronize me. I know how core works; it's why I had to take a sabbatical from it.
These should be converted into proper type based objects, to begin with, because they are so simple and because they do very specific types of logic (which should be unit tested... separately).
Just by creating
VersionChecker, it adds this class to core's technical debt and won't be able to be removed until 9.x, regardless of when it becomes "officially" deprecated.Which, as you have stated, is likely to happen a lot sooner than later. So what is the point of adding this class if it's likely to get replaced soon anyway?
When we do end up with something "better", we will have created a duplicate BC chain from
drupal_check_incompatibility()toVersionCheckerto "something better". This isn't ideal nor is it good planning.Which brings me to kind of my entire point here: we shouldn't move things into a class just for "the sake of X". We should be evaluating what the old procedural code is attempting to "solve" and figure out if there is a better [8.x] solution, make a plan, and then move forward with that.
We should always avoid adding a relatively useless "1:1 mapping class". That has rarely served us well in the past unless its explicit purpose is to provide BC support (which this isn't).
I'm really not trying to be argumentive here. I was just offering a better solution than simply doing a c&p + rename to some arbitrary class.
I didn't change the status to CNW is that I understand that some of y'all might not see it this way and I don't want to hold up the issue... but...
I just felt it was necessary to bring attention to how much of a PITA
drupal_check_incompatibility()has been in the past.I believe it's a mistake to continue down a path that doesn't serve any real value other than "getting it out of an include file", which isn't enough of a reason to simply push through an issue.
Comment #45
mile23Try a patch if you'd like.
Comment #46
MixologicApologies If I came across as patronizing, its just that this particular issue is a hard blocker for a very important community need, that of semantic versioning in contrib, and this function is only used twice in core, and likely doesnt have a lot of uses in contrib - the only example I'm able to find is project_dependency, on drupal.org.
Can you help me understand
PITA for who? Where else is this getting used?
Nevertheless,
drupal_check_incompatibility()exists now, is part of the public api, and therefore has to continue to return its less than ideal array structure on the off chance that some contrib or custom code is currently reliant on it. We cannot remove that until drupal 9 without potentially breaking somebodies site.In both usages in core, we do this str_replace to pass in a version that can actually be checked. Is there any reason we couldnt refactor that to live inside of the checkIncompatibility method? It sorta changes the api, but arguably the api wouldnt work without it.
In both usages in core, we do this str_replace to make checkIncompatibility actually work. Would it make sense to refactor this into the method itself? Seems like the method needs the strings in a stripped format, would it be harmful to strip them inside the method?
Comment #48
mile23Reroll of #39, updated deprecation messages for 8.7.0.
Comment #49
effulgentsia commentedAssuming this is true, raising this to Major. However, can the issue summary be updated to explain why it's a blocker for that? I'm assuming there's something in contrib or on drupal.org that is incapable of loading common.inc and calling a global function that's defined there? Whether that's the reason or something else, it would be good to document it in the issue summary.
This issue still has the "Needs change record updates" tag. What in the CR needs to be updated?
Comment #50
mile23This issue is a bit of a mess because the process of trying to make core do semver is also a mess. :-)
It's a blocker for #1612910-384: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)
I think the idea here was to put version-check-y stuff in one place. In other issues like #2641658-30: Module version dependency in .info.yml is ineffective for patch releases we'd make it a component in order to have some exportable maintained code that does the work of converting between Drupal-flavored and semver for use by d.o and others.
It needs a change record update if we're not going to make
VersionCheckerpart of a component.Comment #51
andypost8.7.x now
guess it need trigger deprecation error
8.7.x
Comment #52
MixologicIssue summary annotated with explanations.
Comment #53
mile23Not sure what #51 is referring to...
Turned this into a component, because we had some consensus that was a good idea. I put it in a component called Extension, but there might be a better place. Adding it to Utility means other consumers will end up with all the dependencies there, such as drupal/core-render, even if they don't need them.
Turning it into a component necessitated adding an optional argument to parseDependency(), since components should never rely on core being present.
Comment #54
mile23Got the messages all wrong, in different ways!
Comment #55
larowlanshould we use the third argument here e.g.
explode(':', $dependency, 2)I realise this is just moved code
should these share a common trait?
can we wrap these as per the coding standards? will be a bit easier to read/review diffs in future
Comment #56
mile23#55.1: In that section of code we're only removing the project name from $dependency and storing it.
#55.2: Similar but different data, so it's not portable.
#55.3: Done.
#55.4: It looks like
ModuleHandler::parseDependency()can supportviews_ui(8.x-1.1-alpha12)but notviews_ui(8.x-1.1-beta)because the regex wants the last character to be a digit. This gets back to the inflexibility of this code and why we need to update it in other issues. So let's make a follow-up for that: #3001344: ModuleHandler::parseDependency() can deal with beta1 but not beta'needs followup' was in relation to having a semver shim issue, which is here (and other places): #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc)
Comment #57
mile23Updated CR.
Comment #58
kim.pepperHad a quick scan and found 1 nit:
I read 4 keys. Is it a standard to use '=' for optional keys?
Comment #59
mile23Good catch. :-)
Comment #60
MixologicIf/when this goes in, it would be great to expand it to handle all historical drupal versions, such that drupal.org can use this component when dealing with legacy projects. But I guess it premature to open a follow-up until it actually exists.
Comment #61
mile23What we need here is a well-stated plan/meta. #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) is up to 407 comments.
Comment #62
borisson_Just nitpicks, nothing major in here.
Is
nullcorrect here?I think I'd prefer to have FALSE instead. I think that more correctly contains the meaning of what we return.
If ya'll agree, we should also update the code to reflect this :)
This will probably make the patch harder to implement though, so probably not a great idea.
by ::checkIncompatility?
I think this also supports themes right? We should probably mention that. Or remove the word module entirely, I think dependency conveys enough meaning.
I don't like using empty like that, but that is just pure personal preference. I prefer seeing
$core_compatiblity === NULL ? $core_compatibility : static::...Not just because it removes the empty on a non-iterable thing, but also because it tests for the positive instead of the negative case.
This could lead to trouble when there are multiple colons in
$dependency. Should we guard for that by throwing an exception when there is > 1? Or by doingexplode(':', $dependency, 1);This code isn't super complicated, but it looks like that because of the nesting and use of regexes.
We can't really do anything about the use of the regex, but we can improve the nesting by reversing them.
It looks like this might end up with just an open bracket, and no closing bracket, are we testing for that usecase?
It also looks weird that the original version always starts with a space, I think that should be the worry of the UI code, not this code.
Comment #63
phenaproximaThis is a real nice patch.
This seems to me like a good candidate for a value object, which the deprecated procedural functions can unwrap into an array. That might be out of scope for this issue, though...?
Why not assertSame() here?
Should we also test greater-or-equal and not-equal, at least?
Comment #64
borisson_I agree with #63.1 this sounds like an ideal usecase for a valueobject, I think the rule is that we try to avoid intruducing new array-api's, but I'm not sure how that works for conversions like this.
Comment #65
larowlan> That might be out of scope for this issue, though
I agree that a value object should be the ultimate goal here, but I feel that would ordinarily be out of scope.
However, once we put an api out there, especially a component, we've got to support it for ever. An api is for life, not just for christmas. Or is that a puppy. You get the idea.
But on the other hand are we sure we want to mix adding a new API with a story that blocks semver support in contrib.
Personally, I'm leaning towards making this something we want to support in the future. Thoughts?
Comment #66
kim.pepperI agree with @larowlan. New APIs should be OOP. Would it be worth punting to a follow up in the 8.7.x cycle? The API isn't 'official' until there is a stable release right?
Comment #67
kim.pepperI'm going to have a crack at it.
Comment #68
kim.pepperI've added a value object as per #63.1 #64 and #65
I haven't addressed all the feedback in #63 and #64
Comment #70
kim.pepperLooks like I missed core/tests/Drupal/Tests/Core/Extension/DeprecatedModuleHandlerTest.php
Comment #72
larowlanThanks Kim for keeping this moving
You'll need to retain the old return here, so will need to unpack the value object into an array
are there uses for these setters. alter hooks I guess?
Comment #73
kim.pepperComment #74
larowlan> I think this needs to be changed for callers of VersionChecker::parseDependency()
Yes, in core. But we have to retain the existing functionality for contrib.
Comment #76
kim.pepperRe #74 discussed this with @larowlan in slack and confirmed VersionChecker::checkIncompatibility() is still returning a string, so no BC issue with that.
Fixed a couple more instances calling VersionChecker::checkIncompatibility() with an array instead of VersionInfo.
Comment #78
kim.pepperComment #79
andypostGreat work! I looks ready except nits & probably needs test to make sure that deprecated methods still works and throw exceptions
Links needs "See https..." https://www.drupal.org/core/deprecation#how-function
But no dot at the end!
should be a @todo
Comment #80
phenaproximaWe don't need the initial !empty() check. If getVersions() returns an empty array, the foreach will short-circuit and we'll jump straight to the end of the function.
Also, I wonder if we should ever NOT have an 'op' key in $required_version. If there's no operator, shouldn't it just default to = or ==?
Can we add a @see for ::parseDependency()?
Nit: There's an empty blank line here.
"Project" is a Drupalism. Can this be explained a bit in this comment? Also, the comment here mentions $dependency, but in the context of a value object, it's not clear what that is.
I'm on the fence about adding this, although it *does* provide a nice fluent interface...
"Gets the name"...of what? :)
Same here.
TBH, I'm not sure we need setters in this class. I can see no reason these value objects would ever need to be mutated. If this is for the fromArray() method, we still don't need setters; in that method, we'll still be able to set protected properties of the instance directly.
This doc comment should explain what each version looks like.
I don't think $version should be an array of 'op' and 'version' -- this should just accept two arguments, $op and $version, and $op should default to =.
I don't know if we need this; it's only used by the backwards compatibility layer. Maybe we should move this code to another place (like ModuleHandler::parseDependency()).
Can we also add test cases for RC, beta, greater than or equal, and less than?
? Not sure this adds anything.
Comment #81
kim.pepperThanks for the great feedback!
#79
#80
Comment #82
andypostPatch changes naming a bit & some clean-up
not sure that original version still needs to be prefixed with space, we could keep it in BC layer (toArray)
Comment #83
andypostI think we should repurpose related issue to replace
[A-Za-z]with allowed[alpha|beta|rc]as https://www.drupal.org/node/1015226 states about releasesEDIT not sure about deprecated "unstable"
Comment #85
kim.pepperFixes #82 and #83
We still need to add extra test cases.
Comment #86
phenaproximaOh, it's getting close!
$required_version['op'] is always added, so we can remove the isset() call.
Nit: This can use the ?: operator for brevity:
$core_compatibility = $core_compatibility ?: static::getCoreCompatibility();It seems weird to me to pass the parentheses to setOriginalVersion(), since it's just there to support a Drupal-specific UI. IMHO, the parentheses should be added by the UI/calling code; setOriginalVersion() should just take the original version string with no parentheses.
Should be "Can be used in the UI..."
I think the type should be array[], since it's an array of arrays.
Nit: Technically, this method returns static, not $this.
The return type should be array[].
Can we add a @see referring to the $versions property, so it's clear what $versions should contain? Also, the parameter type of $versions should be array[] since it's an array of arrays.
Can we document what we would expect the $version array to look like? Also, this is returning static, not $this.
Should be "...VersionInfo to an array..."
$versionInfo should be $version_info, since our coding standards use $snake_case.
Comment #87
kim.pepperThanks for the review:
We still need the extra test cases as per #80.12
Comment #88
phenaproximaShould be "The array structure may contain the following keys".
Other than that, looks great. RTBC from me once fixed and green on all backends.
Comment #89
kim.pepperFixes #88
Added tests for >=, < , plus beta and rc as per the feedback in #80.12
Comment #90
phenaproximaThis looks awesome. Thanks, @kim.pepper!
Comment #92
alexpottWhy not require $core_compatibility and lose the complexity in dealing with \Drupal in component code?
I think this code should be in the value object.
This is a mutable value object. So it means you can create things that don't make any sense. Also
I think this should be called
createFromArrayto be consistent with thecreatemethod. Also whilst the rest of the value object is fine without testing I think this method might deserve a test since it has logic.Comment #93
alexpottHere's a patch takes a slightly tweaked approach. It introduces an extra value object Dependency which manages all of the dependency related info and moves all of the version stuff into VersionInfo and completely gets rid of VersionChecker. All the value objects are immutable so you can't create things that don't make sense. And the API for checking whether something is compatible with another version string from somewhere is a new
\Drupal\Component\Extension\VersionInfo::isCompatible()method that returns a Boolean.The patch also adds a BC test for
drupal_check_incompatibility()thats passes against HEAD (if you remove the expected deprecations). Regardless of whether we go with this approach we need to continue to have this test.Also we need to add test coverage for:
This code in system_requirements(). The patch in #89 broken this because VersionChecker::checkIncompatibility() trims the brackets and spaces of the version string so
$compatibility = rtrim(substr($compatibility, 2), ')');is not going to work. I've fixed this in the attached patch.Comment #94
alexpottHere's why I think immutability is important. With #89 the following code is possible:
Basically setVersions and setOriginalVersion contradict each other and this returns
?!?!?!
Comment #95
alexpottFixing the one test that is going to fail.
Comment #97
andypostThis separation looks great only
$core_compatibilityby-passing to version info parser looks strangePatch fixes docbocks, removes lazy version info parser
The change in
core/modules/system/src/Form/ModulesListForm.phpfits in 8.7 (string change) - we mostly should escape each var separate, so once we touch this code anyway better to split module & its version varsComment #98
alexpottWe can remove this property from Dependency too then. If we remove the lazy thing. We do lose this slight intended side effect though:
We lose the slight optimization here of not having to parse the version stuff. But also it's interesting that we don't check version info here.
+1 to separating out the variables in the translatable string.
Comment #99
MixologicJust to throw this out there again, I had to redo the version parsing when I built the composer facade, and built a test for it so that it would handle *all* of the oddball edge cases that people have, over the years, put into their info and info.yml files that we see on drupal.org.
http://cgit.drupalcode.org/project_composer/tree/project_composer.module...
The test I wrote for that is here: http://cgit.drupalcode.org/project_composer/tree/project_composer.test#n54
Perhaps some of those cases can be adapted into this class?
Im not sure if we do or do not want to introduce things like adding the ~ and ^ operator or not at this juncture. It should still be BC compatible, just with expanded capability.
Comment #100
andypostI think
$core_compatibilitycould be renamed to$prefixto be less "core" specific and ++ #99@alexpott optimization makes sense, but is only place where version info not used?
Comment #101
larowlanThis is looking good, we need an issue summary (and title) update to reflect the new approach.
Couple of minor observations:
could we initialize this first and avoid the else?
Do we need to mention it? There is no use of it in the method.
This isn't needed - if name isn't present we throw an exception?
Comment #102
mile23+1 on #99.
Comment #103
kim.pepper+1 to new approach. :-)
Comment #104
alexpott#99 feels like great and important follow-up material.
Re the lazy version parsing - here's another use-case #2855026: Installation profiles do not support project:module format for dependencies - still not sure about it. So we can always implement later if we find that the performance optimisation is worth (i'd hazard a guess that it is not).
One thought is that at some point in Drupal's future it'd be great if we offloaded all of this version checking stuff to composer and made it not our problem. But that gets into discussions about composition vs installation that would only side track the benefits of doing this now.
I've added some missing test coverage for version strings with commas. This has been supported forever we just haven't had core test coverage.
Still need to add tests for the update path code.
Comment #105
alexpottAdding the missing test coverage for system_requirements()
Comment #106
alexpottI've started to update the issue summary and whilst doing that it makes me question our class names. I feel that
Dependencyis a good name. It describes that it represents a dependency on a project with a name and might have requirement denoted by an operator + version string. But I feel thatVersionInfois not a great name. I would expect VersionInfo to deal with version strings like8.x-1.1or something and give you the ability to get major, minor or patch number and any modifiers such asbeta. But thats not really what it is doing. It's dealing with an operator + version string and allowing you to check it against another string according to Drupal's specific rules.Given the above I wonder if it should be called
VersionRequirementor maybe evenDrupalVersionRequirementbecause the logic is so Drupal specific. I also think that we might mirror the isCompatible() method up toDependencyas that would allow Dependency to support different version schemes.Going to hold off on more issue summary and change record updates until this discussion is over.
Comment #107
alexpottPatch attached moves the up the isCompatible() check to the Dependency object which feels and looks nice. We still have separate logic between a dependency and version string but dependency no longer emits VersionInfo objects which looks good to me.
Also the patch goes a bit further in the BC layer front by making Dependency object implement deprecated ArrayAccess methods. This should allow us to use them in the dependency graph without breaking BC but also to fully deprecate dependency info being passed around as arrays.
Comment #109
alexpottSome fixes... so atleast some functional tests pass :)
Comment #110
alexpottSome more fixes.
Comment #111
alexpottLol forgot to remove the backward compatibility function added previous that's now no longer necessary because we do the ArrayAccess thing.
Comment #113
alexpottSlight tweak to where some of the BC stuff is happening.
Comment #114
alexpottSo for me the two remaining questions are:
drupal_check_incompatibility()as this is the only non test usage. I kind think so because it's additional API that's completely unnecessary.Comment #115
alexpottDone #114.1 - less new API has to be a good thing.
Comment #116
alexpottRealised that we should support Dependency objects in drupal_check_incompatibility() because people might reasonably be passing things in from
$extension->requireslike core used to do.$extension->requiresnow contains a list of Dependency objects. The good news is that we can revert the method back to how it was and it'll use the ArrayAccess BC layer and work nicely! Added test coverage for this situation.Comment #117
andypost@alexpott it looks awesome!
++ on naming - it needs more opinions
ArrayAccess mthods may use to throws specific exception message for each property
About naming it looks more like
DrupalVersionRequirementand looks like it missing "between" operationI bet Dependency should be full namespace and messages could be more specific on used method to catch usage better
Comment #118
borisson_DrupalVersionRequirementsounds great, another ++ for that.Comment #119
alexpottWhilst reviewing this patch I've discovered a significant bug in the current code both HEAD and with the patch. It's caused by how we prepare the version string for checking compatibility against it.
The suspect code is:
If
$required_file->info['version'])is8.x-8.x-beta1this would mean that we'd check the version using the stringbeta1where the intention is to pass8.x-beta1. A version string of8.x-8.x-beta1might feel unlikely but it is very possible. Panels is on8.x-4.xfor example. I think we should move the$version = str_replace(\Drupal::CORE_COMPATIBILITY . '-', '', $required_file->info['version']);functionality into\Drupal\Component\Extension\DrupalVersionRequirement::isCompatible()so callers don't have to think about this at all.Comment #120
alexpottAddressing #118 and #119.
Should we implement these for full BC? I'm inclined to not because I don't think modules should have been messing with information in ->requires but someone somewhere probably has. The upgrade path would be to replace the entire value object instead of messing with the internals.
Comment #121
alexpottI've updated the change record - https://www.drupal.org/node/2756875
Comment #122
alexpottOpened a followup for #119 - see #3004174: Dependency version checking breaks for contrib versions like 8.x-8.x-rc1
Comment #123
MixologicRe:
DrupalVersionRequurementcomposer semver library calls a version string and an operator aConstraintComment #124
alexpott@Mixologic that sounds exactly like what we want - so maybe
DrupalVersionConstraint. There's part of me that's really tempted to rip the patch apart once more and move\Drupal\Component\Extension\Dependencyto\Drupal\Core\Extension\Dependencyand to create a newVersioningcomponent instead. And then we could call the classDrupalConstraintto reflect that is a Drupal-ism.Comment #126
mile23Looking pretty good. +1 on @mixologic's naming suggestion, so it's a little more like semver.
Versioning is probably a better name for the component. So +1 on #124, except:
Other than the fact it's a Drupalism, nothing currently in the component seems to need anything from core. So I'm inclined to say leave the whole thing in the component, so other tools can parse info files and then use the component to do comparisons the way core does.
Maybe add some comments explaining that we implement \ArrayAccess for BC, see this CR.
Comment #127
alexpottI think #126 actually makes my discussion about the component name more relevant - all the info parsing stuff is in
Drupal\Core\Extensionand ifDependencywas there too we wouldn't have to the the \Drupal::CORE_COMPATIBILITY dance.Comment #128
alexpottHere's the big move around with
Dependencygoing toDrupal\Core\Extensionand a newVersioncomponent.Questions & thoughts that arise from doing this.
\Drupal\Component\Version\Constraint::isCompatible()and\Drupal\Core\Extension\Dependency::isCompatible()should change their name to::isSatisfiedBy()- this is inline with some of Composer's SemVer language.Comment #129
alexpottI went ahead and deprecated
\Drupal\Component\Version\Constraint::toArray()less unnecessary internals exposed in Drupal 9 sounds good to me.And here's the patch.
Comment #130
alexpottOh and now we get the lazy constraint parsing for free.
Comment #131
alexpottForget to address the comment @Mile23 is asking for in #126
I went for Version over Versioning as the new component name because according to PHPStorm's dictionary Versioning is not a word and I couldn't be bothered to argue with it.
Comment #132
alexpottComment #134
MixologicThis is fabulous. Only have one minor api suggestion, but Im not attached to it at all. Consider the following bikesheddish:
I think we might consider making $core_compatibility an optional argument to the constructor, since its not a requirement that the compatibility exists in the version string.
And if the constructor's core_compatibility is optional, so should the call to parseConstraint.
Everything else in there LGTM.
Comment #135
alexpott@Mixologic i'm not sure about that. Especially in the light of #3004174: Dependency version checking breaks for contrib versions like 8.x-8.x-rc1. I think we should require a core compatibility because it is quite entangled with our Drupalised version strings like
8.x-4.1-rc1. Yes the code in parseConstraint will work the same for>8.x-4.1-rc1and>4.1-rc1but that feels kind of by-the-by and an accident of history (I might be wrong).I guess one option would be to make it optional on both the constructor and isCompatible() but that feels icky and something we'd need to sort out now rather than later.
Comment #136
andypostMinor clean-up, I also see core compatibility as long term solution, in d9 we will need to be able to process older versions as well and not clear when contrib will use composer/semver versioning for constraints
Comment #137
mile23+1 on #134 because we'll soon have a moment where someone will install a module under D8 that says
core: 9.xin its info file. Core shouldn't assume it's a D8 extension.This could be a bit of a bikeshed, like @mixologic says, so maybe do a follow-up. We can change the API a little in 8.7.x before 8.7.0 release, right?
Also still wishing
Dependencywas in the component... Though again it might be more obvious why later, and we can do it then.Comment #138
MixologicIf the eventual goal is that the contrib ecosystem will be able to release modules that are d9 compatible before d9 is released, and that those modules are simultaneously compatible with d8 so that people can upgrade to them prior to upgrading to d9, then we'll need to eliminate the idea that the core compatibility is embedded in the version string, and we'll need to ensure that contrib can use semantic versioning for drupal 9.
i.e. we never want to see a contrib module with a
9.x-a.bas its version string. If a contrib module has specific core version requirements, then that has to be expressed in a dependency on the system module, not as a version string bolt on, because we don't have a sane way of expressing 'both 8.x and 9.x' in that string.The only reason we need the core_compatibilty atm is so we can strip it out if it happens to be there. We could be really bold and just get rid of the argument entirely by changing
$p_core = '(?:' . preg_quote($core_compatibility) . '-)?';
to
$p_core = '(?:8.x-)?';
given the assumption that we should not support any 'core compatibility strings' versions beyond 8.x in our info.yml files.
Comment #139
mile23This conversation is exactly why it should be a follow-up. :-)
We already have #1612910: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) and #2313917: Core version key in module's .info.yml doesn't respect core semantic versioning where this conversation is a little stalled and a lot dependent on the current issue. I'm not sure there's a follow-up specifically for whether to deprecate the core compatibility string for D9. #1612910-399: [policy, no patch] Switch to Semantic Versioning for Drupal contrib extensions (modules, themes, etc) and subsequent conversation seems to come closest and is a policy issue.
I think the solution we have here gives us BC, and is flexible enough to mutate a bit in follow-ups. And we can fold in the wisdom from #99: #3004459: [PP-1] Fold in dependency parsing wisdom from project_composer
Comment #140
alexpott@Mixologic yep I totally agree with #138 and #139. One of the reasons I separated the Dependency and Drupal\Component\Version\Constraint logic is that hopefully we'll be able to detect in Dependency when we have a SemVer constraint like
4.5.0-devand not use our constraint logic at all but use Composer\SemVer so we don't have to maintain our own set of versioning tools.Comment #141
larowlanAdding review credits for those who helped shape the patch
Comment #142
larowlanAre we sure we want core in the name here?
should we be using the methods instead of the deprecated array access?
Comment #143
andypostNormalize dot at the end of link for deprecations
@larowlan re #142
1) yes, core components always namespaced by core prefix (check other components)
2) yes we have to, because access to both keys are deprecated so I added expected deprecatations
Meanwhile for some reason tests are passed fine before patch which added 2 expected deprecations to
\Drupal\Tests\Core\Extension\DeprecatedModuleHandlerTest::testDependencyParsing()Very probably
@legacydoc-comment allows to skip not declared expectationshere all expected deprecations should be listed
Comment #144
alexpottAn important point about #142.2 is that the use of the deprecated array access is from a method that is also deprecated. And deprecated code is allowed to use deprecated code :)
Comment #145
alexpott@andypost removing the fullstop from the deprecation message is not really warranted. Currently the examples on https://www.drupal.org/core/deprecation all have a fullstop for what it's worth.
Comment #146
andypost@alexpott does it makes sense to file follow-up to fix it? This fullstop at the end of link really annoying for terminals where "dot" expected as part of URL
Comment #147
alexpott@andypost yeah - I dunno - it'd be great to have a PHPCS rule for exception messages and @trigger_error() so we have consistency. I'm not sure I care about which way we go but the inconsistency feels wrong.
Comment #148
alexpott#2855026: Installation profiles do not support project:module format for dependencies adds some new usages of ModuleHandler::parseDependency(). Removed them.
Comment #149
kim.pepper@larowlan's issues in #142 have all been answered/addressed so back to RTBC!
Comment #150
gábor hojtsyTagging for Drupal 9 as per all the issue this postponed.
Comment #151
eric_a commentedWe need a follow-up issue to add the new component to core/composer.json and to \Drupal\Tests\ComposerIntegrationTest. Or do it right here.
Comment #152
mile23+1 on #151.
Comment #153
eric_a commentedComment #154
eric_a commentedComment #155
MixologicThat was the exact right thing to do. Thanks.
Back to RTBC.
Comment #156
larowlanCommitted 550e914 and pushed to 8.7.x. Thanks!
Published change record
Comment #158
wim leers👏
Comment #160
dpagini commentedThis might not get seen, but I'm having an issue upgrading to 8.7 and I believe it's related to this change. The `system_requirements()` method is getting an old cached version of my modules list, and the module has a `$file->requires` call that is, in 8.7 expected to be a Dependency object, but in my application, it's just an array. I basically bring my current database down against new code, and try to visit the update.php page, and get...
Error: Call to a member function getName() on array in system_requirements() (line 816 of core/modules/system/system.install).Anyone else? Any ideas?