Allow profiles to define a base/parent profile [continue of #1356276]

new patch for 9.4.x and 9.5.x

passes all automated tests except 2

oops, almost forgot interdiff

Crediting people who worked on #1356276: Allow profiles to define a base/parent profile. The list was gathered by xjm. See #3291387-6: Unable to add additional comment or changes to issue number 1356276 (5xx - Server Error) for details.

Per the Recipes, Starter Kits, Distributions strategic initiative - see also the original idea issue and a project created for the initiative, complete with documentation and issues - install profiles are proposed to be largely (completely?) replaced by "recipes," which will be composable. Assuming that work goes ahead, this issue will be rendered obsolete. Therefore, tentatively closing as a duplicate.

If anyone feels this issue should instead remain open, rather than immediately commenting here, the most productive approach would probably be to engage with the strategic initiative. Does the concept of recipes as proposed meet the use case(s) for this issue? If not, why not and what changes would be needed?

Hi @nedjo. Yes you are correct that recipes will take over but that is part of D10 I believe and also it does not mean that profiles will become useless. Also when it come to D9, profiles are still very much in use.
In my project, we have a custom parent profile and three more custom sub-profiles of the parent in a multisite setup. For some project work, I added an update hook in the parent profile but when I ran updb, it threw me error that the parent profile is not installed. This scenario can be very likely in other projects as well. And the fix for this issue will solve the mentioned scenario.

Putting this to NW since previous did not pass validation.

So yeah, we're in a challenging situation.

Many sites are running some version of the patch on this issue and so have an interest in continuing to develop the patch until such a time as a replacement is available via recipes.

But, barring some very unlikely change in the Distributions and Recipes initiative, this approach will never be applied to Drupal core. It not really in anyone's interest to keep open an issue that's never going to proceed.

My view is:

• It's fine to continue to post updated versions of the patch here, but
• the status should return to and remain at "Closed (duplicate)", unless
• someone has engaged with the Distributions and Recipes initiative as I suggested in #61 and confirmed that somehow this issue still fits with plans for core.

We need to keep this issue open because our distribution "WxT" which is used by many government departments and ministries for both internet and intranet are dependant on this patch and new core releases tend to break the patch so we need to keep this open and continue rerolling the patch for the foreseeable future. Please do not close.

I'm glad to see that WxT's project page already lists "the tarball distributed here on drupal.org is deprecated".

I'm going to piggyback on this issue since the changes happening with #3274916: Update Documentation about Plan to Sunset D7 Distribution Packaging and the Packaging Allow List in August 2023 will likely impact anyone still applying this patch with the Drupal.org drush make packaging.

#3285191: [meta] Only support Drupal core installs managed by composer describes benefits of the larger shift away from packaging tarballs on Drupal.org.

Correct me if I'm wrong, but this patch isn't needed for sites once their configuration is loaded during the site install process, so sites that are successfully running could remove the patch and work from their (presumed) exported configuration?

@DamienMckenna,
There is one other thing... A profile can contain modules. if a profile has modules the you are inheriting the profile you might also want to inherit those modules.

My understanding is that normally Drupal will look in the following location for extensions (modules/themes) in this priority.

docroot/core/modules
docroot/modules
docroot/profiles//modules
docroot/sites//modules

If an extension exists in more than one location it will be used in the location lower on the list.

When inheriting a base profile the extension search would go in this order
docroot/core/modules
docroot/modules
docroot/profiles//modules
docroot/profiles//modules
docroot/sites//modules

So after the install the base profile would still need to be in play if it contains profile specific modules that don't need to be anywhere else, but may still need to be used by child profiles.

@jrglasgow I used to think being able to have multiple versions of modules "trumping" each other in the hierarchy was a great feature of D7. So much so that I wrote https://www.drupal.org/project/profile_status_check to provide groups using an install profile a UI to see if they were ahead of behind the version included in the install profile. My thinking was that groups could update the version of a module in docroot/modules before it was updated in the profile or stay behind a version if they had added additional modules that weren't ready to be updated.

10 years later, this sounds like madness to me... especially when leveraging Composer in the build process. While it makes sense to define version range support of dependencies different composer.json files, actually having multiple copies of different version of the same code creates far more problems than it solves. It makes leveraging the composer.json files for security and licensing compliance extremely difficult.

Even managing upstream-configuration/composer.json the way Pantheon recommends is challenging because Composer has so many root-only elements.

@kreynen,
I cannot argue with the madness of it all. I have been working on moving all the sites I work on away from using a base profile at all. Since the base profile in question was not providing configuration I have instead moved the modules in the base profile into separate repositories where Composer can manage them individually.

@jrglasgow: You could still use Composer to download the other profile and then use symlinks to copy those profile-locked modules into another part of the codebase where they're accessible. Alternatively, change composer.json's configuration to store the "parent" profiles in modules/contrib and use them from there.

I have created a version of the patch 3266057-6.patch that looks to be working in Drupal 9.4.5.

Agreed with #72; removing credit for the patches that were taking us in the wrong direction and hiding them from the list. Thanks for raising the issue.

xjm, There was some confusion also with which child issue was going to continue the work from 1356276 so I appologize if I misunderstood what @RoSk0 was working on but yes it looks like he's done some good work.

If I hadn't done any work on this issue the WxT distribution would not have been able to upgrade to D93x back last year until someone else made a D93x version of the patch and not all of them work btw. I basically rerolled it myself a few times until I got something that worked for the WxT distribution. I recall having to reroll this patch myself several times between D9.1 and D9.4.

It has been tricky to reroll this monster huge patch , there was a lot of lines of code conflicting with upstream changes even within minor releases.

As far as I'm concerned lots of people have earned credit here and I don't see a reason to remove credit.

@xjm, I think you're underestimating the complexity of this issue by simply "removing credit" for what's mentioned above, this has been a huge community effort just maintaining this patch through the various versions of core and it has been going on since November 29 2011 with over 700 comments and counting.

There's 11 years of work into this issue, I don't think anyone involved deserves credit removal humiliation here. Especially speaking for myself as I recall spending quite a bit of time going through every single line of conflicting code several times over the past few years rather than waiting for someone else to do it.

@RoSk0, here's a straight up reroll of #6 for the current HEAD of D9.4.x, no changes otherwise and includes an interdiff as none was provided by #71.

This patch applies cleanly to 10.0.x and 10.1.x

This is a straight up reroll of #78 with no changes except to fix a few lines related to the __constructor for core/lib/Drupal/Core/Config/ConfigInstaller.php

applying #78 onto D10 caused a reject core/lib/Drupal/Core/Config/ConfigInstaller.php.rej

I did a straight up reroll , there's no need for an interdiff I made no real changes since #78 the conflict was related to a removed deprecation message.

re: #80, @Ankit.Gupta, your patch is incomplete , and you've provided no interdiff to explain the missing 23 kilobytes of patch data, when making a patch for this issue you have to add the new files and THEN do this:
git diff HEAD > D10_patch_name_example.patch

re: #79

Running PHPStan on *all* files.
 ------ ----------------------------------------------------------------------- 
  Line   core/modules/config/tests/src/Functional/ConfigImportBaseInstallProfi  
         leTest.php                                                             
 ------ ----------------------------------------------------------------------- 
  58     Cannot unset offset 'dynamic_page_cache' on array{testing: 0}.         
  90     Cannot unset offset 'syslog' on array{testing_inherited: 0}.           
 ------ ----------------------------------------------------------------------- 


 [ERROR] Found 2 errors                                                         


PHPStan: failed

Looks like D10 is using PHPStan

This file is offending PHPStan
core/modules/config/tests/src/Functional/ConfigImportBaseInstallProfileTest.php

reroll

Interdiff file

@RoSk0, great work, thanks! Ya I saw that bartik entry too I was going to make the change but you beat me to it. Thanks :)

@Ankit.Gupta, your patch is incomplete (broken) because it was incorrectly rolled.

it's quite simple to roll.

apply a patch that needs rerolling (not yours because it's terribly broken)

git apply patchname.patch;
git status . 
#review the changes, look for .orig files, remove those if you're satisfied
#review the changes, look for failures in the applying of the patch, those result in .rej files
#now fix the problems identified in the .rej files and remove them as you complete the fixes
git add foldername1 foldername2 foldername3 filename1 filename2 filename3 #and so on
git status .
#review to make sure you haven't missed anything
#if satisfied now generate the new patch 
git diff HEAD > patchname.patch
#notice the keyword HEAD here, this adds to the patch the new files that the patch adds , so they are not lost like in your broken patch above

to generate an interdiff I'll give you a quick synopsis:
Revert the patches
apply the original patch, repeat the above steps just before git diff HEAD
then reverse the patch, remove any .orig or .rej files, do nothing else
now apply the new patch
now do a full diff, this will be the interdiff.

Closed https://www.drupal.org/project/drupal/issues/2743197 as a duplicate and moved over credit.

Not sure why the Custom Commands Failed on the 9.5.x branch.

"subsubprofile" is failing cspell. If it's super important to be worded that way I would add to the dictionary file

Also the D10 patch looks like it failed to apply.

Drupal 9.5.3 was released.
https://www.drupal.org/project/drupal/releases/9.5.3

#3337327: CSpell dictionary is out of sync
#3336828: File dictionary.txt is not sorted

It would be really great if the issue summary was updated to say that this issue is about maintaining the patch for those that need it but that for the many many reasons explained by @nedjo above and by others on the original issue it will never be a part of core. In fact if you starting a build from scratch today using this patch would not be a good choice because you would be taking on the maintenance and tech debt burden of this issue.

@joseph.olstad thank you. Great additions to the issue summary.

@alexpott, thanks for your great work on core and for the recipes initiative, very ambitious and looking forward to it!

Meanwhile, it would be nice to fix the cspell issue, subsubprofile I believe this should be added to the cspell dictionary because it makes a lot of sense for this patch and should be accepted as a proper terminology unless we go with grandsonprofile which could work also or grandSonProfile or grand_son_profile, not sure if the camelcase or snake case is needed/desired in this case. I'd like to see the tests pass on this for 10.0.0 at least so that we can begin our D10 upgrade with confidence.

EDIT: My apologies. Looks like in #104 patch I have missed out few changes in testing_subsubprofile. Kindly ignore this patch.

I tried re-rolling the patch for Drupal 10. I confirm that this patch applies to 10.0.3 version of Drupal.

But yes this will need more work for the above said comments.

Try to fix CCF #104

#104 has no interdiff yet is 20 kilobytes smaller than #95
#105 has a massive interdiff relative to #104 which has no interdiff, 8 kilobytes smaller than #95, no justification/explanation for what happened between #95 and #104.

Refer to #88 / #78 as a baseline.

@joseph.olstad, I can confirm infact that:
#105 isn't a valid patch if applied on Drupal 10.0.3
#104 doen't work (I have a profile which has Standard as parent and any element of Standard Profile seems to be installed)

Re-rolled the most recent 9.5.x patch after this week's 9.5.4 release. Running the code check locally, I'm still seeing the CSpell `Unknown word (subsubprofile)` failure.

Ran into this on Drupal 9. Guessing it is likely that 10.0.x has a similar problem. If no one has a chance to get to it sooner, I'll try to loop back for the Drupal 10 reroll as well.

Here is 10.0.x reroll

In working with the patch from #108 I found that while the patch applies cleanly but the base/parent profile functionality doesn't seem to be working correctly. I'm seeing errors during install similar to what I would see if the patch wasn't applied.

As a result, I went back to the next oldest 9.5.x patch. I'm attaching a re-roll of that patch. I've also tested install with this patch and am not experiencing the same issues. As a result, I've marked the files from #108 to not display.

Also, thanks for the 10.x patch @nicoschi - I plan on testing that soon.

hiding 104/105

Thanks, Kirill, for the important notice.
Noted; and taken.

It feels that we are working in two tracks.
I agree with you that it is better to have one perfect track.
To continue with your track then.

describe changes for #95

Re-rolled my latest patch from #1356276: Allow profiles to define a base/parent profile
From comment #866 in #1356276
for Drupal 9.5.x

interdiff--1356276-688-9-4-x---3266057-95-9-5-x.txt

Working to detach using the sub profile method, in fever of Drupal Recipes.

Following with #distributions-and-recipes

@RoSk0, thanks :)

The Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Rerolled #117 to 10.2.x

Rerolled #120 to 11.x and replaced deprecated function assertObjectHasAttribute.

I tried out the patch in #120. It applies cleanly on the released version of 10.2, but from my experience it seems like there might be a regression in the re-rolled patch. When installing using a profile that has this base/parent relationship, not all of the dependencies from the parent are being installed.

If I have a chance to dig deeper into this I'll follow up here. But wanted to post a comment in case anyone else runs into something similar.

@brianperry, the changes made in the interdiff look incorrect to me, I'd start by redoing the changes related to the interdiff to make it as similar as possible to patch 117.

https://www.drupal.org/files/issues/2023-11-30/interdiff-117-to-120.txt

I want to depart from this madness completely since I'm on 10.2.2, and there appears to be no patch and also appears it will never be incorporated in core, so I'd rather not deal with the headache of having to keep track of this patch anymore. I have a profile that has the setting of "base profile" set to "standard". How can I go about changing the current custom profile to NOT inherit the base profile? Can I just remove the "base profile" entry and it's good, it will default to inheriting the standard profile modules/themes/etc? Or do I need to go and put those in manually?

This page, https://www.drupal.org/node/1127786#s-details shows the modules for the standard profile but it looks like it was for Drupal 7 and I can't find any information anywhere else as what dependencies are in the standard profile now.

Any progress on the fix for #122?

I also would love to drop dependency on this patch as @jsweetack has said. I inherited a site that has this patch and I have no idea why it needs it in the first place.

Since we are 6 months since #120 and 10.3 is in RC, am I correct to assume that the only real way forward is to migrate to a new Drupal install without this dependency?

PK2

Rerolled #121 to 10.3.x

Same as comment #122. Patch applies cleanly for Drupal 10.2.6

But the issue persists.

Just for kicks I tried upgrading to Drupal 10.3.1 and applying patch from #129. The issue persists but the error I'm getting is with less unmet dependencies as before.

Anyone know how to fix this?

Patch from #129 applies on 10.3.1

But getting warning while I navigate to Add Content:
Warning: Undefined property: Drupal\Core\Extension\Extension::$ancestors in Drupal\Core\Extension\ProfileExtensionList->getAncestors()

Added the below fix on top of the patch and generated a new one.

diff --git a/core/lib/Drupal/Core/Extension/ProfileExtensionList.php b/core/lib/Drupal/Core/Extension/ProfileExtensionList.php
index cbaae106d3..a211365966 100644
--- a/core/lib/Drupal/Core/Extension/ProfileExtensionList.php
+++ b/core/lib/Drupal/Core/Extension/ProfileExtensionList.php
@@ -62,8 +62,10 @@ public function getAncestors($profile = NULL) {
 
     $extension = $this->get($profile);
 
-    foreach ($extension->ancestors as $ancestor) {
-      $ancestors[$ancestor] = $this->get($ancestor);
+    if (!empty($extension->ancestors)) {
+      foreach ($extension->ancestors as $ancestor) {
+        $ancestors[$ancestor] = $this->get($ancestor);
+      }
     }
     $ancestors[$profile] = $extension;

We were able to fix this by moving the config Yaml files in the child profile that were throwing the error to the /config/optional directory.

Just thought to mention this if anybody faces a similar issue.

Do we have patch for 10.3.5 ?

Patch for 10.3.9

Rerolled for 10.4.4

Fix embarrassing copy pasta mistake with last patch.

Is there a realistic chance that this will somehow make it into the core?

No. The issue's description was updated to reflect that...

This issue is about maintaining the profiles patch for those that need it *but* that for many reasons explained by nedjo below and others in the parent issue profiles defining a base parent profile will never be a part of core. Click to find out more about the Recipes, Starter Kits, Distributions strategic initiative,

Distributions are dead. I say this with a certain amount of authority after creating one of the first distribution projects on Drupal.org (2009), announcing Distributions at DrupalCon SF (2010), managing the Packaging Allow List for many years (August 2011-2023), creating a PoC of Unamni as a Subprofile using this patch (2018) and starting the process to #3266927: End Install Profile Packaging on Drupal.org in August 2023.

Distributions are dead. Long live Install Profiles (and Recipes)!

Edit: If there are use cases that this patch supports that a more robust Install Profile + Recipes does not, I would love to hear about those... but this issue thread probably isn't the best place to have that conversation. Maybe https://www.drupal.org/project/distributions_recipes or, if you are a Mastodon user, ping me @kreynen@fosstodon.org.

Use explicit nullable type to prevent deprecation errors with PHP 8.4

Hello,

Anyone working on this patch for Drupal 11 support?

As pointed out in #140, recipes are there now. Thus, I think this (long-time open) issue is a too massive change that's not worth it doing any more, i.e. I think it would make sense to mark it as won't fix.

@fago, this issue is just sticking around for the few people using it still, but anyone relying on this should be moving to recipes or something else as mentioned since this will not ever be merged and is very much use at your own risk.
Maybe postponed is a better status so people don't think there is work to be done.

I'm hesitant to close it without a fair amount of notice since it seems to be understood that it won't be merged, but will be open for people to collaborate.

Maybe closed won't fix is ok except it just makes it far harder to find.

Patch for Drupal 11.2.6

Please test.

@bala_28 Seems like patch remove $parser and then when DB is not installed and `/core/install.php` is called, I get error:

Error: Call to a member function parse() on null in install_profile_info() (line 796 of core/includes/install.inc).
install_profile_info() (Line: 1294)
_install_select_profile() (Line: 465)
install_begin_request() (Line: 119)
install_drupal() (Line: 53)

Yes @havran. Missed variable is updated in the new patch.

Updated patch for Drupal 11.3.1.

Needs testing.

I work with some configurations inside profile and seems when sub-profile is used and I want revert some configuration from inside profile, this configuration is skiped and original module configuration is applied. Github Copilot try fix that and create this patch:

diff --git a/core/lib/Drupal/Core/Config/ExtensionInstallStorage.php b/core/lib/Drupal/Core/Config/ExtensionInstallStorage.php
--- a/core/lib/Drupal/Core/Config/ExtensionInstallStorage.php
+++ b/core/lib/Drupal/Core/Config/ExtensionInstallStorage.php
@@ -133,8 +133,34 @@
           if (!isset($profile_list)) {
             $profile_list = $listing->scan('profile');
           }
+          // Collect config from all ancestor profiles (base profiles) as well
+          // as the active profile. This supports sub-profiles where the base
+          // profile provides config that should be available for revert.
+          $profiles_to_scan = [];
           if (isset($profile_list[$this->installProfile])) {
-            $profile_folders = $this->getComponentNames([$profile_list[$this->installProfile]]);
+            try {
+              /** @var \Drupal\Core\Extension\ProfileExtensionList $profile_extension_list */
+              $profile_extension_list = \Drupal::service('extension.list.profile');
+              $ancestors = $profile_extension_list->getAncestors($this->installProfile);
+              foreach ($ancestors as $ancestor_name => $ancestor_extension) {
+                if ($ancestor_name !== $this->installProfile && isset($profile_list[$ancestor_name])) {
+                  $profiles_to_scan[$ancestor_name] = $profile_list[$ancestor_name];
+                }
+              }
+            }
+            catch (\Exception $e) {
+              // If the service container is not available yet (early install/bootstrap)
+              // or getAncestors() fails, ancestor profiles will be skipped; the active
+              // profile is always added below.
+            }
+            // Always add the active install profile last so its config takes
+            // precedence over any ancestor (base) profile config.
+            $profiles_to_scan[$this->installProfile] = $profile_list[$this->installProfile];
+          }
+          // Scan profiles from least specific (base) to most specific (active)
+          // so that the most specific profile's config takes priority.
+          foreach ($profiles_to_scan as $profile_name => $profile_extension) {
+            $profile_folders = $this->getComponentNames([$profile_extension]);
             $this->folders = $profile_folders + $this->folders;
           }
         }

I think something similar missing in sub-profiles patch.

I use now Drupal core 11.3.5 with 149 subprofile patch. Then I apply also this patch and now my updates with configurations revert() works.

Btw. patch is made by Github Copilot (Claude Opus 4.6) and finalize by opencode (also with model Claude Opus 4.6 with thinking enabled).