Pantheon (and acquia) require you to include the settings.php in the repo. It would be better if fetcher just created a fetcher.settings.php and then added an include for it to settings.php if one didn't exist already.

# Grab the fetcher settings if available;
556 is_file(dirname(__FILE__) . '/fetcher.settings.php') AND require_once('fetcher.settings.php');

Also maybe name it fetcher.dont_commit.settings.php to get the point across about not adding that file to the repo?

Other benefit is that it should be easier for others to start using fetcher with existing sites. :)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

tizzo’s picture

Actually that would leave you with uncomitted changes in settings.php.

Tbe current recommended work around for acquia/pantheon sites is to create a site folder for the local domain. This prevents git from wanting to checkin the settings.php changes all the time and totlaly works. Maybe we should create a host specific site folder if we see a settings.php in git?

tizzo’s picture

I should also say you can have it create a host specifc folder by using the `--site="yourdomain.local" param.

frankcarey’s picture

That workaround seems really kludgy to me though.

Currently the entire settings.php file is uncommitted, so I'm not sure how critical that would be, and the idea is that you would commit the change to settings.php once it's made and fetcher would never need to add it again. In fact, you could replace the need for the site-settings.php file and just use the settings.php file for that (as almost everyone does). Some organizations have rather complex / refined settings.php files, so to use it with fetcher w/o extra work, you have to move those files into a new file called site-settings.php and let fetcher create the settings.php file for you... a process that isn't obvious for someone new. Then if you want to deploy that code using your own scripts or another vendor, you're missing your custom settings.php file.

Seems like fetcher should be less obtrusive.

How about just creating a fetcher setting for what to name the settings file that fetcher creates? Then, a site's yaml file could make the call?

tizzo’s picture

I think you have a point about fetcher being overly opinionated right now. The current approach is pretty clean if you're not on pantheon or acquia, but many (most?) people probably are on one or the other.

There are two kinds of settings, those specific to the environment and those that are site wide. The former are things like database credentials, solr environment urls, and redis key prefixes. These are settings that are not worth persisting and that leak sensitive data if they're added to VCS. The latter are things like string overrides, domain suite configurations, and fastpath settings that should be common to all environments. There are also often some in-between settings that you want persisted but that vary from environment to environment (setting an environment indicator or disabling mail when not in prod).

The challenge is keeping one set of stuff out of VCS while letting the other set in (and intelligently switching between them). Some settings should be allowed to be somewhat ephemeral and outside vcs (acquia and pantheon provide many of these for you in their environments with magic includes), others permanent but switched on by environment, and still others permanent and always on.

Fetcher's current approach is automatically generating the database credentials and then detecting the site-settings.php in your site folder and requiring that file if it's found. The name of that file is configurable (you can find all of the current configuration keys in the KEYS.txt file in the repo) but the behavior is fixed. Fetcher also sets a variable indicating the environment *before* the site-settings.php include allowing you to perform switching inside site-settings.php.

As we've said, Acquia and pantheon unfortunately require you to check in settings.php which has always seemed like an anti-pattern to me. Many companies avoid checking in settings.php, some using a site-settings.php equivalent and some advocating an [environment-name].settings.php pattern. A few inverse the relationship and check in a settings.php with the shared stuff and then require an ephemeral file that's kept out of VCS from settings.php. Unfortunately I don't think there's a clear pattern or best practice.

Pantheon checks in a completely empty settings.php file (having it there it's required for some reason but I can't remember what that reason is) . Acquia checks in a snippet like the following:

  if (file_exists('/var/www/site-php')) {
    require('/var/www/site-php/sitename/sitename-settings.inc');
  }

That means we need to smuggle our local database creds into that file or avoid using the file. Using a hostname site folder totally solves the problem and leverages a core feature so I'm not sure why that seems like a kludge, though it definitely makes it hard to share the config that *does* belong checked in between environments.

So to play nicely with the empty settings.php provided by acquia or pantheon, maybe you're right - we need to add something to the bottom of settings.php (probably a conditional require as above to pull in an ephemeral file that's *not* added to VCS), or conditionally define the credentials. The second one still seems wrong but so does modifying the settings.php file. I've been hesitant to have fetcher leave a mark on your repository, but maybe that would be less opinionated than the way we're doing it now. We could also detect that they have a .gitignore file and add fetcher.settings.php to it. So fetcher.settings.php would not replace site-settings.php, you'd instead be either committing that stuff to settings.php instead or you might want the environment specific includes I mention above.

I'll give it some thought but I think I like this idea.

With any of these approaches the required activities should be moved into tasks in the task stack so that they could be easily disabled or swapped out at run time for other workflows. We now have a decent architecture to allow that kind of thing.

frankcarey’s picture

..not sure why that seems like a kludge, though it definitely makes it hard to share the config that *does* belong checked in between environments.

Yeah, it's certainly that, and that many people are used to using sites/default so it's a clever trick, but adds a confusing element that isn't obvious to someone new to fetcher (or even me who's been using it a while)

Many companies avoid checking in settings.php, some using a site-settings.php equivalent and some advocating an [environment-name].settings.php pattern. A few inverse the relationship and check in a settings.php with the shared stuff and then require an ephemeral file that's kept out of VCS from settings.php. Unfortunately I don't think there's a clear pattern or best practice.

Yup, I've used both variations, but there IS at least one common and necessary denominator of either approach, settings.php. It really doesn't matter which method is chosen because a settings.php MUST be there for anything to work (whether it's ephemeral or not). So fetcher could take these steps in it's ensure tasks for example..

1) Ensure that a settings.php exists. If not, create one by copying the default.settings.php file.

2) Check if $database['default'] is set and not empty OR if the fetcher environment variable is not set. If it is, create a fetcher.settings.php file (with JUST the necessary
$database['default'] settings AND/OR the fetcher environment variable.)

3) Check if $database['default'] is set now. If not, add the include snippet to the end of the settings.php file.

In this way, fetcher only does what's necessary and nothing more to get the site up and running, and however other people want to deal with their settings.php is totally up to them and not fetcher's concern other than filling in pieces that it finds missing.

So fetcher.settings.php would not replace site-settings.php, you'd instead be either committing that stuff to settings.php instead or you might want the environment specific includes I mention above.

Yup, like I said above:

In fact, you could replace the need for the site-settings.php file and just use the settings.php file for that (as almost everyone does).

If they were using an ephemeral settings.php and adding an include manually before when setting up a site. They can still totally do it manually each site setup like they did before, but fetcher will at least create the settings.php file for them first... OR they can configure the fetcher settings to include the file directly... OR they can create a site-settings.php file for that stuff... OR they can just check-in their settings.php file and put code above or below our snippet depending on the order they want to do things and if they need the environment variable or not. If they have their own custom deployment scripts, then fetcher should still be able to play along nicely.

The other nice thing about this method is that I'm pretty sure it would be 100% compatible with both existing fetcher sites and ones being added to fetcher because if they don't have a settings.php file, fetcher will create one that includes fetcher.settings.php file in the process. If they are an acquia/pantheon site, fetcher will add the snippet to the end (which will show up on your next git diff, thus highlighting the change!) If you've checked in your entire settings.php file with db credentials that you share locally and on the server, you are probably a crazy person, but whatever, fetcher will play along with your madness and not overwrite your db credentials in fetcher.settings.php.

frankcarey’s picture

How about fetcher.temporary.settings.php ?

I like "fetcher" because it gives you a hint about how the hell it was created, and "temporary" clarifies that you probably want to not commit it to VCS. and then "settings.php" so you also know what it's related to.

Also how about being even less obtrusive and making the default snippet be something like..

  # Grab the fetcher settings if available;
  if (is_file(dirname(__FILE__) . '/fetcher.temporary.settings.php') {
    require_once('fetcher.temporary.settings.php');
    $database['default'] = (!empty($database['default']) ? $database['default'] : $fetcher_temporary_settings['database']['default'];
  }

.. so fetcher isn't even overriding the database creds directly, but allows you to use the variables as you wish if you want to modify the snippet to your own needs / logic.

grndlvl’s picture

Status: Active » Needs review
FileSize
4.48 KB

First pass at adding a fetcher specific settings.php file. We automatically create a 'settings.php' file when one doesn't already exists that includes the 'fetcher.dont_commit.settings.php'.

grndlvl’s picture

Assigned: Unassigned » grndlvl

After discussing with @tizzo I'll be making some additional changes.

1) Add fetcher.settings.php to working directory '/var/www/SITE'
2) Append fetcher.settings.php to settings.php when not included.

grndlvl’s picture

Status: Needs review » Needs work
grndlvl’s picture

Status: Needs work » Needs review
FileSize
0 bytes

We have moved the fetcher settings to the working directory with file name 'fetcher.settings.php'.

We also now append to the settings.php the include to the fetcher.settings.php if it doesn't exist in the settings.php already.