Problem/Motivation

We have reached a point where the current D10 port of Localize will need testing by contributors who do not have access to a local instance, and will gradually need to be demonstrated. Also, developers do need a common point of reference. Thus, we would like to be able to set up a staging instance.

Proposed resolution

Set up a staging instance on the Drupal infrastructure.

Remaining tasks

Determine how to :

  • [x] add entry for the "migrate" database in the "settings.all.php",
  • [x] update project to be Drupal 10 and have all dependencies up to date,
  • [x] write up instructions on what to run (ie: migrations), in which order, etc. as if to set up everything from scratch on a new machine,
  • update it (manually or not?),
  • DA (Drupal Association) - set up such an instance,
  • DA - allow migration of data from the D7 database.
  • DA - integration with the SSO system

Comments

FMB created an issue. See original summary.

fjgarlin’s picture

This is great!!

I can help with the files setup assuming that it'll have the same setup as the newer D10 sites. The needs seem similar to the www port.

I don't see an entry for the "migrate" database in the settings file for the repo. I guess it assumes the same credentials as the "default" database and that the name of the database is "migrate".

It would be great if the above would be added to the "settings.all.php" file and also that Drupal core is brought to Drupal 10 as next steps.

Migrations-wise, I guess once we run them and we swap the sites to the new one, will that be it? Or do they have to be run in any special way?

fjgarlin’s picture

Issue summary: View changes

Updated the IS to add the previous things.

fjgarlin’s picture

Issue summary: View changes
fjgarlin’s picture

Issue summary: View changes
fmb’s picture

Does this need integration with the SSO system?

Eventually, yes.

fjgarlin’s picture

Issue summary: View changes
fmb’s picture

Status: Active » Postponed

Postponing until migrations are definitely up and running.

fmb’s picture

Status: Postponed » Active

Reopening as migrations are nearly ready.

fjgarlin’s picture

I think this falls into the same blocker as the www.drupal.org migration. We need to put the old database into a new server somehow, with an updated DB version, and that needs infrastructure work.

Once this is done, we just need to add a "helm" folder to it and set the migrations to run, but we are not there yet.

I will communicate it with the team so they are aware of localize being almost ready.

fjgarlin’s picture

One of the requirements will be to have Drupal core and the modules in use up to date, and preferably on the latest D10 version.

fjgarlin’s picture

We are meeting later on and I will mention this, but wanted to leave it in the issue.

Steps before going live (some of these are already in the Remaining tasks section)

  • Finish the pending work
  • The DA will need to do a review and add the infra-related files
  • This project needs to be using the new SSO system first, which is not live yet
  • Once the above is good, we should try running through the upgrade using staging’s data.
  • For that, we need to make sure the upgrade steps are documented and work. So we'll need details on how to set it all up step by step, how and when to run the migrations, etc. This could be in the readme file in https://gitlab.com/drupal-infrastructure/sites/localize.drupal.org
fmb’s picture

We discussed today on this matter with fjgarlin and Gábor Hojtsy.

  • As we mainly need this instance for tests, we will not be needing SSO at this point.
  • As versions of MySQL greatly differ, database dumps need to be converted somehow before they can be imported into the instance. That is the reason why we will not be running migrations frequently: only once to begin with.
  • Documentation (README) needs to be enhanced so that sysadmins can know what to do in order to launch a new instance on an actual server, not only locally.
  • This instance might be ready after Drupalcon. In order to share environments during the con with people who cannot or are not used to installing a local environment, we might be using tools like DDEV share (or maybe Drupalpod?).
drumm’s picture

As versions of MySQL greatly differ, database dumps need to be converted somehow before they can be imported into the instance. That is the reason why we will not be running migrations frequently: only once to begin with.

We are doing a general upgrade of MySQL for all Drupal.org production sites in the coming weeks, so this will get better.

This instance might be ready after Drupalcon. In order to share environments during the con with people who cannot or are not used to installing a local environment, we might be using tools like DDEV share (or maybe Drupalpod?).

Please remember to keep the actual data confidential. It is okay to make a public development instance of the site, but do keep it configured as securely as you would a production site.

fmb’s picture

Please remember to keep the actual data confidential. It is okay to make a public development instance of the site, but do keep it configured as securely as you would a production site.

Certainly, we will not grant users more privileged roles than needed to evaluate the new instance as a regular user, thanks for reminding me.

fmb’s picture

Title: Staging instance for D9+ Localize port » Staging instance for D10 Localize port
Issue summary: View changes
sanduhrs’s picture

Issue summary: View changes
fmb’s picture

Issue summary: View changes
teebeecoder’s picture

Assigned: Unassigned » fmb
fmb’s picture

Assigned: fmb » Unassigned

It took longer than expected™, but we have finally finished testing migrations. Drupal core and projects are up to date, and I added some instructions in the README file. We feel the time has come to set up a staging instance. Please tell us if something is missing.

fmb’s picture

Issue summary: View changes
fjgarlin’s picture

Thanks for the work on that.

This is then in the same position as the D10 version of www.drupal.org, it needs:
- New SSO system to be in place
- A way to access the D7 database

Once we have these things, we'd be able to try in a dev/stage environment.

fmb’s picture

We would like to be able to allow testers to work on a common instance. We do not really need SSO right now. How long until you figure out a way to access the D7 database? Can't you work with a dump in the meantime? It is certainly far from being as huge as on drupal.org ^^

fjgarlin’s picture

Do you need the migrations to be running constantly or is it just a one-off?

Would uploading the DB that you have locally (where I assume migrations where tested against) work? You wouldn’t be able to run migrations but would be able to use the site, right?

fmb’s picture

Do you need the migrations to be running constantly or is it just a one-off?

One-off.

Would uploading the DB that you have locally (where I assume migrations where tested against) work? You wouldn’t be able to run migrations but would be able to use the site, right?

Yes, that would do.

drumm’s picture

Yes, this will be a one-off migration with some downtime as the migration runs.

Before staging is built out, we’ll want to make sure the codebase is fully updated for Drupal core, modules, etc.

We recently launched the Drupal 10 version of API.Drupal.org. We’ll want to make sure everything we learned from that is incorporated into the site build.

We'll want to start a checklist for running and launching the migration, so it can be repeated a few times in staging to find issues, and get an idea of how long it will take. What commands are needed to run the migration? What else is needed?

fmb’s picture

@drumm we added some instructions in the README. Is it OK to you?

fjgarlin’s picture

On these instances you should include either settings.production.php or settings.stage.php

Note that those files are automatically included in the right environment. Anything specific to stage or prod should go there, and if it's sensitive it should be a environment variable via "getenv".

Of particular importance is the activation of the "live" config split, since most languages are disabled on dev environments because too many of them dramatically slows down installation.

Is this something that is set up via the previously mentioned settings file for production? How is that expected to work per environment?

Queues are decoupled from cron, so we need to set up: ...

How often do those queues need to run? Are they one-offs or they need regular running?

fmb’s picture

Status: Active » Closed (won't fix)

We ended up setting up a private staging instance.