Original Issue from Ergonlogic

In various issues, we've begun discussing API-breaking changes. So let's start planning for Aegir 4.x. As much as AegirNG remains of interest, I think we should make it a "child" project, within the larger Aegir Project, and pursue it in parallel.

META Issue: AegirNG Architecture Discussion

ergonlogic has posted a work-in-progress codebase and an architecture diagram and outline of "AegirNG" on GitLab: https://gitlab.com/aegir/aegir/wikis/architecture

As this is a GitLab Wiki, there is no place to discuss this directly on their site.

Please use this issue to discuss all future versions of Aegir 4.x and beyond.

Comments

ergonlogic created an issue. See original summary.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented
Project: Hosting » Hostmaster (Aegir)
Version: 7.x-4.x-dev » 7.x-3.x-dev

Actually, this should be in the hostmaster queue...

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented

I cannot seem to create a 7.x-4.x dev release, despite pushing to a 7.x-4.x branch. This is thus blocking me from assigning this and other issues to the proper version. Anyone have any ideas?

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented
Version: 7.x-3.x-dev » 7.x-4.x-dev

Actually, the 7.x-4.x branch had already been created, but a change in d.o made it necessary to edit it, and click the "Show in project’s download table when branch is supported" checkbox.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented

I'm consider adding a 7.x-5.x branch to hostmaster, so that we can keep the 7.x-4.x branch for issues we're actually planning to tackle in the short- or medium-term. This'd also allow us to clear the 7.x-3.x branch of most feature requests. Thoughts?

colan’s picture

colan
he/him
Primary language English
Location Toronto 🇨🇦
CreditAttribution: colan at Colan Schwartz Consulting commented

Why not just keep short term issues in in 3? 3.5, 3.6, etc... Unless you want to have short, medium and long-term branches.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented

The 7.x-3.x branch is currently considered stable, and so should generally only be open to bug and security fixes. Since we don't have semantic versioning on d.o, we can accept API additions within a major version. The issues I'm suggesting be flagged under the 7.x-4.x-dev version would entail non-backwards compatible API changes.

However, rather than just shift all feature requests to the 7.x-4.x-dev version, I'd prefer to keep it focused on active issues that we realistically plan to include in Aegir 4. We have a number of postponed or otherwise idle feature requests that I'd rather not have clutter up the queue. We should just close them as won't fix, but that seems rather final. Putting them under version 7.x-5.x-dev and as status postponed seems most accurate.

colan’s picture

colan
he/him
Primary language English
Location Toronto 🇨🇦
CreditAttribution: colan at Colan Schwartz Consulting commented

I still don't really understand the point of having both 4.x and 5.x branch. Don't we only need one branch for non-API-breaking changes (3.x), and one for API-breaking changes (4.x)? If you're thinking 5.x is for pie-in-the-sky AegirNG things we don't want to put in 4.x, why not? If someone wants to work on these things, and get them into 4.x, what would be the problem? If they don't actually get done before 4.0, we could just change the version then.

To have a clear list of items we know we want to get into 4, it may make more sense to use issue tags like "Planned for version 4". We wouldn't have to mark things Postponed if they're not on that list. (I'm not a fan of that status for this purpose anyway as I feel that it inhibits folks from submitting patches.)

I think having an additional branch would add more confusion than value, and it would be one more thing to keep up-to-date.

Assuming we're dealing with only 2 active branches (3.x and 4.x), I see a couple of options:

  1. Keep committing non-API-breaking changes to 3.x, and then periodically merge 3.x into 4.x.
  2. Merge 3.x into 4.x now, and then stop allowing commits into 3.x until they've made it into 4.x. This would avoid periodic branch merges, but would slow the process.

In either case, API-breaking changes would go into 4.x only.

I may, however, be missing something as I'm not sure what was done historically.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented

Considering the volume of stagnant feature requests, I'd rather not have to push them all into 4.x and then again into 5.x, if there are no plans to work on them for 4.x. But I realize that's non-standard, and motivated mostly out of laziness :p . Anyway, I'm fine with the plan outlined in #8.

That said, I think we should consider closing some feature requests as "won't fix". We've historically just left these open, in case anyone ever did decide to tackle them. This has led to a cluttered and hard-to-manage issue queue. Alternatively, were we to migrate to GitLab or Github, this problem will likely largely take care of itself.

colan’s picture

colan
he/him
Primary language English
Location Toronto 🇨🇦
CreditAttribution: colan at Colan Schwartz Consulting commented

For old requests that aren't getting done, I'm not suggesting we change the version pointer for all of them whenever there's a new major release. Just leave them as they are. But we can change the status to the new "Closed (outdated)" if the version is no longer supported.

For example, when 4.0 materializes (or whenever 2.x is no longer supported), we can update the status on all open 2.x issues with "Closed (outdated)". I'm sure there's a drush command / script out there to do this. If someone really wants to work on one of these, he/she can reopen it in the latest version with a patch.

Any thoughts on (1) vs. (2) in #8 then?

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented

Let's stay with #1 ("Keep committing non-API-breaking changes to 3.x, and then periodically merge 3.x into 4.x.") for now. We can switch to #2("Merge 3.x into 4.x now, and then stop allowing commits into 3.x until they've made it into 4.x.") once we're preparing for an alpha release of 4.0.

Jon Pugh’s picture

Jon Pugh
he/him
Primary language English
Location Newburgh, NY
CreditAttribution: Jon Pugh at OpenDevShop Inc (CLOSED) commented
Title: META: Aegir 4 » META: Aegir NG Architecture
Priority: Normal » Major
Issue summary: View changes

Rejuvenating this issue for AegirNG discussion. See https://gitlab.com/aegir/aegir/wikis/architecture

Jon Pugh’s picture

Jon Pugh
he/him
Primary language English
Location Newburgh, NY
CreditAttribution: Jon Pugh at OpenDevShop Inc (CLOSED) commented
Issue summary: View changes
Jon Pugh’s picture

Jon Pugh
he/him
Primary language English
Location Newburgh, NY
CreditAttribution: Jon Pugh at OpenDevShop Inc (CLOSED) commented

Chat log pointer with a lot of our discussion: https://hefring.mig5.net/bot/log/aegir/2017-10-05#T659712

xmacinfo’s picture

xmacinfo
he/him
Primary language French
Location Canada
CreditAttribution: xmacinfo at xMac info commented

Is there a roadmap or feature list for Ægir NG? Or is it still too early to create that type of document?

colan’s picture

colan
he/him
Primary language English
Location Toronto 🇨🇦
CreditAttribution: colan at Colan Schwartz Consulting commented

See the board at https://gitlab.com/aegir/aegir/boards. I believe that's all we have so far.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
CreditAttribution: ergonlogic commented

The initial 5.x architecture has mostly been implemented, though it has evolved somewhat along the way. Here are a couple of sequence diagrams that describe the current front-end-backend architecture.

Our overriding goal for 5.x has been to simplify the DX for adding, altering and overriding workflows. That is, we've been concentrating less on developing Drupal life-cycle operations, and more on making it easy to develop them.

As it stands, we only have very basic operations in place that allow:

  • deploying, verifying and deleting D7 platforms, and
  • installing, verifying and deleting D7 sites.

However, each of those operations is made up of atomic tasks, such as "write a vhost" and "provision a database". These are re-usable in other operations, such as cloning or upgrading D7 sites, as well as other applications, such as D8, WP, etc. So, adding new operations and supported applications is fairly straight-forward.

Tasks are a combination of:

  • fieldable "Task" entities in the front-end, and
  • simple Ansible roles in the backend.

Tasks are easily created and configured via the UI and bundled into operations (also entities). Whenever an operation is dispatched, all the fields attached to its tasks are passed to the backend. The default (and currently only) backend passes those field values into Ansible roles that map one-to-one with the front-end tasks. These, in turn, do the "heavy lifting" that we used to handle ourselves in Drush/Provision. Because the scope of these Ansible roles is limited to a single atomic task, writing them is pretty straight-forward too.

We'll be updating and fleshing out the new docs to cover this and other components over the coming weeks.