Updated: Comment #5

Problem/Motivation

Far more core issues are filed against "base system" than anything else -- as of this writing, 900 total open issues, with 70 of them unresolved majors or criticals. The label is so vague, and the category so broad, that it makes it difficult to triage this queue, identify stakeholders, etc.

The maintainers for "base system" in MAINTAINERS.txt are:

Base system
- Károly Négyesi 'chx' http://drupal.org/user/9446
- Damien Tournoud 'DamZ' http://drupal.org/user/22211
- Moshe Weitzman 'moshe weitzman' http://drupal.org/user/23
- Kat Bailey 'katbailey' http://drupal.org/user/172987
- Larry Garfield 'Crell' http://drupal.org/user/26398

All people who deal with low-level plumbing, for sure. But what, in particular? And can these people really provide maintainership for so many issues?

Base system is not:

  • The installer (has a separate entry)
  • Routing (has a separate entry)

Base system might include:

  • Bootstrap
  • The kernel
  • The container
  • Stuff in core/includes that doesn't have its own listing, especially stuff from common.inc and bootstrap.inc.
  • Stuff in Drupal\Core that doesn't have its own listing.
  • Stuff for core/includes, Drupal\Core, or system.module that got misfiled.
  • Stuff people don't know how to file.
  • General requests for new features, policies, etc.

Proposed resolution

Let's refine the "base system" component.

  1. Identify specific sub-categories that could be added as separate components:
    • Look for patterns in the "base system" queue.
    • Identify components in Drupal\Core and core/includes that do not have their own listings, and determine which (if not all) could be added as their own component.
    • Get feedback from "base system" maintainers and core developers.
  2. Add these components to 3060.
  3. Add these components to MAINTANERS.txt, and identify maintainers for them.
  4. Triage the "base system" queue to move relevant issues to new (or existing) components where appropriate.

Queues

All open issues: https://drupal.org/project/issues/drupal?version=8.x&component=base+system
Majors and criticals: https://drupal.org/project/issues/search/drupal?status%5B%5D=1&status%5B...

Files: 
CommentFileSizeAuthor
#43 base-system-2050763-43.patch1.47 KBxjm
PASSED: [[SimpleTest]]: [MySQL] 58,412 pass(es).
[ View ]
#43 base-interdiff.txt722 bytesxjm
#41 base-system-2050763-41.patch1.28 KBxjm
PASSED: [[SimpleTest]]: [MySQL] 58,531 pass(es).
[ View ]
#33 base-system-subcategories.patch993 bytesxjm
PASSED: [[SimpleTest]]: [MySQL] 57,566 pass(es).
[ View ]
#10 2050763-component-defs.patch911 bytesCrell
PASSED: [[SimpleTest]]: [MySQL] 57,214 pass(es).
[ View ]

Comments

xjm’s picture

Issue summary:View changes

Updated issue summary.

Crell’s picture

+1 in concept. I don't quite know what "base system" means. :-) I was added largely because most of the stuff I've been working on for WSCCI touches all the things, but that doesn't mean I'm really a core generalist at the moment.

Crell’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue tags:+Component maintainers
chx’s picture

There's a lot of nimby going on with this one.

xjm’s picture

I think one easy category to separate out is the container, service registration, etc. Should we add "Dependency injection system," with @katbailey and maybe @Crell as maintainers?

xjm’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Another clear separate subsystem is the HTTP stuff.

xjm’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue summary:View changes

Updated issue summary.

Crell’s picture

chx: No NIMBY. Just figuring out how we can scale properly.

xjm: That's probably a good first step. It still doesn't help define what "Base system" means, though, unless the end goal is to eliminate "Base system" as a thing.

xjm’s picture

Well, my primary goal is to define what base system isn't, so that we can divide this queue up into more manageable, logical chunks. I'm hoping whatever's left after we do that first pass or several will help answer the question.

chx’s picture

Crell, I meant that the base system is often nimby itself. Not that this issue is. Ie. people will throw issues in there just so someone else can deal with it.

Crell’s picture

chx: Ah, gotcha. I think you're right, which is why breaking it up better is a good thing. :-)

Crell’s picture

Status:Active» Needs review
StatusFileSize
new911 bytes
PASSED: [[SimpleTest]]: [MySQL] 57,214 pass(es).
[ View ]

Attached patch just splits Kat and I out of "base system" to a new DI section. I'll ping Kat for a +1.

msonnabaum’s picture

Having a "Dependency Injection" system seems really wrong to me. The important bit is part of bootstrap, the rest isn't worth an entry.

chx’s picture

Also.. with all possible humility, why are you leaving me out of that? (Aside from the fact I am on core vacation.) I think I have my hand in the DrupalKernel quite enough....

katbailey’s picture

I have to agree with Mark - the idea of a Dependency Injection system as a separately maintainable system does not make much sense to me.

Damien Tournoud’s picture

To be honest, I don't think we can do much better then "base system". This includes the early bootstrap, the kernel and container, the extension system and a few other related subsystems (secure PHP include support, multisite support via the settings subsystem, the Drupal global class). All of those are strongly related if not strongly interdependent, and this is by design.

Everything else I consider not being part of the "base system", and that includes most of common.inc and the HTTP rendering pipeline.

Crell’s picture

chx: Well, I think that points to the challenge here. Where does "DI system" end and "core bootstrap" end? It's not at all a clear distinction.

Damien: So perhaps we need to split out "bootstrap" from base system, and let base system continue as "the other stuff"? Is that what you're suggesting?

xjm’s picture

@webchick, @msonnabaum, @effulgentsia, @moshe weitzman, and I just went through an exercise to categorize the majors and criticals filed against "base system":
https://docs.google.com/a/acquia.com/spreadsheet/ccc?key=0AusehVccVSq2dF...

It actually appears that a majority of issues in "base system" are mis-filed. We also identified the following new components that we should probably add:

  • http pipeline
  • bootstrap (includes the classloader)
  • module system (module handler)
  • session system
  • library system

So, next steps:

  1. Agree on the list of new components above.
  2. Add those components to the queue and MAINTAINERS.txt
  3. Identify maintainers for them.
  4. Recruit some folks to triage the "base system" queue, using the spreadsheet above as a starting point, and then carrying that same triage on to the normals. (system.module and "other" might need some similar triage.)
Crell’s picture

What exactly is HTTP pipeline, and how does it differ from routing or sessions? (Sessions are cookies, which are are part of HTTP.)

xjm’s picture

@Crell, take a look at the issues in the google doc and how we've categorized them? Probably the best starting point. :)

Crell’s picture

I did, but it didn't really clear it up for me. :-) Hence my question. I know I've used the term "HTTP pipeline" in casual conversation a lot, but that doesn't necessarily provide guidance for people to know where to assign an issue. Eg, what is routing other than "handling of HTTP requests"?

xjm’s picture

So, in my head, the routing system is what Drupal does internally, whereas the HTTP whatever is what Drupal does externally to accept a request and serve a response back.

The session handling seems like its own thing to me. I guess it might fall under HTTP whatever, but it's certainly not routing.

Edit: Someone more knowledgeable than me should probably participate in this discussion instead of me. ;)

msonnabaum’s picture

Yeah, pipeline might not be the right word, but it does seem like there's a category for handling of http requests and responses.

Crell’s picture

So what about "HTTP handling" (which would involve most things relating to HttpFoundation and HttpKernel, including cookies and sessions) and "Routing" (which is specifically things that happen inside the RouterListener and down, which is a fair bit).

(I'm probably a maintainer on both of those at the moment, but they could have different co-maintainers.)

msonnabaum’s picture

I think it may be worth separating sessions. Also, I'd consider content negotiation in http-whatever.

Crell’s picture

Why would sessions, which are just applied cookies and we want to be using HttpFoundation for, be is own component?

(Not that I want to have to deal with sessions, especially given testbot, but I don't get why it's not part of HTTP.)

xjm’s picture

Well. So "sessions" has two parts: The HTTP session (external) and the user session object (internal). I think most of the stuff from our spreadsheet is about the "external" part anyway, though.

We currently have node access as its own component, even though node access is clearly part of node.module, because it's architecturally complex enough to merit its own domain experts.

xjm’s picture

we want to be using HttpFoundation for

Oh. We want to. But are we?

Damien Tournoud’s picture

Session is part of Authentication, which should be part of the Authentication/Authorization subsystem, IMO.

I would like to see Bootstrap and Module merged into a Base system category, too.

xjm’s picture

Ah, +1 for Authentication/Authorization subsystem.

Crell’s picture

Auth/Auth I can see making sense. I'd be against merging Module into base, though. The way I see it, each subsystem should be thought of as reasonably self-contained. That's just good architecture. Modules should be self-contained away from the base "boot up, find controller, return" process. It should not be tightly coupled. (It may be at the moment; I'm fine with setting an aspirational structure for us to drift further toward.)

Damien Tournoud’s picture

@Crell: Early bootstrap, the kernel and container, settings, the multi-site code, the class loader, the extension systems are all coupled by design. All those are part of a single entity, even if of course you can (and should) try to separate the responsibilities inside it cleanly.

If we want a different name then "base system" for those, I would suggest "bootstrap system".

msonnabaum’s picture

Agreed, all that is coupled by design, so it naturally fits in bootstrap. Module's is more of a grey area, but I lean towards it just being part of bootstrap.

xjm’s picture

So we have consensus on:

  • Bootstrap system
  • Authentication and authorization system
  • Asset library system (No one has commented on this, so I assume it's noncontroversial. I added "asset" because I was worried people would confuse "library" with includes generally or vendor.)

But less so for:

  • Module system or module handler system
  • HTTP handling system (or whatever)

I'd prefer both of them to be separate, since it simply makes issues easier to find and understand, and I can see those two as having separate maintainers. The first is pretty clearly defined; the second one is harder to delineate.

So, let's try the attached to start. Please indicate which of you consider yourselves maintainers for what. E.g., I'm assuming katbailey, Crell, chx, and the classloader folks would be under "Bootstrap system" based on the part DamZ and Crell agree on.

xjm’s picture

StatusFileSize
new993 bytes
PASSED: [[SimpleTest]]: [MySQL] 57,566 pass(es).
[ View ]

Actually attaching.

xjm’s picture

Also, for the five current base system maintainers -- if you consider yourselves only focused on one or more of the subsystems we've identified, and not of the catch-all bin, let me know and I'll remove your name from the other part. Thanks!

xjm’s picture

Title:[Policy, no patch] Refine "base system" component» Refine "base system" component

Well, there kinda is a patch.

xjm’s picture

Another reason that "Module system" makes sense as separate to me is that there are parts of module handling that are unrelated to bootstrap: enabling modules, parsing dependencies, and so on.

alexpott’s picture

+1 to clarifying and splitting up the nebulous catch-all that is "base system"

I'm leaning towards DamZ's and msonnabaum's view that module could be included in bootstrap since I think that an essential part of "booting up" is reading system.module.yml and system.theme.yml and doing the right thing. I can see that installing, disabling (may it go away) and uninstalling are a grey area for some, but again, I think these operations are intimately tied to the bootstrap process since any major change in this functionality will affect the bootstrap process too.

Not yet sure what I think about "HTTP handling system (or whatever)"

I'm in agreement with adding these:

  • Asset library system
  • Authentication and authorization system
  • Bootstrap system
Damien Tournoud’s picture

I could see a "Request processing" system as a super set of "HTTP handling". Right now all the requests the poor system is able to handle are HTTP requests (mainly because the Bootstrap system itself is critically broken), but that should not be that way. Console requests, queue processing, etc. should all be handled by the same pipeline (as they are in most mature frameworks).

xjm’s picture

"Request processing" makes sense to me. For now, it would just be HTTP request/response stuff?

I'm increasingly in favor of a separately maintained component for "Extension system" with #2024083: [META] Improve the extension system (modules, profiles, themes) as a goal (probably ultimately for D9 based on the scope of the changes needed).

Crell’s picture

I'm comfortable with Request Processing (which would encompass HttpFoundation/HttpKernel and the stuff we have hanging off of those, basically).

xjm’s picture

StatusFileSize
new1.28 KB
PASSED: [[SimpleTest]]: [MySQL] 58,531 pass(es).
[ View ]

Hopefully the final list. Let's timebox remaining feedback on this issue to three days from now (Aug. 30) so we can move forward on this. Thanks!

Crell’s picture

Does "routing" get absorbed into "request processing"?

Either way, I should probably get moved out of base system then and into request processing.

xjm’s picture

StatusFileSize
new722 bytes
new1.47 KB
PASSED: [[SimpleTest]]: [MySQL] 58,412 pass(es).
[ View ]

Hm, I think they're separate, although the fact that the attached has you as the only maintainer for both is interesting and worth noting.

Let's do this for now and then identify additional maintainers as appropriate, unless there's anyone else who clearly needs to be moved.

tim.plunkett’s picture

Status:Needs review» Reviewed & tested by the community

+1

webchick’s picture

Status:Reviewed & tested by the community» Fixed

Ok, looks like this'll work as a start; we can always refine later on if it's not good enough.

Committed and pushed to 8.x. Thanks!

Status:Fixed» Closed (fixed)
Issue tags:-Component maintainers

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary:View changes

Updated issue summary.

xjm’s picture

Issue summary:View changes
Issue tags:+Component maintainers