Continuing from the work below, the following tasks still remain:

1) Put the DB manager itself in the DIC. (Likely requires refactoring the Database class to be an instantiated object, not a static class.)
2) Put all possible DB connections in the DIC. Probably keep "database" and "database.slave" for the default DB, in addition to whatever else gets added.
3) Move db_*() to bootstrap.inc, and have them wrap drupal_container().
4) Remove db_set_active() and Database::setActiveConnection().

Original issue

The database service was originally included in the bundles patch, but I think it got inadvertently removed when the compilation stuff got pulled out. There are now a couple of issues adding this back in because they need it, e.g. #1606794: Implement new routing system and #1269742: Make path lookup code into a pluggable class.
Also the cmi stuff should be using this. Patch forthcoming...

CommentFileSizeAuthor
#30 1759152-30-db-dic.patch1.94 KBAnonymous (not verified)
#11 1759152-11-kill-db_set_active.patch2.1 KBAnonymous (not verified)
#1 1759152.database-service.patch1.89 KBkatbailey
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

katbailey’s picture

Status: Active » Needs review
FileSize
1.89 KB
RobLoach’s picture

Status: Needs review » Reviewed & tested by the community
Crell’s picture

Status: Reviewed & tested by the community » Needs review

Crell-Approved.

Crell’s picture

Status: Needs review » Reviewed & tested by the community

Erp.

moshe weitzman’s picture

Are we just moving around code here? The diff looks identical on + and -.

katbailey’s picture

@moshe, yeah it certainly looks like that but the code where it's being removed from is actually surrounded by /* and */ ;-)

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Seems harmless enough...

Committed and pushed to 8.x. Thanks!

catch’s picture

Status: Fixed » Active

We're not actually using this anywhere yet. But what happens if I call db_set_active() before calling some methods from a class which has had the database connection injected via the DIC? Will it use the active connection or the original one?

Crell’s picture

Um. I don't think the DIC is compatible with db_set_active(), honestly. I think what happens is whichever connection is active when you first call the DB out of the DIC will be what gets used for the database and database.slave keys forevermore.

Can we just get rid of db_set_active()? I wanted t do that in Drupal 7, too. It's an ugly legacy approach.

catch’s picture

We can get rid of it, I just don't think we can keep both, which is why I re-opened this ;)

Anonymous’s picture

Status: Active » Needs review
FileSize
2.1 KB

i like the idea of killing db_set_active() so much i rolled a patch for it :-)

RobLoach’s picture

Status: Needs review » Reviewed & tested by the community

++

webchick’s picture

Status: Reviewed & tested by the community » Needs review

Uh. What the heck? db_set_active() is an incredibly useful function when pulling in results from external databases. Why would we remove this feature?

Crell’s picture

Because db_set_active() is toggling global state, which is an increasingly bad idea that is going to break more things as we move forward. Also, you shouldn't be using it anyway even now; you should be using Database::getConnection() to get the connection you want and then calling ->query() et all against it directly.

webchick’s picture

Can we not simply change the innards of db_set_active() so it just does that?

webchick’s picture

Also, I don't get this:

"Because db_set_active() is toggling global state"

It's simply calling Database::setActiveConnection(), same as what the patch is replacing calls to db_set_active() with. There's no global state involved..?

Crell’s picture

The whole point of db_set_active() is to change a global static variable. That is its entire reason to exist. Messing with a global static and using the DIC are not compatible approaches. We need to pick one. We pick the DIC.

Anonymous’s picture

Status: Needs review » Needs work
chx’s picture

What people want to say is that the task here is to remove setActiveConnection itself and db_set_active too. There are extremely few places left, just kill it.

chx’s picture

The following two code are equivalent:

db_set_active('students')
db_query($query)
db_set_active()
Database::getConnection('default', 'students')->query($query);

Note: Views already does the latter.

There's another use case where you want to trick an existing module just using db_query to use another database. We need to make sure those wrappers are doing drupal_container()->get('database') and then instead of the superb rare db_set_active one can just change the service in the DIC.

catch’s picture

Yep that's what I was after.

One more question, but I don't think we need to do it here, would be good to discuss in a follow-up though.

Why are we hard-coding the default and slave connections here? It feels like it should be OK to loop through the $databases global and register each available connection as a service?

There's another use case where you want to trick an existing module just using db_query to use another database. We need to make sure those wrappers are doing drupal_container()->get('database') and then instead of the superb rare db_set_active one can just change the service in the DIC.

Yep I agree with this too, that's the main use-case for db_set_active() that's not covered by getConnection but since we can do that with the DIC it should be fine.

Crell’s picture

Component: wscci » database system
Issue tags: -WSCCI +Dependency Injection (DI)

Since we've already discussed moving the db_* functions out of database.inc, we could move them to bootstrap.inc and replace their guts with container()->get('database') (or whatever). Then they become dumb wrappers for the DIC just like a number of other functions have become. We then deprecate them when we're able to, same as everything else.

catch: Eh, mostly because we went with the simple-for-now approach. I still want to refactor the Database class into a DatabasManager object, not a static class, which would then be more flexible. I'm fine with improving the service keys, and then either exposing database.manager as a service and/or iterating over known connections and creating a database.$key.$target service for each. Actually the latter is probably best, as then we can always change the connection sent to a service by messing with the container.

So it sounds like the remaining tasks for this issue are:

1) Put the DB manager itself in the DIC. (Likely requires making that an object, not a static class.)
2) Put all possible DB connections in the DIC. Probably keep "database" and "database.slave" for the default DB, in addition to whatever else gets added.
3) Move db_*() to bootstrap.inc, and have them wrap drupal_container().
4) Remove db_set_active() and Database::setActiveConnection().

Sound like a plan?

Also refiling since this issue is no longer WSCCI related now that the main connection is in there.

catch’s picture

That sounds good to me. If it's easier we could close this and start a new issue as a followup? But since the patch was about 10 lines in the first place it seemed OK to re-open in this case.

Crell’s picture

Updated the summary accordingly.

Anonymous’s picture

#22 looks good to me. i was speculating yesterday in #drupal-contribute about needing to put connections in the DIC and kill Database - i think that's the first time i've guessed right about this stuff. mostly i just guess wrong.

catch’s picture

Priority: Normal » Major

Bumping to major since this blocks #1764474: Make Cache interface and backends use the DIC and nearly every other move x to the DIC issue.

Crell’s picture

beejeebus, if you want to have a crack at this let me know and I'll try to get you up to speed on anything you need for part 1. (The rest should be fairly perfunctory.)

Anonymous’s picture

now i re-read #22, i'm unsure about why we'd even need a db connection manager if we add all the connections to the DIC.

if all the connections are registered, and db_* functions just wrap the DIC, and we get rid of the notion of 'active', what does the manager do now?

Crell’s picture

That's where lazy instantiation of connections from info arrays happens, if nothing else. Plus, if you want to mess with multiple database connections at once (migration and custom code?) it may be easier to just ask for the manager directly and get the connections yourself, since then you get to define which keys you want rather than relying on the DIC to get it right. (The DIC configuration will only be right 95% of the time.)

Anonymous’s picture

Status: Needs work » Needs review
FileSize
1.94 KB

so here's the dumbest possible code to add all registered connections to the DIC.

not sure if we want to do this in stages.

chx’s picture

That's good, covers the second, rarer use case of #20 can we eliminate the first use there, completely killing db_set_active and setActiveConnection as well. This would drastically simplify the Database class as well.

Edit: erm, lays the groundwork for the second use case in #20 doesn't completely do it. Yet.

sun’s picture

Don't want to disrupt the party, but the original/committed patch in #1 makes the database service available in the kernel only.

The database service has to be available outside of the HttpKernel in early bootstrap already. There's not necessarily a HTTP request/kernel involved.

We need to move the service registration into drupal_container().

Anonymous’s picture

yeah, I have similar concerns about at what point in request we add this to the DIC, but I don't have anything intelligent to add.

Crell’s picture

Erf. Anything that happens pre-kernel needs to just go away. We've already reached that conclusion, just haven't gotten around to doing it yet.

Rather, I think that's a good argument for the DB manager to be available outside of the container, but once you're in the kernel you should be accessing it (or the connection objects it spawns) from the container.

Crell’s picture

Regarding the patch, my Dreditor is MIA right now but getConnectionsInfo() is guaranteed to be confused with getConnectionInfo(). :-)

Anonymous’s picture

"Erf. Anything that happens pre-kernel needs to just go away."

ok. not sure i agree, but its at least a clear, low-level, early bootstrap position to discuss.

we don't even get to $kernel = new DrupalKernel('prod', FALSE); until we've run drupal_bootstrap(DRUPAL_BOOTSTRAP_CODE). that is, after many bootstrap phases and a metric shit-tonne of included code.

so is the vision that we move most of those bootstrap phases into DrupalKernel::init() ? or is this where the BootstrapContainer issue comes in? or ... ?

we don't get a container at ::init(), we get it somewhere after ::boot(), so code between ::init() and after ::boot() should also use ... ?

"Rather, I think that's a good argument for the DB manager to be available outside of the container, but once you're in the kernel you should be accessing it (or the connection objects it spawns) from the container."

if the DB manager is to be a classed object, how do we get at it without a container? do we pass it everywhere? or ... ? it's hard to make sense of an argument that suggests we both make it an object, and access it outside the container.

right now, we seem to be in the worst of both worlds for low-level, early bootstrap code. i understand that we have time to tighten it up, but i'm having a hard time seeing what The Right Way looks like.

sun’s picture

At which point did we decide that the DIC is equal to the HttpKernel? They're separate concepts and components. Saying that one equals the other doesn't make sense to me.

There are plenty of legit use-cases that use the container, but don't want and don't need to bootstrap an HttpKernel. Saying that everything "pre-kernel" should be removed sounds very odd and like a false dichotomy. There's at least Drush, there are individual scripts, and there are unit tests. The latter being what revealed this glitch. (#1774388: Unit Tests Remastered™)

The fix is easy; the lines can be moved as-is, and the database service gets available, without the huge dependency of HttpKernel. That seems to be in line with #1703266: Use a precompiled container for the minimal bootstrap container required outside of page requests

chx’s picture

The kernel has nothing to do with this issue. None. Instead of the corebundle you do this in the emergency container built in drupal_container. So here's the battle plan:

  1. Add a DB service to drupal_container() per connection. We want a factory here not the direct class to avoid actually parsing the connection array.
  2. Make every db_* function use the DIC().
  3. Remove db_set_active() and db_set_active because we covered both cases in #20 by now.
  4. Investigate how to use the testing DIC to avoid the connection add, rename, remove connections stuff. Discuss elinimating this functionality totally. If one needs it, they can manipulate the DIC directly.
  5. Investigate the Database class of what remains. Not much, quite possibly only the factory.
Anonymous’s picture

as a rough, not implemented idea, i would love to see this sort of patch:

diff --git a/core/includes/bootstrap.inc b/core/includes/bootstrap.inc
index fd3b719..754ebff 100644
--- a/core/includes/bootstrap.inc
+++ b/core/includes/bootstrap.inc
@@ -701,17 +701,15 @@ function unicode_check() {
  * Sets the base URL, cookie domain, and session name from configuration.
  */
 function drupal_settings_initialize() {
-  global $base_url, $base_path, $base_root, $script_path;
-
-  // Export these settings.php variables to the global namespace.
-  global $databases, $cookie_domain, $conf, $installed_profile, $update_free_access, $db_url, $db_prefix, $drupal_hash_salt, $is_https, $base_secure_url, $base_insecure_url,
-  $conf = array();
-
   // Make conf_path() available as local variable in settings.php.
   $conf_path = conf_path();
+  $conf = array();
   if (is_readable(DRUPAL_ROOT . '/' . $conf_path . '/settings.php')) {
     include_once DRUPAL_ROOT . '/' . $conf_path . '/settings.php';
   }
+
+  drupal_bootstrap_container($conf);
+
   $is_https = isset($_SERVER['HTTPS']) && strtolower($_SERVER['HTTPS']) == 'on';
 
   if (isset($base_url)) {

from DRUPAL_BOOTSTRAP_CONFIGURATION onwards, you have an early container, and you can alter the initial values in $conf via settings.php.

katbailey’s picture

Re #37, to be clear, the Kernel (not the HttpKernel) is what manages the DIC. And the only reason we still need to bootstrap to that high a level before instantiating it is that in order to fully build the DIC, the Kernel needs to know what modules are enabled, because each module potentially supplies a bundle that needs to be registered to the DIC. So it needs the db. Sad panda.

katbailey’s picture

Though as moshe just pointed out to me, this sadness will go away once #1608842: Replace list of bootstrap modules, enabled modules, enabled themes, and default theme with config gets in :-)

Anonymous’s picture

"Re #37, to be clear, the Kernel (not the HttpKernel) is what manages the DIC. And the only reason we still need to bootstrap to that high a level before instantiating it is that in order to fully build the DIC, the Kernel needs to know what modules are enabled, because each module potentially supplies a bundle that needs to be registered to the DIC. So it needs the db. Sad panda."

i was talking about DrupalKernel, not HttpKernel. though, DrupalKernel doesn't have the DIC until kernel->handle(), which is also when httpKernel is added, so whatever.

getting at the list of modules doesn't require DRUPAL_BOOTSTRAP_CODE, it just requires DRUPAL_BOOTSTRAP_DATABASE. it only requires that because system_list used db_* functions, rather than Database::*. Database::* code works after DRUPAL_BOOTSTRAP_CONFIGURATION. if we cut and paste db_* into bootstrap.inc, then it would work right now after DRUPAL_BOOTSTRAP_CONFIGURATION.

Anonymous’s picture

Title: Add a database service to the DIC » clarify drupal_bootstrap --> DrupalKernel transition

had a good chat with @katbailey in #drupal-contribute, and i think it would be useful to repurpose this issue a bit. there seem to be different ideas about how our early phase code should run, and the relation of the DIC to the Kernel to bootstrap.

so - where do we want drupal_bootstrap() to go in the new symfony world?

given "Erf. Anything that happens pre-kernel needs to just go away. We've already reached that conclusion, just haven't gotten around to doing it yet.", is the goal to kill everything after drupal_bootstrap_configuration() ?

should DrupalKernel do drupal_bootstrap'y stuff in ::init() and ::boot() ? maybe hook_init() becomes an event emitted during ::boot()?

chx’s picture

Title: clarify drupal_bootstrap --> DrupalKernel transition » Add a database service to the DIC

Let's make that a followup. Let's not rush. Getting the database work from the DIC solely instead of the global-like Database class is one step forward.

Crell’s picture

I went ahead and did the database.inc-killing, since there's already a separate issue for it: #1773822-2: Consolidate low-level procedural helpers somewhere

katbailey’s picture

OK, so do we keep this issue for figuring out the whole bootstrap v. kernel instantiation problem then? We really need an issue dedicated to that problem - is this the best place or should we create a separate issue?

Anonymous’s picture

maybe create a separate issue? seems that would be the least controversial...

katbailey’s picture

Crell’s picture

Is this deprecated in favor of #1811730: Refactor Database to rely on the DIC solely? Should it be?

katbailey’s picture

I think this issue can be closed - the bootstrap problems are being dealt with elsewhere, and #1773822: Consolidate low-level procedural helpers somewhere and #1811730: Refactor Database to rely on the DIC solely take care of the database stuff.

Crell’s picture

Status: Needs review » Closed (duplicate)

And it eliminates a major, too. Yay!

Crell’s picture

Issue summary: View changes

Add follow-up todo items.