Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Problem/Motivation
- To demonstrate the MVP for having Views in core, we need to convert at least one core listing to a view.
- The "promoted" node listing at
/node
is a fairly straightforward listing with a few module-specific features, so it is a good test case - New Drupal users often want to know how to customize this listing anyway, so it is an easily discoverable place to add a View in core.
Proposed resolution
- Convert the main
/node
listing to a View.
Remaining tasks
- The next step is to add an upgrade path that enables Views for sites that have the node listing as their front page in Drupal 7. See #75 and #76.
- Identify and create follow-up issues.
Blockers
- #1807020: Add Views and Views UI to the standard install profile
- #1808648: Move tokenization form code into AreaPluginBase
- #1808542: Allow area handlers to override the page title when used in an empty area.
- #1808670: Provide a way for area handlers to be available by area type
- #1809608: Write test coverage for views_fetch_fields
- #1811016: [meta] Decouple tests from Node module
- #1847554: Decouple SummaryLengthTest from the node frontpage (and make it a bit faster)
- #1848200: Random warnings in tests in field_ui.admin.inc
- #1848998: Problems with update_module_enable() / #1849004: One Service Container To Rule Them All
- #1850352: config_import_invoke_owner() should check whether a module exists before invoking its hooks
- #1850158: Bugged assumption in ModuleTestBase::assertModuleConfig()
- #1850418: Provide D7 -> D8 Views upgrade path in contrib, not core
Related issues
- #1776830: [META-1] Installation and uninstallation of configuration provided by a module that belongs to another module's API
- #1807624: Saving and rendering in empty region for 'Global: unfiltered' text does not work
Followups
- #1811816: Benchmark node_page_default vs. node view
- #1847540: [META] Clean up comment module tests and decouple from node
- #1940274: Modules not loaded when they ought to be loaded in \Drupal\Core\Extension\ModuleHandler::buildHookInfo
- #1940306: Convert the node RSS feed to a display of the frontpage view
- #1946526: Clean up Minimal profile modules
- #1961184: "Welcome to Drupal" title (frontpage view "no results" behavior) persists even when there are results
Comment | File | Size | Author |
---|---|---|---|
#189 | drupal-module_handler-1806334--fail.patch | 1.89 KB | ParisLiakos |
#189 | drupal-module_handler-1806334.patch | 1.25 KB | ParisLiakos |
#168 | drupal-1806334-168.patch | 36.22 KB | dawehner |
#168 | interdiff.txt | 778 bytes | dawehner |
#167 | drupal-1806334-167.patch | 36.23 KB | ParisLiakos |
Comments
Comment #1
xjmComment #1.0
xjmUpdated issue summary.
Comment #1.1
xjmUpdated issue summary.
Comment #2
xjmJust saw that @dawehner posted this: #1806100: Replace node listing with a default view. We'll probably want to mark that one as a duplicate and move its patch here.
Should we move this and similar issues to the VDC sandbox queue?
Comment #3
tim.plunkettHere's what I have so far, but it needs the empty behavior, which might have to be either a specific area plugin, or dumbed down. I think I'll write the plugin first, and then we can decide if we really want it to stay the same.
Comment #4
dawehnerSo here is a start:
* Created a custom area handler which handles all this custom logic for the frontpage
* Fixed some other stuff which occurred on this node listing.
Comment #5
tim.plunkettAre we going to export all this crap each time? :(
Why isn't this just part of node_views_data()?
Missing trailing commas
Nice.
Uh?
Comment #6
dawehnerWell we could go through each item and remove default values out before saving.
As i realized we will not get rid of defineOptions even with metadata, because jose
told me that default values are out of scope of the metadata issue.
I manually removed the fields from this export, because actually the view is not using any field
at all (the wizard just adds at least one field all the time, so the user don't get some validation errors).
It is using the global area scope, but i agree it doesn't make sense to be on that as it is too specific.
The last change need for sure some explaination: If you print out <?xml you get also rss in the same line,
the new line fixes that, but i'm open to revert this change.
This happens if you have written to many annotations :p
Comment #7
dawehnerOnce the actual patch is ready i will remove the node module code and have a look whether the node tests still run fine.
Comment #7.0
dawehnerUpdated issue summary.
Comment #7.1
tim.plunkettadded blockers
Comment #7.2
xjmUpdated issue summary.
Comment #8
damiankloip CreditAttribution: damiankloip commentedThere are a couple of issues that will make this patch easier/better I think:
#1808542: Allow area handlers to override the page title when used in an empty area.
#1807624: Saving and rendering in empty region for 'Global: unfiltered' text does not work
Comment #9
dawehnerAssignment for tonight.
Comment #10
damiankloip CreditAttribution: damiankloip commentedAs mentioned in
VDCIRC (Just so it's in here too); We should remove the call to drupal_set_title (in favour of our new area handler) and also remove the code changes to AreaPluginBase.Comment #10.0
damiankloip CreditAttribution: damiankloip commentedUpdated issue summary.
Comment #11
dawehnerRerolled the patch against some current changes and started to realize that a lot of tests
actually rely on the existence of /node to check for example a link to the title.
Due to that all of these tests would require views to work.
Here is a beginning which let use views in two of the identified tests.
Comment #12
dawehnerCommitted the current version so see what all is broken.
Comment #13
xjmI'll work on fixing some of the additional tests.
Comment #14
xjmSo, a lot of the test failures are related trying to take over
rss.xml
rather than to/node
itself. I think we should consider decoupling the feed from the initial node listing conversion, and leavingnode_feed()
in there for now.Comment #15
xjmCurrently, the minimal profile also uses
/node
as the frontpage. I'm removing that configuration and letting it default to/user
.Comment #16
xjmHere's a patch that puts rss.xml back.
Comment #17
xjmAnd removing the node listing front page from minimal. Committed and sent to the bot.
Comment #18
xjmmenu_test.module
is hugely coupled to the node page callback. Ugh.Comment #19
xjmA lot of the tests are also blocked on the standard profile issue.
Comment #20
xjmAnd the outstanding simpletest/plugin manager bugs.
Comment #21
dawehnerThanks for you work!
Based on our experience with the coupling of the tests with the /node page we might should create an issue in the core queue
to discuss a proper decoupling?
Comment #22
xjmYep, there's an issue somewhere already. Trying to find it...
Comment #23
xjmAlright, I filed #1811016: [meta] Decouple tests from Node module. I'll submit some patches through that issue where appropriate rather than simply adding dependencies blindly.
Comment #24
tim.plunkettIn http://drupalcode.org/sandbox/tim.plunkett/1799554.git/commitdiff/6be6883 I worked to get more tests passing.
My theory was to not add Views as a dependency if possible, but to not spend hours decoupling.
Once We get this as close as possible, we can revisit each of those and re-evaluate.
Also, many of these fixes belong in the meta issue, but this is at least an easy way to identify them.
Comment #25
tim.plunkettI'm done for now. The rest are standard profile tests or upgrade tests, I think.
Here's a patch of work to date
Comment #26
xjmI'm still working on this. I've picked a few of @tim.plunkett's changes and my own and filed them as core issues. I'll post a new patch here when I'm done.
Comment #27
xjmI've pushed several more fixes that are also filed as core issues. We can revert some commits once they're fixed in core.
I also cherry-picked the one that webchick committed across into the sandbox branch, and belatedly realized I shouldn't have because then it gets rolled into the patch. So before rolling the patch I did this locally:
Notes for myself on tests to check since I'm getting tired and sloppy:
And then resolving the rest of what fails on the bot, and reverting what gets fixed in core.
More tomorrow.
Comment #27.0
xjmadd another blocker
Comment #27.1
xjmUpdated issue summary.
Comment #27.2
xjmUpdated issue summary.
Comment #28
xjmSo
ContextualDynamicContextTest
,LocaleContentTest
,LocalePathTest
, the misnamedNodeLoadMultipleTest
, andSummaryLengthTest
all fairly legitimately want to test thenode
listing, so adding Views as a dependency there is fine. I think the assertion that checks the front page inNodeTitleTest
should be moved toNodeLoadMultipleTest
. (Edit: and maybe that last one should be renamed, but that is probably out of scope.)Comment #29
dawehnerLet's benchmark the listing already: #1811816: Benchmark node_page_default vs. node view
Comment #30
xjmI reverted the change to
MenuTest
. That test really does want the whole enchilada of the standard profile, so using different paths isn't really justified, I don't think. So the proper fix for that one is blocked on being able to add Views to the standard profile.Comment #30.0
xjmUpdated issue summary.
Comment #31
xjmBreadcrumbTest
also legitimately needs the whole standard profile.Comment #31.0
xjmUpdated issue summary.
Comment #32
xjmI just spent several hours trying to get
CommentInterfaceTest
to not rely on the standard profile, and it's not worth it. We can consider that one blocked on getting standard working too. Opened #1812034: Only use standard profile where necessary in comment.module tests (10% bot speed-up).Comment #32.0
xjmUpdated issue summary.
Comment #33
xjmOkay, here's a patch that removes all the node decoupling in favor of the patches that have all been either fixed or RTBCed in the queue.
Comment #34
tim.plunkettHere's a new patch, just rerolled after #1026616: Implement an entity render controller
Comment #35
tim.plunkettRerolled.
Comment #36
tim.plunkettThis is blocked on #1807020: Add Views and Views UI to the standard install profile
Comment #37
catchIs #1806370: Do full performance testing on the main node listing before and after it is converted to a view really postponed on this, and this really postponed on the standard install profile? I'd like to get some performance testing done of Views 8.x pretty quickly, and before/after front page testing is usually a pretty good example.
Comment #38
dawehner@catch
It is for sure not really blocked as the node listing itself is working so #1811816: Benchmark node_page_default vs. node view is an issue with a propoer performance testing.
Comment #39
xjmComment #40
xjmSee #1818056: Reduce descriptions of default views. Actually the new text is pretty decent. How about:
or even simply:
Comment #41
yoroy CreditAttribution: yoroy commentedpromoted, node and teasers are very Drupally words that we've seen new users struggle with, lets not cram those all into something that should clarifiy things. The first rewrite in #40 seems better to me.
Comment #42
wundo CreditAttribution: wundo commentedI don't understand why this patch requires Views UI, Shouldn't only Views be enough?
Comment #43
damiankloip CreditAttribution: damiankloip commentedAre you referring to #36? Otherwise, I can't see where the dependency is coming from in this issue? :)
Comment #44
wundo CreditAttribution: wundo commented#36 and the issue description: "Switch to the VDC sandbox installation. Be sure to enable the Views and Views UI modules (necessary until they are added to the standard profile)"
Comment #45
xjm#44 That's just to emulate how it would be with standard profile installed once #1807020: Add Views and Views UI to the standard install profile is fixed. You won't have this view and not have Views UI enabled out of the box. You can choose to disable Views UI later, but best to user-test this issue with it on.
Besides, half the fun is seeing the little context links to edit the view right there on the front page. :)
Comment #46
lpalgarvio CreditAttribution: lpalgarvio commentedcheck Admin Views for ideas and code
http://drupal.org/project/admin_views
Comment #47
xjm#46, no, we are using the patch in this issue.
Comment #48
xjmComment #50
xjmThe variable/state value from #1824834: Covert default_nodes_main variable to CMI system should be removed entirely by this patch as well.
Comment #51
dawehnerJust posting a rerole to see whether the tests are green.
Comment #53
dawehnerLet's fix a bunch of the views related tests by decoupling them from the frontpage view.
Comment #55
dawehnerTotally forgot to add the new base class. The interdiff is relative to comment #51
Comment #57
xjmJust a reroll. I'll try to pick off more of the failures.
Comment #58
xjmOh. Recent patches seem to sort of... not include the view. That could be causing part of the difficulty. ;)
Comment #59
xjmThis should help...
Comment #64
xjmAnd the handler...
Comment #66
xjmSo something weird is going on when the test runner runs
config_import_invoke_owner()
-- it tries to invokeviews_config_import_create()
even though Views is not supposed to be enabled. Locally I can work around this by disabling Views in the parent site (parent site environment leaking into the child I guess), but I'm confused as to why it's happening on testbot.Related:
Comment #67
sun@xjm: Can you provide the
debug_backtrace()
for that call?Comment #68
xjmSo the backtrace is over 1 GB with arguments because of the DIC stuff. Good times!
Attached is the full call chain without args, and then the relevant part of the backtrace with the view object truncated.
Comment #69
xjmI wonder if #1780396: Namespaces of disabled modules are registered during test runs is causing this?
Comment #70
xjm@tim.plunkett solved it with this:
So back to decoupling stuff. :)
Comment #71
xjmAttached:
Filed as separate issues:
Comment #72
xjmOh, I should also note that the comment module test changes should be compatible with #731724: Convert comment settings into a field to make them work with CMI and non-node entities as far as I know.
Comment #74
xjm#1848200: Random warnings in tests in field_ui.admin.inc should fix the field UI test failures.
Filed #1848202: Decouple OpenID tests from node.
Comment #75
xjmSo I think between the issues above and this patch we've got it down to just the "real" failures--the bad assumption in
ModuleEnableDisableTest
that all a module's config is in its namespace (#1776830: [META-1] Installation and uninstallation of configuration provided by a module that belongs to another module's API), and the lack of an upgrade path.The upgrade path (shown in the interdiff) causes the upgrade path tests to choke with:
Halp. :(
Comment #76
sunI've seen that update.php exception while working on and debugging many other patches in the meantime - definitely not related to this issue, and thus we should create a separate, dedicated issue for it. (update.php has not been adjusted for the kernel at all recently, which clear appears to be the cause, but then again, update.php cannot boot a regular kernel in the first place, which gets us into a very hairy situation... anyway, separate issue.)
Comment #77
xjmOh, this still needs to be changed to:
A list of nodes marked for display on the front page.
Contains.
Needs verb.
Indentation fail.
Comment #78
xjmI'll wait for testbot to catch up and expose the bug before filing an issue per #76. The bit from #70 may merit its own issue as well.
Comment #78.0
xjmUpdated issue summary.
Comment #80
xjmComment #81
xjmFiled #1848998: Problems with update_module_enable().
Comment #81.0
xjmUpdated issue summary.
Comment #81.1
xjmUpdated issue summary.
Comment #82
xjmComment #82.0
xjmUpdated issue summary.
Comment #82.1
xjm.
Comment #83
dawehnerProvided an upgrade test for the frontpage view.
Comment #84
xjmThis should be wrapped in a condition for the frontpage being node, I think?
Eventually we will want to convert more listings, all of which would require Views to be enabled, so we would add those conditions then as well.
Comment #84.0
xjmUpdated issue summary.
Comment #85
xjmSeparated #70 into #1850352: config_import_invoke_owner() should check whether a module exists before invoking its hooks.
Comment #85.0
xjmUpdated issue summary.
Comment #86
xjmFiled #1850418: Provide D7 -> D8 Views upgrade path in contrib, not core which blocks the update hook added in this issue.
Comment #86.0
xjm.
Comment #86.1
xjm.
Comment #86.2
xjmUpdated issue summary.
Comment #86.3
xjmUpdated issue summary.
Comment #87
xjmRecent cleanups and chasing head. @dawehner is correct that te expected value of the
system.site.page.front
inSystemUpgradePathTest
is already'node'
, but we can't put that assertion there because Views will not ne enabled forSystemUpgradePathTest
. We can add @dawehner's assertions in aViewsUpgradePathTest
onceupdate.php
is fixed.Comment #89
sunQuite important consideration, as I noticed the "Convert" in the issue title only now:
This patch attempts to actually remove and replace the non-views-based listing of nodes on /node with a views-based listing.
Consequently, this adds a relatively huge dependency on Views module to Node module. Essentially, you will not be able to use Drupal in any basic/sensible way without Views module anymore.
I'm not sure whether that is a good idea. Views is nice and all for creating and overriding custom, user-configurable listings. But Views is also a mammoth thing. Doing this conversion here, instead of overriding the route/path with a view, would essentially set the precedent for "convert all listings into views" — having the end result of rendering Drupal unusable without Views module being enabled - which in turn would make Views a required module. I do not think that Views is ready for that yet. I always assumed that to be D9+ material (if at all).
For D8 it would make more sense to me if Node module would ship with a default view that allows to replace the /node listing with a views-based listing. I wouldn't really care whether that default view is enabled or disabled by default. However, the essence is that there is only a views-based listing if Views module is enabled. If Views module is not enabled, then Node module is still fully operational and able to provide basic content functionality, which includes a basic listing of nodes.
If you want to pursue the full conversion/replacement idea, then I think there should be a separate issue first to discuss the consequences of doing so (which are much larger than creating and supplying this simple node default view).
Comment #90
webchick"Consequently, this adds a relatively huge dependency on Views module to Node module. Essentially, you will not be able to use Drupal in any basic/sensible way without Views module anymore."
I actually don't think that's true. With Node module you can create content types, you can pull them into Panels pages, you can list them within an Organic group, you can request them with REST-based web services, etc. The fact that core has in the past shipped with a non-turn-offable page + RSS feed called "node" that arbitrarily exposed teasers of all nodes on your site that have an arbitrary checkbox checked, is what the bug is, IMO. I don't see a "dependencies[] = views" line added to node.info, so IMO the patch is fine.
Comment #91
cosmicdreams CreditAttribution: cosmicdreams commentedProduct vs Framework. This is about the default we want Drupal to ship with. With the default config of a Drupal 8 installation we're making the choice here to provide a configurable collection of nodes at /node.
Don't want that? Think we're hard-wiring a dependency of node.module on views.module?
Here's the litmus test:
1. install Drupal 8 with minimal installation profile
2. Create a content type
3. Add a bunch of nodes of that content type
4. Create a display of a collection of nodes without views.
It CAN be done. Therefore, no dependency.
Comment #92
xjmYeah, I pretty much agree with #90.
I will say that converting (at least some) core listings to Views has always been part of the initiative's scope, and stated on our roadmap, meta issue, blog posts and etc., from the very first brainstorm we held back in May and in everything we've posted since. :)
We chose the node listing specifically because it's very much a feature. It's both easily discovered when you install Drupal, and easily replaced with something else. (How many sites do you have that actually use the default front page? My first question when I installed Drupal 4.7 was how to get rid of it.) In this patch, the node listing view is a part of the standard profile only, and that's a very deliberate choice.
However, @sun raises an important point that we have been debating in various channels for awhile, about whether we should convert listings that are more fundamental to basic site operations. There are pros and cons, and it needs very careful consideration, though out of scope here. We are planning on discussing this topic at a sprint this weekend, and I'll do a writeup based on that discussion and file an issue for it.
Comment #93
xjmSorry, small but critical typo in my last post: the promoted node listing view as the frontpage is a part of the standard profile only. The view is always available if you install both node + views, for administrators to do with as they wish, or not.
Comment #94
xjmAlso, as way of general reassurance since I keep finding my own posts insufficiently clear:
We do not intend for Views to be a required module in Drupal 8. We also do not intend to add a dependency on Views to fundamental site building modules like node and taxonomy. We may make optional features of these modules dependent on Views. I'll link the new issue here for everyone's reference once we file it, and then we can discuss the details more there. :)
Comment #95
xjmHere's the followup from our discussion at the London sprint: #1864980: [meta] Figure out how to integrate Views into core
Comment #96
YesCT CreditAttribution: YesCT commentedReading through... especially #83 and #87 are upgrade path tests still needed?
Was there another issue number for "once update.php is fixed"? Or was that to be part of this issue?
There is so much done already here. I know xjm is super busy (blocks!!!). Maybe some detailed next steps will help bring in someone more people to support this?
I'm sure anyone eager could try a reroll of #87 to continue the chase of head.
Comment #97
dawehnerSee #1848998-81: Problems with update_module_enable()
Comment #98
xjmYes, this issue is totally blocked on #1848998: Problems with update_module_enable(), not my participation, and has been since Nov. 23 in #81. (It's not a coincidence that I started on the blocks patch shortly after that.) We cannot write passing upgrade path tests until the upgrade path works. :) Everything in #87 is still true, and the summary is still current.
Also, it's assigned to me still, by design. :) This is still the top priority issue in VDC and work on it will resume once it's unblocked.
Comment #99
YesCT CreditAttribution: YesCT commentedComment #101
xjm#87: views-1806334-87.patch queued for re-testing.
Comment #103
dawehnerIf we need some time anyway we could move this to another issue, as it's actually unrelated to that issue.
We certainly need a rerole after that time, if it's not done until monday I'm happy to split it up.
Comment #104
dawehnerJust a plain old rerole and codestyle fix.
Comment #106
dawehnerJust let's wait on #1848998: Problems with update_module_enable() and a lot of them should get green.
Comment #107
xjmRerolled after #1848998: Problems with update_module_enable()
Comment #109
dawehnerA list of follow ups:
Comment #110
xjmI think #1893850: Cleanup relationship tests and don't use the node module should probably go in first; it will simplify this patch.
Comment #111
dawehnerRerolled against lot of changes, and fixed many of the failures.
It feels like we should really get at least one conversion in before feb 18th?
Comment #112
YesCT CreditAttribution: YesCT commented1 extra newline. (non-blocking)
1 extra newline (non-blocking)
test_view seems not specific enough. I suggest test_frontpage
(I'm not sure. Why is a test view being tested instead of testing the actual view being added?)
glossary?
Oh, is it that the frontpage is enabled by default, so to test some views are disabled by default and can be enabled, so use a different view than frontpage?
Probably the comments need to change.
Comment #113
dawehnerThanks for the review! Please testbot run that patch :)
Fixed
Fixed
The idea is to not use the frontpage anymore, so if you test against a generic view, we use test_view.
Adapted the comments.
Comment #114
YesCT CreditAttribution: YesCT commentedwriting down some things from an irc conversation with @dawehner
this and other substitutions of using the test_view (already existing, and not introduced as part of this patch) instead of the frontpage view makes sense.
previously the frontpage was used for test unrelated to the frontpage because it was a view that was handy.
this also makes sense, is replacing the use of the frontpage view to do some (non-frontpage specific) testing with a different default view that starts disabled.
from the interdiff, due to some merge problems this render function have been twice in the title.php. So this is just removing that.
So, far, to me, the code makes sense, comments are updated, coding standards look good, and the recent interdiff looks good. this is rtbc except...
next steps:
@dawehner says one test is failing locally, but might be unsure of how to fix it.
also, a manual test of the frontpage would be good
Comment #116
dawehnerThis will potentially fix like 90% of the problems.
Comment #118
tim.plunkettOh! Those are the same failures I fixed in #1851086: Replace admin/people with a View. Which makes sense.
Comment #120
tim.plunkettOkay, that fails just because config('system.site')->get('page.front') == 'node', and views is not on.
In #83 and #84 this was discussed, but there was never an upgrade path added.
Comment #121
dawehnerPlease!
Comment #122
dawehnerSo let's test the patch now.
Comment #124
tim.plunkett- $this->assertTrue($this->container->get('module_handler')-moduleExists('views'), 'Views is enabled after the upgrade.');
+ $this->assertTrue($this->container->get('module_handler')->moduleExists('views'), 'Views is enabled after the upgrade.');
:)
Comment #126
tim.plunkettThat'll teach me to run tests locally first.
Comment #127
dawehnerI should have done this as well :)
Let's stop forward.
Comment #128
catchI know #1811816: Benchmark node_page_default vs. node view is open but it'd be good to get a before/after before this goes in.
Both for xhprof profiling, and also for the difference in the page size since the markup will change here as well.
Comment #129
Dries CreditAttribution: Dries commentedLooked at the code and it looks good to go. I agree that we should look at the profiling results first though.
Comment #130
YesCT CreditAttribution: YesCT commentedtagging for profiling
Comment #131
effulgentsia CreditAttribution: effulgentsia commentedProfiling is work :)
Comment #132
dawehnerI did a simply xhprof run with 50 total nodes and came to the following result.
Comment #133
Fabianx CreditAttribution: Fabianx commentedI also redid some of my original benchmarks, so those are comparable with old data (http://drupal.org/node/1811816#comment-6675896);
This was with 1000 runs - same results. APC autoloader is off.
Views is and keeps being 32 ms worse in this benchmark, but with 50 nodes and no caching on that is not too bad.
But that overhead I already explained in http://drupal.org/node/1811816#comment-6670756 and it keeps being valid.
Comment #134
dawehnerOn mu/pmu you basically see how much memory a full loaded view object needs.
In general 32ms worse seems to be acceptable, especially if you can turn on caching. If we do that though, we need first a better support for cache tags, as currently you might end up with new nodes not appearing on the node listing page, as the page is cached.
Set back to RTBC? (Not sure whether that is the proper status).
Comment #135
catch2mb for a fully loaded view object seems a bit wrong to me, or at least maybe that's gone up rather than down since 7.x. I'm trying to remember when I lasted tested this for 7.x-3.x but can't find the issue at the moment. Might try to look myself another time (unfortunately no chance this week).
#1605290: Enable entity render caching with cache tag support is probably going to have to move to being independently critical between Views, EntityNG and Twig.
Comment #136
BerdirI've been doing some tests too and it looks quite nice here
Disclaimer:
* I'm using the APC class loader, which removes those 10% class loader stuff from @dawehner's screenshot.
* I've disabled views_ui although that didn't seem to make such a big difference.
* Something that needs to be kept in mind is that function calls have a considerable cost when profiling. So when having more function calls, then it's actually less bad than it looks with xhprof.
Notes:
* The time difference is quite small for me, actually (overview.png)
* Seeing a handful of additional cache loads for the plugins, but that's about it for queries.
* I can confirm the additional 2MB in the overview but I can't really track them down. There's a lot of "movement" in theme functions, it looks like the main node rendering happens now in theme@2 and no longer theme@3, so one level less nesting but it seems to sum up. There's a lot of red (more memory) lines below then but it's a lot of small stuff, so nothing obvious to improve (functions.png)
So, it looks quite good to me. Will try to do another round of "What happens if I add 20 recent comment views blocks on a page" in the other issue to see if there some generic overheads but I think a lot was already improved.
Also did a quick try with a field based view (title, image, body) which looks visually very similar to the node listing but is 34% faster (still 1MB more memory, though). No idea how that will work with an EFQ backend but I think it's a nice example for the "we're comparing apples with oranges" argument. I think the number of sites that have to worry about performance (apart from the crappy shared host performance problems) and are using the default node view are very low. Even simple blogs often use views because they want a different sort order, or a filter or something like that.
Therefore, having a view there by default instead of an arbitrary node listing that nobody uses makes *a lot* of sense, because it means the typical frontpage profiling is profiling the same thing that real sites will use. So in short, a lot of words for saying +1 :)
Comment #137
sunMmm... @Berdir's bench shows a 6% "improvement" in
ModuleHandler::load()
.That looks highly suspicious.
Comment #138
Fabianx CreditAttribution: Fabianx commented#137: This is within error margin. Without doing a test 1000 times and then use aggregate or minimum, you'll always have such "error margins".
And even then it can happen. I check for mine (minimum of 1000 runs) and I have "1" ms difference.
#136: Creating a field based view is a great idea and I'll benchmark that as well. Also I'll redo benchmarks with APC classloader on.
I also had an error in my benchmark :-/ as my min code did not actually run:
Here is new and now accurate numbers, I'll fix xhprof-kit and will retest more automatically soon:
This now is benchmarked like:
XHProf_Start
index.php
XHProf_End
while the previous benchmarks had been just using "devel".
Benchmak setup - No APC autoloader
* PHP 5.3
* APC Autoloader - Off
* node vs. frontpage (view) vs. frontpage-fields (view with fields)
Benchmark results: /node vs. /frontpage
Permalink for easy browsing: http://www.lionsad.de/xhprof/xhprof_html/?run1=512a1f7f217a6&run2=512a20...
Benchmark results: /node vs. /frontpage-fields
Permalink for easy browsing: http://www.lionsad.de/xhprof/xhprof_html/?run1=512a1f7f217a6&run2=512a22...
----------
Benchmak setup - With APC autoloader
* PHP 5.3
* Classloader: APC
* node vs. frontpage (view)
Benchmark results: /node vs. /frontpage
Permalink for easy browsing: http://www.lionsad.de/xhprof/xhprof_html/?run1=512a40915a5dd&run2=512a40...
----------
I will now always give both numbers and permalinks to XHProf Runs. :-)
Comment #139
BerdirDiscussed our results with FabianX in IRC, some more conclusions and thoughts.
- As you can see in the updated benchmarks, 7ms of the additional time is spent in the classloader and they go away when you use APC as I did.
- I'm doing the profiling manually ( usually run 3 times and use the fastest run) so there is a higher chance of errors. That said, my numbers seemed to be relatively stable.
- I'm using PHP 5.4, which probably contains a few performance improvements.
- We were testing with 50 nodes. That means that the code path where there actually is a difference between node_default_page() and views is a relatively small part of the total execution, ~80% of the time is spent in drupal_render_page. So small differences there (e.g. if you have different users that need to be loaded or a single one, logged in or not) change the total execution time and therefore the procentual difference between node_default_page() and views.
- FabianX and I also did an example test with just a single node, where we had comparable results of ~20% overhead despite quite different absolute numbers.
- @msonnabaum also mentioned in IRC that the used flags and whether xdebug is enabled or not have huge difference in xhprof. I had xdebug enabled on my laptop and it slowed it down ~30% (not used that in the published results, just an example). Did a test without XHPROF_FLAGS_CPU on my desktop and it was another ~25% faster.
All that said, I'm still not sure what a fair comparison would be, 1 node certainly isn't. And 50 is too much. And a single listing as a whole is tricky because views has a one-time overhead and many many sites will also have a view listing or two (or 10) in a block anyway.
IMHO, the shown regression in terms of performance ( be it my numbers of those of others) are worth it. Especially because we showed that there are various ways to make that up again for sites that care. Enable caching, switch to a field based view, ...
Comment #140
Fabianx CreditAttribution: Fabianx commentedI agree with berdir. +1 for RTBC
Comment #141
Fabianx CreditAttribution: Fabianx commentedI agree with berdir. +1 for RTBC
Comment #142
catchStill need a diff of page size here, that's something that could be done by a novice contributor, so tagging novice.
Comment #143
Fabianx CreditAttribution: Fabianx commented50 Nodes displayed:
Node: 95.02 KB (14.76 KB compressed)
Views: 98.58 KB (15.11 KB compressed)
Measured with Chrome Network tab as anonymous user.
Comment #144
dawehnerInstallation still shows some debug messages :(
Comment #145
dawehnerAfter some debugging the problem is the following: The moduleHandler sort of caches hook_hook_info way to early, so there is no file included for node.views.inc
On later time of the installation (for example another batch request) the modules are enabled, so the hook info is there and views can fire it's hooks.
Comment #146
YesCT CreditAttribution: YesCT commentedtaking novice off since the page size comparison was done.
Comment #147
dawehner#1929656: ModuleHandler::setModuleList() should call resetImplementations() in order to avoid edge cases is an issue to solve the problem.
Comment #148
YesCT CreditAttribution: YesCT commentedI think that means we are back to rtbc.
Comment #149
YesCT CreditAttribution: YesCT commented#126: vdc-1806334-126.patch queued for re-testing.
Comment #150
dawehnerWell, It's more like that this issue is blocked on the other one :)
Comment #151
dawehner#126: vdc-1806334-126.patch queued for re-testing.
Comment #153
dawehnerRerolled.
Comment #154
ParisLiakos CreditAttribution: ParisLiakos commenteddebug messages still there:(
Comment #155
YesCT CreditAttribution: YesCT commentedmaybe these are the debug messages referred to?
they were not introduced by this issue though..
Comment #156
YesCT CreditAttribution: YesCT commentedoh #144 says installation. So I'll try manually testing.
Comment #157
YesCT CreditAttribution: YesCT commentedhere they are:
Comment #158
ParisLiakos CreditAttribution: ParisLiakos commentedNow, after applying the patch installer dies:
Fatal error: Call to a member function id() on a non-object in /var/www/d8/core/modules/menu/menu.module on line 210
Lets see what bot says, mybe my installation is messed up
#153: drupal-1806334-153.patch queued for re-testing.
Comment #160
ParisLiakos CreditAttribution: ParisLiakos commentedthis should take care of that..no idea yet what happens with the debug messages
Comment #161
dawehner@rootatwc
How should that fix the messages when install drupal? I would love to understand that :)
Comment #162
ParisLiakos CreditAttribution: ParisLiakos commentedNo, this is to fix the fatal error (see #158)...debug messages are still there..i am still investigating, but i have an idea
Comment #164
dawehnerAh great. In general I still think we should have tests for that behavior.
Comment #165
ParisLiakos CreditAttribution: ParisLiakos commented#160: drupal-1806334-159.patch queued for re-testing.
Comment #166
catch3kb is a fair bit, although in context it's unfortunately not that much.
What about the difference in document.getElementsByTagName("*").length ?
Comment #167
ParisLiakos CreditAttribution: ParisLiakos commentedthis should get rid of debug messages..getHookInfo acts on a full moduleList, but none of this modules is loaded yet..and then hookInfo is cached
Comment #168
dawehnerAs we iterate over all modules anyway, can't we just do $this->reload() before the foreach?
Comment #170
ParisLiakos CreditAttribution: ParisLiakos commented#168: drupal-1806334-168.patch queued for re-testing.
Comment #171
ParisLiakos CreditAttribution: ParisLiakos commentedi am gonna rtbc this, since it would be great to have it for the sprint weekends..i think we just need sun's feedback on this last change on moduleHandler..if he is fine with it i dont see any reason not to commit this...but, i ll leave the rest on maintainers:)
Edit:
document.getElementsByTagName("*").length:
Before:
Logged in: 451
Logged out: 100
After:
Logged in: 457
Logged out: 102
Comment #172
xjmI tested the patch manually and confirmed that the debug messages are gone. I did notice a JavaScript bug with the contextual links on the View, but that's out of scope here so I'll file it separately. I think this is finally ready!
This change is fine. It's essentially the same code, just more clearly factored. :)
Comment #173
xjmOh, @rootatwc is referring to the change in #167. That does need review -- please don't mark your own patches RTBC. ;)
Comment #175
dawehner@xjm
If it's his patch, I can mark it as RTBC :)
The change in #160 is perfect. The change in #167 is totally overridden by #168, so this seems to be fair.
Comment #176
dawehnerNice, killed all the tags!
Comment #177
webchickRock!! Awesome work, folks! It looks like catch's questions/concerns have been addressed. This will be SO awesome to have in core for the Drupal Global Sprint Weekend.
Committed and pushed to 8.x. YAY!!!!
Comment #178
lpalgarvio CreditAttribution: lpalgarvio commented=)
Comment #179
lpalgarvio CreditAttribution: lpalgarvio commentedgreat work =)
Comment #180
Fabianx CreditAttribution: Fabianx commentedYay!
Comment #181
yched CreditAttribution: yched commentedMinor : core/modules/node/lib/Drupal/node/Plugin/views/area/ListingEmpty.php :
Stale class name in the @file docblock ?
Comment #182
dawehnerOh nice catch!
Opened a novice follow up for that: #1938132: ListingEmpty contains wrong @file
Comment #183
damiankloip CreditAttribution: damiankloip commentedI didn't want to ruin the party here either, but we should change the empty area handler :/, So I created: #1938414: frontpage (/node) view should use unfiltered text for empty area handler
Comment #184
andypostedit we have special CommentLinksTest for the purpose
Any reason this assertion was removed?Comment #185
sunIt does not really look like we got to the bottom of why modules are suddenly not loaded when they ought to be loaded.
I additionally do not understand why this calls into
reload()
instead ofloadAll()
.We definitely need to change this into
loadAll()
, and we should also figure out why this is suddenly necessary, as it should not be necessary — instead, there's likely a race condition or bogus/incomplete procedure elsewhere in the code that needs to be fixed.I do not really understand where the front page is configured to be 'node' in the Minimal profile now?
Comment #186
YesCT CreditAttribution: YesCT commentedtagging Novice for the task of going through this issue, reading the comments, identifying needed follow-ups and opening those issues.
For anyone who decides to tackle that task, See http://drupal.org/node/1155816 for issue summary template info, and be sure to include a link back to this issue in the issues that are opened as follow-ups. Be sure to also add the followups to this issue summary's follow-up section. (I'll update the issue summary with this remaining task.)
Comment #186.0
YesCT CreditAttribution: YesCT commentedUpdated issue summary.
Comment #187
dawehnerThe frontpage of the minimal profile now is not /node anymore, it's /user, because that's the default value of system.site->page.front. Do you think we should install views in the minimal profile, just to have /node available, this feels really wrong :)
You are right, if modules are loaded already, loadAll() will be enough. If modules aren't loaded, loadAll() will be okay as well.
Comment #188
ParisLiakos CreditAttribution: ParisLiakos commentedActually loadAll() wont work, because $this->loaded is TRUE..
this happens because during installation some modules are enabled without calling module_enable()..so, most of the time in the installer, only system's module hooks are loaded
This is not a problem when using module_enable() because it explicitly calls load() on the new enabled module:
So if you manually update the moduleList somewhere in your code, you *must* call load or reload methods if you want the new hooks from this module to be available..
test coming
Comment #189
ParisLiakos CreditAttribution: ParisLiakos commentedHere we are..I dont know if we want module handler to keep moduleList and loadedFiles synchronized though. in my opinion i think it should...But maybe this is not the best way, maybe we should call reload() in resetImplementations() like dawehner originally proposed in irc
Comment #190
xjmFWIW, I set this back to NR because I suspected some of what's in #185. I had been pinging people to try to get review of that change, but then it got RTBCed and committed overnight. :P
Can we spin off followup issues please for #185 through #189? We're at almost 200 comments here. (Also, the failing test in #189 isn't.) :)
Comment #191
sunFollow-ups first, please.
FWIW, the suspicions in #188 basically tell that 1) the problem only exists in the installer, and 2) the problem only exists in the installer. We might want to look into the installer. ;)
That would also explain why the test is not able to detect the problem. We'd have to write a dedicated DUTB test that is actually able to resemble the installer environment of 0 enabled modules.
re: #187.2:
Let's also create a follow-up issue to discuss the fate of Minimal profile. The cumulative result of this change doesn't make much sense. We either need to 1) Add Views, or 2) Remove Node, or 3) Both.
Comment #192
stpaultim CreditAttribution: stpaultim commentedI tried to spin-off the first issue raised in #185 to #1940274: Modules not loaded when they ought to be loaded in \Drupal\Core\Extension\ModuleHandler::buildHookInfo. But, I didn't fully understand it, so I'm not sure how helpful that was.
Also, created #1940306: Convert the node RSS feed to a display of the frontpage view based upon notes in the issue summary.
There is another issue here regarding the minimal profile, which I'm not comfortable spinning off cause I not sure what to say. Maybe someone else will have to finish this (and tidy up what I've done).
Comment #193
dawehnerThanks for creating the followups!
Comment #193.0
dawehnerupdated remaining tasks, not sure about the task already there regarding upgrade path
Comment #194
xjmI really don't think this is a problem. @sun can file a followup for that if he likes. :)
Comment #194.0
xjmUpdated issue summary.
Comment #194.1
xjmUpdated issue summary.
Comment #195
stpaultim CreditAttribution: stpaultim commented@YesCT suggested I go ahead and create the follow-up issue for comment #1806334-191: Replace the node listing at /node with a view so I created #1946526: Clean up Minimal profile modules
Comment #196
David_Rothstein CreditAttribution: David_Rothstein commentedI just bumped #987242: The "Promoted to front page" checkbox doesn't do anything if the /node front page listing isn't used to major (and retitled it).
So does there also need to be an issue to add recommends[] support, so that Node module can recommend other modules (like Views, or Menu) be installed along with it? (There's an issue at #820054: Add support for recommends[] and fix install profile dependency special casing which sounds like that, but I believe it's actually about something a bit different.)
Comment #197
xjmFiled #1961184: "Welcome to Drupal" title (frontpage view "no results" behavior) persists even when there are results.
Comment #197.0
xjmAdded a follow-up issue.
Comment #197.1
xjmUpdated issue summary.
Comment #198
damiankloip CreditAttribution: damiankloip commentedThat looks like a duplicate of #1956912: Title area handler sets the title even when it should not (results in "Welcome to Drupal" never going away).
Comment #200
PanchoWe seem to be missing a change notification here.
Think a single notice for all "Replace x with a view" would be enough or even better.
Comment #201
xjmComment #202
webwarrior CreditAttribution: webwarrior commentedProposed change notification:
https://drupal.org/node/1895160#comment-7783017
Comment #203
catch#2083415: [META] Write up all outstanding change notices before release
Comment #204
webwarrior CreditAttribution: webwarrior commentedChange notification added:
https://drupal.org/node/2084727
Comment #206
ParisLiakos CreditAttribution: ParisLiakos commentedComment #206.0
ParisLiakos CreditAttribution: ParisLiakos commentedUpdated issue summary.
Comment #207
xjm