Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
Problem/Motivation
#1874534: Introduce a dblog / watchdog views integration Introduced dblog views integration. Now is possible to replace the logs report with a view.
Proposed resolution
- Provide the a view to administer logs.
- Keep the original admin as fallback.
Remaining tasks
- Refactor DblogTest::filterLogsEntries() to use
isset
instead ofis_null
- Adapt the tests to extend the new BTB base class. See: #2863372: Convert dblog web tests to BrowserTestBase
User interface changes
The fieldset for Filter Log Messages has been removed, the exposed filters for Severity and Type are always visible.
Without views
With views
API changes
None
Data model changes
None
Original report by klonos
In #1874534-27: Introduce a dblog / watchdog views integration @Psikik added this but it was deemed best to put it in a separate follow-up issue. Well, here it is ;)
Comment | File | Size | Author |
---|---|---|---|
#169 | interdiff-2015149-164-169.txt | 2.27 KB | dagmar |
#169 | 2015149-169-test-only.patch | 29.11 KB | dagmar |
#169 | 2015149-169.patch | 29.1 KB | dagmar |
#168 | watchdog.txt | 19 KB | mondrake |
#164 | interdiff-2015149-158-164.txt | 1.39 KB | dagmar |
Comments
Comment #1
Psikik CreditAttribution: Psikik commentedHere's the view by itself. You'll need to apply the patch in #1874534: Introduce a dblog / watchdog views integration to get this one to work.
Comment #2
alesr CreditAttribution: alesr commentedReviewed the patch and it looks good.
We should replace /admin/reports/dblog with this view. The view page is currently on /admin/reports/watchdog.
Comment #3
dawehnerRight, we also should set the patch to needs review so we know whether tests do pass.
We should really use the mini pager.
Comment #4
dawehnerBlocked on #2027115: Allow views to override existing routing items
Comment #5
damiankloip CreditAttribution: damiankloip commentedComment #6
klonosWhy hate fullstops? :P
Comment #7
damiankloip CreditAttribution: damiankloip commentedI thought everyone hated fullstops?! No, just me then... ;)
Comment #8
xjmComment #9
damiankloip CreditAttribution: damiankloip commentedHere is a reroll and update of the default view config to add UUID, dependencies, description, properly typed values etc..
Comment #10
dawehnerThings we have to figure out:
Comment #12
damiankloip CreditAttribution: damiankloip commentedI particularly like point #2 :)
Comment #13
dawehneryeah, i hate people which try to bring up always 3 points
Comment #14
PolHi all,
I just tried this patch and here's my feedback:
I have a bit of free time and I'm willing to work on this, could you guys help me by telling me what to do (and not to do) first ?
Thanks !
Comment #15
dawehnerGood idea but sadly as you said reality sucks! I am quite sure that we cannot make watchdog entries entities due to the performance as well as subsystem dependencies problems.
I guess we could drop it but yeah alternative we have to write an alternative field handler.
We should not care ... tbh.
Good catch! Could you post a new version of the patch? This is at least a step forward...
Oh interesting. Did you figured out why? Is this just a random css problem?
Comment #16
xjmAgreed, we've dropped the fieldset for admin/people and admin/content with the signoff of our usability maintainers. It actually makes the form cleaner.
And yeah, I think we probably want to do the field handler for the icons for this conversion to get in.
Thanks everyone!
Comment #17
Psikik CreditAttribution: Psikik commentedGoing to take sometime to update this again.
Comment #18
alansaviolobo CreditAttribution: alansaviolobo commentedwith the fixing of this, https://www.drupal.org/node/488570 could also be closed.
Comment #19
alansaviolobo CreditAttribution: alansaviolobo commentedI believe, this would also be a matter of adding another field to the view
https://www.drupal.org/node/155324
Comment #20
PolNever saw the replies till today !
I will provide a new patch within the day probably.
Comment #21
dawehnerI guess the way forward on this issue is to write a custom area handler for the clear all form and be done with it. This approach should work, even it maybe feels hacky for some people.
Comment #22
alansaviolobo CreditAttribution: alansaviolobo commentedCreated a patch with the view for dblog entries.
Comment #24
jibranComment #25
PolI'm going to take care of this.
Comment #26
PolNew patch.
URL of the view: admin/structure/views/view/watchdog
I'm trying to get the CSS included when viewing the view page, I'm looking for help on this.
Comment #27
PolComment #29
dcam CreditAttribution: dcam commented#26 applies to HEAD. Removing the "Needs reroll" tag.
Comment #30
mgiffordUnassigning stale issue. Hopefully someone else will pursue this.
Comment #31
dagmarIn this patch I added the following items:
There is a single item I see could be fixed, the Clear Logs button is displayed at the top of the view instead of the bottom.
Screenshot of the current status attached.
Comment #33
dawehnerJust a drive by review
quick nitpick: We don't use UUIDs in exported content
Let's inject the form builder ...
I'm a bit curious why the form itself doesn't include the library ...
Comment #34
dagmarThanks @dawehner
I moved the view into config/optional, I hope this fix the 100 errors from the previous test.
Done.
Please let me know if you meant the code I included in init() method.
This is because the library for the CSS is added in the the table and we where not rendering the table. I attached the same library to the clear log button form.
Comment #36
dagmarThis should fix the remaining tests.
Comment #37
dawehnerThis migrate test is just a bit sad.
Let's use the create method to inject it properly and also document this variable on the class
we could use ::class here
Comment #38
dawehnerComment #39
dagmarOk, new approach. Instead of implementing an area handler, lets implement a Exposed form. With this approach is possible to place the Clear log form at the bottom of the view.
Regarding the formBuilder I'm leaving this on purpose because this could be moved into another issue (There is another call in renderExposedForm) and we could unify and document it properly once #2642598: Create an interface for the public methods on ExposedFormPluginBase gets in.
Comment #41
dagmarAdding schema for the dblog exposed form, not sure if this will fix the issue with the tests.
Comment #42
dagmarUpdated issue summary.
Comment #43
dagmarClear logs button is not redirecting to the confirmation page.
Comment #44
dagmarLinked:
#1635646: Admin users should know site report permission allows for more than viewing
#2165941: Write tests for system reports
Comment #45
dagmarAll tests for the default recent logs UI are now applied to the views version.
Comment #46
dagmarSome final adjustments. I think this is now ready to go.
All the tests applied to the standard admin/reports/dblog are also run over the UI provided using views.
Comment #47
dagmarAccording to #2642598-47: Create an interface for the public methods on ExposedFormPluginBase @file comments are not longer required on PSR/4 classes.
Comment #48
dawehnerDoes someone mind posting an image of with and without views?
You could use
DblogClearLogForm::class
I like those new methods!
Comment #49
dagmarComment #50
dawehnerComment #51
dawehnerNice addition of the responsive classes!
Comment #52
dagmarThis will make not necessary: #582622: provide hook for dblog_filters.
Comment #53
catchI understand this maintains feature parity, but it seems odd to me. I'd rather move it to a tab or something to avoid making it part of the views integration.
Comment #54
catchOr it could be an action link to a confirm form instead of a tab.
Tagging for usability review.
Comment #55
yoroy CreditAttribution: yoroy at Roy Scholten commentedI think we should maintain the fieldset around the filter options. It's not really clear at the moment that the "Filter" button applies to the multi select boxes above. (Design principle: group related items)
We could merge the two fieldsets and provide the more standard red "Delete" link:
Maybe #2506449: Transform "Clear log messages" submit button into a link in admin/reports/dblog can be repurposed to do that.
Comment #56
yoroy CreditAttribution: yoroy at Roy Scholten commentedComment #57
dagmarI posted a patch here #2506449: Transform "Clear log messages" submit button into a link in admin/reports/dblog that needs some code review. I'm deleting usability review tag since @yoroy already did a usability review there.
I'm setting this as postponed until that patch be committed, then I can provide a new patch here much more simple to review here.
Comment #58
catchJust committed #2506449: Transform "Clear log messages" submit button into a link in admin/reports/dblog so this is unblocked.
Comment #59
dagmarHere is the new patch without the ExposedForm plugins.
Comment #60
dagmarAttached screenshots comparing the versions with Views and without Views.
Comment #61
dagmarIn older versions of #1635646: Admin users should know site report permission allows for more than viewing we had some extra coverage that I'm including here.
Comment #63
dagmarThat test pass locally for me.
Comment #64
dagmarSomeone would like to do a final review about this patch? @dawehner @damiankloip ?
Comment #66
dagmarRe-rolled against 8.3.x.
Comment #67
dawehnerThis looks simply nice for me
Comment #68
alexpottRemoving this assertion doesn't look right. It looks like it is there to provide a positive assertion with the assertNoRaw()
Comment #69
dagmarWell I removed that assertion because views trim sightly different strings and that assertion didn't pass using the full text. Lets try with this approach.
Comment #71
dagmarSorry I included the code from another issue in the same patch.
Comment #73
dagmarRerolled.
Comment #74
LendudeI never like using #id here, since it can be validly altered by previous form alter calls, I think
$form_state->get('view')->id() == 'watchdog'
is more robust.And don't we want some sort of update path for this? I'd love to have this in my existing sites without having to manually import it, but I'm fine with a follow up for that if people agree this would be a 'nice to have'
Other then that, this looks great to me and all feedback in previous issues has been addressed.
Comment #75
dagmarThanks @Lendude here is the new patch with your suggestions.
Comment #76
LendudeI think config changes are usually done in post_update hooks (see views.post_update.php for some examples)
I think this needs all needs to be inside an if (\Drupal::moduleHandler()->moduleExists('views')) or users will see this message when views isn't actually installed. Which would be confusing.
Also the order of the calls in
if (!View::load('watchdog') && \Drupal::moduleHandler()->moduleExists('views'))
should probably be the other way around. But that should be resolved too if you move the \Drupal::moduleHandler()->moduleExists('views') call to a higher level if()Comment #77
dagmarThanks @Lendude. I tried to use the dblog.post_update.php without luck, the only way I could ran the updates was using hook_update_N.
Applied the other suggestions.
Comment #78
Lendude@dagmar this post_update works for me.
Comment #80
LendudeSilly testbot saying it failed when it didn't!
Comment #81
jibranNice work this is almost ready just few observations.
This should be
$form['#attached']['library'][] = 'dblog/drupal.dblog';
. We don't want to overwrite the existing values.We need an upgrade path test for this.
Instead of doing this we should do it like this.
Comment #82
dagmarThanks for your feedback!
Comment #83
dagmar1, 2 and 3 fixed. Let see what the test bot says.
Comment #85
dagmarComment #87
dagmarComment #89
dagmarComment #91
dagmarUnrelated failure: #2828559: UpdatePathTestBase tests randomly failing
Comment #92
yoroy CreditAttribution: yoroy at Roy Scholten commentedThank you @dagmar for staying on the case here. Hope this makes it in soon now!
Comment #93
LendudeAlmost there I think.
This needs to be taken out. Along with the two files in fixtures. The way I read it, a watchdog View is now being added before running the updates and then a check is done if it exists after running updates. If you change the update test to:
The test actually fails because the View already exists before ->runUpdates().
So my suggestion would be to take that out and add the assertNull, it's always good to test 'before' and 'after' states.
Comment #94
dagmarWell, that makes totally sense :). Thanks!
Comment #95
dagmarSome final cleanup.
Comment #96
LendudeWe have tests, we have an upgrade path, we have an upgrade path test. I think this is ready.
@dagmar++
Comment #97
jibranThanks for addressing the feedback @dagmar. Nice work on the upgrade path and tests. @Lendude++ for awesome feedback in #93. RTBC++. Just a minor concern related to post update vs update_N hook here but I think @catch or @alexpott can make it clearer before the commit.
Comment #99
LendudeUnrelated fail in
Drupal\locale\Tests\LocaleUpdateTest
.Comment #100
dagmarI would like to know if this will require a change @catch @alexpot could you clarify that? This issue has been ready for almost 2 weeks and we don't have much more time to put in into 8.3.x.
Comment #101
dawehnerI think a post update is fine here. The system is in a working state before running that particular update.
Comment #103
dagmarComment #104
alexpottI've tested view and update manually. All looks good. A post update is fine. I've backported this to 8.3.x because I see little risk in doing.
Committed and pushed 5987dac to 8.4.x and e2a87a2 to 8.3.x. Thanks!
Some coding standards fixes.
Comment #107
alexpottLooking at the existing CR https://www.drupal.org/node/2084727 - I actually think that this could do with it's own since it has been so long and using the same CR feels wrong since you can't set the release correctly.
Comment #108
alexpottComment #109
dawehnerNice work!
Comment #110
alesr CreditAttribution: alesr commentedNice to see that patch finally coming to D8.
Great work!
Comment #111
dagmarChange record added here, still in draft for a review: https://www.drupal.org/node/2850115
Comment #112
dagmarFirst follow-up #2849797: dblog link broken with > 1000 watchdog entries
Comment #115
alexpottTasks shouldn't be introducing bugs in the alpha stage of a release. Revert thing change to fix #2849797: dblog link broken with > 1000 watchdog entries. Also we need to ensure that the default behaviour when creating a view on log entries does not add a wid column with a thousand separator since that is exactly what happens atm.
Comment #116
alexpottI think both the uid and wid columns defined in dblog_views_data() should have the type of standard and not numeric.
Comment #117
alexpottIn fact perhaps we're going to need to fix #115/#116 in a separate issue first since that is going to need a hook_update_N to fix existing views. :( - glad we found this before 8.3.x though.
Comment #118
alexpottIn fact reviewing this change again I think we should split out all the changes to dblog_views_data() since I think we need to provide an upgrade path for some of them. For example:
If they've set some defaults on an existing view - isn't it now broken?
Plus the uid ought to use the
user_name
filter cause then it'd get auto-complete if you exposed it.Comment #119
dagmarThanks @alexpott here is the new patch with the fixes.
Actually they don't. They are indexed by id like this:
So if you have some defaults, views internally stores the ID of the log level. The only thing that actually changes is the order in the exposed filter and the use of human descriptions instead of machine names.
Addressed the other items in this patch.
Comment #120
alexpott@dagmar but the problem is that the uid and wid fields in dblog_views_data() shouldn't be numeric - there never is a occasion where a thousand separator is appropriate. For either field. Plus uid should be using the user_name filter.
Comment #121
dagmarThanks @alexpott now I see what you mean.
Comment #123
dagmarRemoving the last failing tests now that we don't have the separator anymore.
Comment #124
dagmarSorry, moving again this to RTBC to get @alexpott attention. The functionality provided by patch #123 is the same than committed on #113
Comment #125
alexpottSorry it appears my original commit was premature.
As stated earlier I think this needs to go in another issue because I think this needs an upgrade path for existing dblog views.
Looking at this again I'm not sure we should be doing this like this. If you remove the exposed form you'd lose the icons - that's weird. hook_views_pre_render() is a better choice. See views_test_data_views_pre_render() for an example.
Comment #126
dagmarThanks @alexpott
This patch removes all the controversial modifications to the
dblog.views.inc
file which makes the upgrade path non required. Let me elaborate:wid
andtype
, the standard field handler works out of the box after rebuild the registry.type
filter change, as I explained on #119 the values stored are numeric and they work just fine after the upgrade. In fact the exported version of a view with some selected values on the servery filter looks like this for both scenarios:Please don't see explanation as an act of laziness, I spent 2 hours finding an elegant way to write an upgrade path and another 2 doing several tests to ensure this stills works after upgrade.
Great suggestion! I adapted the code to apply the styles to all views that use the watchdog as base table, so we can have icons in several views that shows logs.
Comment #127
dagmarAlthough no upgrade path is needed, this should be able to be tested. The upgrade path is simple rebuild the registry. I will work on the upgrade path test.
Comment #128
dagmarI will need some help here from the views maintainers: #2851293: dblog is using the wrong views field, filter and relationships definitions
Comment #129
xjmThanks @dagmar.
Comment #130
dagmar#2851293: dblog is using the wrong views field, filter and relationships definitions was committed and pushed to 8.4.x. We can continue working on this issue now. I would like to see others continue this patch so I can review it as new dblog maintainer.
Comment #131
dagmarComment #132
pritish.kumar CreditAttribution: pritish.kumar at OpenSense Labs commentedUploading the reroll patch.
There are 2 changes which were present in the most recent patch which I have not applied.
diff --git a/core/modules/migrate_drupal_ui/src/Tests/d6/MigrateUpgrade6Test.php b/core/modules/migrate_drupal_ui/src/Tests/d6/MigrateUpgrade6Test.php
index 50829f0..8e1f3ab 100644
--- a/core/modules/migrate_drupal_ui/src/Tests/d6/MigrateUpgrade6Test.php
+++ b/core/modules/migrate_drupal_ui/src/Tests/d6/MigrateUpgrade6Test.php
@@ -63,7 +63,7 @@ protected function getEntityCounts() {
'user' => 7,
'user_role' => 6,
'menu_link_content' => 4,
- 'view' => 12,
+ 'view' => 13,
'date_format' => 11,
'entity_form_display' => 15,
'entity_form_mode' => 1,
diff --git a/core/modules/migrate_drupal_ui/src/Tests/d7/MigrateUpgrade7Test.php b/core/modules/migrate_drupal_ui/src/Tests/d7/MigrateUpgrade7Test.php
index 77bc2ab..99b29ee 100644
--- a/core/modules/migrate_drupal_ui/src/Tests/d7/MigrateUpgrade7Test.php
+++ b/core/modules/migrate_drupal_ui/src/Tests/d7/MigrateUpgrade7Test.php
@@ -64,7 +64,7 @@ protected function getEntityCounts() {
'user' => 4,
'user_role' => 3,
'menu_link_content' => 7,
- 'view' => 12,
+ 'view' => 13,
'date_format' => 11,
'entity_form_display' => 16,
'entity_form_mode' => 1,
Comment #134
pritish.kumar CreditAttribution: pritish.kumar at OpenSense Labs commentedComment #136
dagmarThis is the problem reported by the automated test system.
This should not be included in this patch.
This file should not be modified
Comment #137
jofitz CreditAttribution: jofitz at ComputerMinds commentedCorrections to the reroll.
Comment #138
dagmarThanks!
This should be removed.
This should be removed.
This is not necessary anymore.
This is not necessary. We have update functions that take care about this.
Technically new tests should be BTB test now. But we didn't upgrade dblog tests yet. I think is ok to convert them in: #2863372: Convert dblog web tests to BrowserTestBase
Comment #139
jofitz CreditAttribution: jofitz at ComputerMinds commentedComment #140
dagmarSorry I missed this line:
This should be using the new
dblog_types
plugin.I tried the patch using a 8.3.x code with 2000 logs, ran the upgrade path and checked that all work as expected. The only difference between the original log listing and the view listing is the pager that is using a mini pager.
So change the pager from mini to full and the filter plugin from in_operator to
dblog_types
and this will be ready to be committed.Attaching the expected interdiff for the next patch.
Comment #141
jibranI think we should make/keep it mini.
Comment #142
dagmarAccording to #1909474: Use light (lite) pager by default all core listings are now using minipagers. So agree with jibran. Lets keep the mini pager for logs too. Users can configure it to use the full pager if they need it.
The filter plugin change is still required though.
Comment #143
dagmarUpdated the issue summary to reflect the remaining work needed here.
Comment #144
nicolas.rafaelli CreditAttribution: nicolas.rafaelli as a volunteer and at Globant commentedHope this help
Comment #145
dagmarThis is looking really good. There is a small problem in mobile though. All the columns of the view are hidden.
The seems to be:
Responsive should be set to ''
Also,
Severity Level
in the exposed filter should be renamed toSeverity
Comment #146
hanoiiAttached is the patch with that sorted and the interdiff. The responsive fix was set to
priority-high
instead of''
.Comment #148
hanoiiSorry, proper patch now, forgot to `git pull` :s.
Comment #149
dagmarThanks everyone!.
I ran the following tests.
Due the fact all the the tests applied to the default list are applied to the view too. I think this is now ready to go.
Comment #150
dagmarComment #151
catchDo we really want to do enable the view in an update? This is a good case for a 'choose-your-own-adventure' update where we can give people a choice whether to do it or not, but we don't have the infrastructure for that yet. We could also defer the update to a follow-up and have this discussion there maybe.
Why not isset() here? Not sure the specificity of is_null() helps at all.
Comment #152
catchComment #153
dagmarI think is better to have an update that actually warn you this is going to be enabled (and you can disable it later) rather than make this optional, which actually is not optional at all because if you install dblog and views is enabled, the default view will be activated by default without asking you if you want to avoid it.
This is how is currently implemented in filterLogsEntries() I tried to make this as transparent to review. There are a lot of things to improve in the DblogTest that are tracked in other issues. (#2882525: Move suspicious string in DblogTest into a KernelTest, #2848529: Move DbLogTest::verifyCron to a kernel test, #2863372: Convert dblog web tests to BrowserTestBase)
Comment #154
dagmarOk. I think
Why not isset() here? Not sure the specificity of is_null() helps at all.
can be fixed here. I still think this have to enable the view using an update.Comment #155
dagmarSince #2863372: Convert dblog web tests to BrowserTestBase was committed. Now is necessary to modify from which base class is extending the test for views.
Comment #156
dagmarRe-rolled after:
#2867304: Convert web tests to browser tests for migrate_drupal_ui module
#2863372: Convert dblog web tests to BrowserTestBase
Also fixed #154.
Comment #158
dagmarThis patch is a few kb smaller because the
DbLogViewsTest.php
was duplicated in the previous patch.Comment #160
catchThat's a fair point on the update, disabling the view is easier than re-creating from scratch, and it's admin-facing only.
Committed/pushed to 8.4.x, thanks!
ummm just realised this was at code needs review, for some reason I thought it was back to RTBC. Since it was a re-roll and one-line change from the previously RTBC patch, going to leave it in I think, but just owning up to the mistake...
Comment #162
catchMeh reverted, but if someone reviews and is happy with it I'll try to re-review/commit fairly quickly to make up for the false alarm.
Comment #163
mondrakeLooked at it live in the interim phase between commit and revert :)
The two things that jump to my eyes:
Sorting by 'timestamp' descending is confusing if you have multiple entries at the same clock time (happens a lot in development...). I think it would be better to sort by 'wid' descending to to ensure entries are shown in the proper sequence.
no newline
Comment #164
dagmarThanks @mondrake and @catch.
Comment #165
mondrakeGo @catch go 😀 (Re #162)
Comment #166
mondrakeActually no, sorry, I think that the change in #164 for sorting is not enough.
It seems like the table format ordering overrides whatever 'sort criteria' are defined, i.e. the
part should also be set to 'sortable: true', 'default_sort_order: desc' and
to be set to 'default: wid'
Comment #167
mondrakeBTW: I double checked - the header specifying the sorting sequence in
orderByHeader
in the legacyDbLogController
isjust confirming that currently sorting is by 'wid' descending.
Comment #168
mondrakeUploading an export of the watchdog view after changes I made along #166 - works for me. Hope it can help.
Comment #169
dagmarThanks @mondrake. You are right, and actually this means lack of test coverage.
I added the modifications you mentioned and more test coverage to ensure this is working as expected.
Comment #170
mondrakeCan we blank out the 'sorts' section? it is irrelevant as the table format ordering overrides it and bad UX, a non experienced user will change it with nothing happening
Comment #171
dagmarNo, because the default view also allow to sort by those values. Let's defer a full UX of the listing for other ticket. The objective of this ticket is to provide the same functionality we have currently but with a view.
Comment #173
dagmarComment #174
mondrakeOk then, RTBC. There's a small code style issue as identified by codesniffer, but it can be fixed on commit if @catch agrees.
Comment #176
catchGood catch on the sort and thanks for the additional test coverage.
Fixed the code style issue on commit.
Committed/pushed to 8.4.x, thanks!
Comment #177
dagmarThanks everyone. I published the change record.
Comment #179
scarletdrupal123 CreditAttribution: scarletdrupal123 commentedIt is very difficult to read the error message by going to database downloading blob file and then reading it , in case website has broken. Still not able to fix the issue in my website.