Problem/Motivation
File management system is a black-box in Drupal. There a some assumptions related to that and not a lot of people seem beeing aware of them (like #1399846: Make unused file 'cleanup' configurable). Having possibility to check status of files on a Drupal site would make a nice addition and help people to understand what is actually going on.
I believe this would be a nice UX improvement and a starting point for contrib to build on.
We started discussion about this on DrupalCon PDX code sprint and I seemed like there was a lot of interest in this.
Proposed resolution
Create a simple view at admin/content/file that lists files on site with basic info about each file (created date/time, usage, type (temporary, permanent), owner, ...). Additionally add another view that displays detailed usage information for a given file and link to it from file listing view.
Remaining tasks
- create file listing view
- create file usage view
User interface changes
- Adds new page, leaves everything else untouched
API changes
- no API changes
Related Issues
- #1399846: Make unused file 'cleanup' configurable
- #2032893: Deprecate the function _views_file_status()
| Comment | File | Size | Author |
|---|---|---|---|
| #136 | 2005166_136_follow_up.patch | 756 bytes | slashrsm |
| #136 | interdiff.txt | 288 bytes | slashrsm |
| #134 | 2005166-follow-up-134.patch | 468 bytes | fubhy |
| #129 | 2005166_129.patch | 36.29 KB | slashrsm |
| #129 | interdiff.txt | 978 bytes | slashrsm |
Comments
Comment #1
slashrsm commentedAdd some tags....
Comment #2
gábor hojtsySounds like a view with some bulk operations. I think discussing/perfecting the view in itself is valuable even if we don't immediately add bulk operations. Who wants to take this on?
Comment #3
slashrsm commentedI can work on that. Will try to prepare a patch in a day or two.
Comment #4
xjmComment #5
xjmSee also: #1823450: [Meta] Convert core listings to Views
In particular:
#1851086: Replace admin/people with a View
#1895160: Convert admin/content to a View, keep a non-views fallback with no bulk operations
Comment #6
andyceo commentedGreat Issue! It is very handy function for site administrators.
Please have a look to this related projects: https://drupal.org/project/auditfiles and https://drupal.org/sandbox/andyceo/1131236
From this projects you can find that main problem with files is sync between database and file system.
If I can help with something, please let me know.
Comment #7
slashrsm commentedThis is definitely not a complete solution, but it is a first step.
Comment #8
slashrsm commentedAdded some filters and empty text.
Comment #9
gábor hojtsyGreat start! Here are a few quick screenshots.
1. No files available. This looks like would need a message or something:
1a. This is how it looks like when the content list is empty:
(We'll not have an add file action here, but an empty table with a message looks like in order :) (Also operations :)
2. Once I had a few files:
(Can we somehow get Views to put a limited size preview into the first column above / below the file name if the file is previewable (with image styles)? If that does not apply, only the filename would show. Would that be too much of a jumble?)
Comment #10
yannickooWe could check whether that file is an image, nice idea! That would looks more like a media library. What do you think about a table and a grid style? What opertations would make sense? File name (contains), mime type, file size and uploaded date or should we expose every column?
Comment #11
yannickooOh, seems like we already have a mime type and status filter.
Gábor, which patch did you use? :o
Comment #13
gábor hojtsyThe two patches were posted with only 16 minutes between them :) I used the first one, did not expect such a quick update.
Comment #14
gábor hojtsy#8: 2005166_8.patch queued for re-testing.
Comment #15
slashrsm commentedWe could check mimetype and display thumbnails for image/*. We could add new field plugin that will handle that. Will think about that and propose some code.
I would not go into grid view as I think that exceeds scope of a "simple file listing". Specially taking close freeze date into consideration.
Comment #17
dawehnerNice idea!
Comment #18
yannickooOkay that's a good point. A contrib module could provide another view which provide a grid view.
Comment #19
klonosI though that this was the whole point of converting admin listings to view: admins can edit them and add/remove fields at will - we simply give them something to start with and only account for what novice users would expect out of the box ;)
Great job so far everybody!
Comment #20
gábor hojtsyLooking at #9 as well. I think its strange that the node overview has the table shown (with the message in the table) but the file does not. I know we are not very consistent about this elsewhere either. Showing the table seems to inform the user more about what will be available here once there are files.
Also, the table shows up very close to the exposed filters. Does the node table have custom CSS to work around this?
Do we want to put in bulk operations or is that not a good idea given you are supposed to operate on these files via the entities they are attached to? Maybe we want to add some help text to that effect then? (I think the usage count would also need some explanation in there).
Comment #21
tim.plunkettThere is a setting in the table settings, "Show the empty text in the table".
Or add
empty_table: '1'to the export under the table settings.Comment #22
slashrsm commentedUpdated patch. Added is thumbnail and empty text now displayed in table to mimic content listing behaviour.
Margins in content listing are added by bulk operations stuff, which we do not have here. I also think that we shouldn't add bulk operations as core relies on fields to take care about files. What kind of operations could we support after all?
Comment #24
dawehnerYeah we probably need tests as well.
Comment #25
slashrsm commented#22: 2005166_22.patch queued for re-testing.
Comment #26
yannickooI think we should make it empty instead of displaying "N/A" - what do you think? I imagine if I would have lots of documents and the thumbnail column is always "N/A". It would be really cool if the thumbnail column only appears if there is an image in the table but that is not possible with Views, right?
I just created a node with files (and images) and right after the node was saved I got three files in the files overview but after deleting the node the file overview still shows the three files. Why are the files not deleted?
Is is the normal behavior that I get redirected to a non-overlayed page after submitting the exposed filters?
A cool feature would be to see *where* a file is used and not only how often...
Comment #27
slashrsm commentedFiles are only marked as "temporary" after beeing deleted from node. Cron will later delete them. You can get more info about that in #1399846: Make unused file 'cleanup' configurable.
Agree. I was planning to propose another view, that would display list of usages for a given file. We could link to that listing from this view. What do you think about that?
Comment #28
gábor hojtsyIs showing where it is used making this table too busy or too slow or to wide or all of them? :) If we want to make the usages appear in a different table, maybe we want to have a View with an argument and the usage count link to that view with the proper file ID filled in, so it can look up usages for that file only?
Comment #30
slashrsm commentedUsage info joins file_usage table which results in N rows for a single file (N == number of usage entries for that file). I created another simple view for usage information that uses argument and linked to it form main listing.
Also applied SUM() over usage in main listing count to display only one row per a file.
Comment #31
yannickooYeah! The File entity does not display the entity ID but it displays the entity label linked to the entity URI. I think this is cleaner than display the entity ID. Why do you display the module and what is the module? It's not the module which provides that file type because it would be file in every case.
Comment #33
slashrsm commented@yannickoo: File entity does not use views to create this listing. In order to display label we'd need to dynamically create relationships to base entity tables of entities that appear in usage list. This seems very difficult. I'm not even sure if it would be possible to do this with views.
Module column displays name of module that registered this usage. This can be useful if you experience usages that should not be there and you want to see which part of the site produced them.
Comment #34
gábor hojtsyWe are getting there! Some notes:
- The clickable number provides very little click target to get to the usage list. Can we make it '1 use' or something like that?
- We should mark a few columns as less important for responsive tables, so we can see this in smaller screen spaces in a condensed view.
- The usage screen is *very* technical with the entity name, id, etc. Agreed with @yannickoo there, but I see the technical limitations. Maybe we just want to include the full entity URL as clickable? (Also improves click target) vs. just the entity name and id.
- Also a preview of the file (if possible) and the file name would be great to display on the usage table in views header text. This should be possible with tokens(?) - so we first see what are we getting info about and then the usages.
Views is so powerful :)
Comment #35
gábor hojtsyBTW current screenshots (admittedly the filename was in the page title already, but overlay makes it hard to recognise that :D)
Comment #36
yannickooDo we have to fix that behavior that after submitting the filters we get a non-overlayed page?
Comment #37
gábor hojtsy@yannickoo: existing bug at #1116326: Support admin overlay in exposed forms.
Comment #38
yannickooGábor what do you think about adding more filters?
Does anybody of you know whether a "replace file" functionality will be added to core?
Comment #39
gábor hojtsyI don't think we should add or anticipate more features here. Like replacing files. We already direct people to find out where the file is used, they can go edit there. This can be extended by contrib. I could personally use more filters and ordering options on columns, but there can be any number of complexity for filters. I can imagine people wanting to search for files above certain sizes to clean up big junk files or other similar things, but they can now customize this. Its all views :D So we don't need to tailor this view for very specific use cases. Maybe one or two more filters would be useful, but it would quickly get confusing if we overdo it IMHO.
Comment #40
yannickooGood point - I totally agree with that!
Comment #41
slashrsm commented- Added some tests
- added "usage/usages" suffix
- configured responsive priorities for columns
- removed "N/A" from thumbnal
- configured thumbnail column to hide if empty
I was thinking about how to provide better info in usage view. We could create new field plugin that would separately load referenced entity and display title/label/name. Also URL would be constructed much more reliably this way.
Downside is that we add some overhead (need to load N entities) by doing this.
Comment #42
slashrsm commentedWas also thinking about the "Module" at usage page. It is definitely useful to have that information listed, but column header does not explain what this data means at all. I was thinking about different column header, but I'm not sure what would be the best solution here. Something like "Related module", "Referencing module" or "Registering module"?
Comment #43
dave reidShowing detailed usage information is out of scope for this listing. If anything let's just show a usage COUNT() in a column. I'm not really happy about adding images either. We're trying to make a poor-man's media library with this View which was against the intentions that I had for this. It would be easier just to keep it simple with a table display and not trying to 'display' any files, and leave the nicer view for contrib and admin/content/files/thumbnails.
Comment #44
ParisLiakos commentedi am against adding a thumbnail as well..the usage thing looks good though
Comment #46
klonosI too think that this should be kept minimal. I don't mind the thumbnail because it's useful when people start naming files image1.png image2.png, etc, but if digging through the file usage shows admins in which nodes (title - not entity ID) the file is being used, then I could live without it.
The cool thing about these pages is that they are views now and admins can tweak them as they see fit. What I would like to see once we are finished with converting all these pages to views is a "Edit this display" link (perhaps at the bottom right of the table) directly linking to the view itself in edit mode.
Comment #47
klonos...
Comment #48
ParisLiakos commentedthats what contextual links would do if they where not removed for admin views..
Comment #49
klonosYeah, do you have any idea why they were removed in the first place re patrida (perhaps link to a related issue/comment)?
Comment #50
tim.plunkettThere is a setting to not show them in the UI. It was turned off at webchick's request for the admin/content and admin/people views.
Also, if you have the ability to edit an admin view, you probably know how to get to the Views UI.
Comment #51
gábor hojtsyI think the usage information is important. Otherwise this is just a page listing things with no way whatsoever to act on them. You cannot act on files themselves, you can only act on them through their "holder entity" essentially. You cannot remove, edit, whatever files direct. So if there is no information and way to lead people to the place where you can act on the file, there is not really a point in having this list I think. All other admin content lists let you act on things, they are not just "here, this is data on your site, but good luck trying to figure out where you can do something with it".
Comment #52
yannickoo#51 +1
Comment #53
olamaekle commentedMaybe we can have 2 displays, one table listing and one thumbnail grid to preview images?
Comment #54
klonosI know, but having a "Customize this display" link available that takes you directly to edit mode spares you from 4-5 of extra clicks and the need to search through the list of views (some sites have quite a few and #1475204: [META] Provide a generic search/filter UI interface pattern for listing pages is not moving).
Comment #55
tim.plunkettPlease open a dedicated issue for that. Both of the 2 views in core have contextual links turned off, and this should follow suit.
Comment #56
ParisLiakos commentedopened #2020265: Bring back contextual links for admin views
Comment #57
klonosThanx ;)
Comment #58
slashrsm commentedTests should be fixed now.
I completely agree with @Gabor Hojtsy about file usage information. I think it is very important information to show, we only need to figure out how to make it less technical and informative enough for every site administrator.
Comment #60
slashrsm commented#58: 2005166_58.patch queued for re-testing.
Comment #62
slashrsm commentedReroll that will hopefully also fix test.
Comment #63
gábor hojtsyTesting this it does not look any different visually than it did before. Let's figure out how to make the usage info more useful.
@Dave: I'm personally fine either way to push arguing about a thumbnail into a followup issue and remove that column for now.
Comment #64
slashrsm commented- thumb removed
- file usage view improved (displays entity label that links to entity page - using custom field handler)
Comment #65
tkoleary commented@slashrsm
I think this needs a link in the menu as well. Otherwise it's not discoverable enough.
Comment #66
tkoleary commentedThat's not optimal from a UX perspective. It should just be another column in this view. We could remove one as well, for instance mime type, since you have the suffix in the file name already.
Comment #67
gábor hojtsy@tkoleary: A file can be used at multiple places, which is what the whole usage counter is about. Are you proposing a file should have multiple rows in that case or only show the first use of the file? Both sound problematic.
Comment #68
tkoleary commentedThe heading is awkward as well.
My suggestion would be
thead: USED IN
td: 1 place
Comment #68.0
tkoleary commentedAdd comment about PDX sprint.
Comment #69
slashrsm commentedHeading and formatting of usage col modified as suggested in #68.
Comment #70
tkoleary commented@Gábor Hojtsy
That's right. I misread the comment. It should link to a new view.
Comment #71
tkoleary commented@slashrsm
Mime type text field should have an autocomplete. Mime types are not necessarily common knowledge.
Comment #72
tkoleary commented@slashrsm
Used in count does not appear to be functioning properly.
Comment #73
dawehner@tkoleary
Thanks for spotting that, but this is not a problem of this issue. can you please fill a different bug report for it?
Please don't let us renamed displays, ... and just use page_1 instead.
Should this be better "file" instead of views?
See above.
Are there other places in core with such a dynamic description? This one here feels really wrong, because it could easily lead to infinite recursion after some refactoring.
Please add typehinting ...
Needs @contains
@inheritdoc.
I don't think we need bool anymore in Drupal 8.
Instead of loading every entity you could also write a preRender method which collects the entities per entity type and load the in bulk.
Shouldn't we better use the views build in link functionality instead of calling l() directly?
Should be better $this->sanitizeValue($entity->label());
Contains ...
Needs docs.
Comment #74
slashrsm commentedI took this directly from "access content overview" permission. See: http://drupalcode.org/project/drupal.git/blob/refs/heads/8.x:/core/modul...
Other comments fixed. I did not find any example of @contains usage in core. Please tell me if I got it wrong.
Comment #75
slashrsm commented@tkoleary: What exactly do you think is wrong with usage count display? I agree with @dawehner about autocomplete field for mime. We are using standard handler here which can be used in any view, which apparently expands over the scope of this issue. Could you open new one and we can work on that separately?
Comment #76
dawehnerThank you!!
Let's use protected instead as people might want to extend the class. ... Should be also camel case, if possible.
Instead of calling entity_load_multiple directly you could also inject the entity manager into this class and use the storage controller of them.
Should be @param array ... An array of file usage.
Comment #77
slashrsm commented@dawehner Thank you for your great reviews and your help with DIJ questions.
Here we go with the patch.
Comment #78
dawehnerThere should be a follow up which tries to move this into a autoloadable place.
Just another nitpick: Should be "Contains \..."
Is referncers a valid english word ^^
Let's move the constructor to the top if you are here anyway... I guess we need do document at least the entity manager.
Afaik we typehint with int instead of integer.
This is quite intensive test coverage.
Can't we use entity NG api now?
This seems to be better implemented as $this->xpath.
Comment #80
tkoleary commentedRelated #2022297: [META] Unified toolset for Views in core
Comment #81
slashrsm commented#77: 2005166_77.patch queued for re-testing.
Comment #82
gábor hojtsyTried the views out again! The only quick feedback I'd have is its strange you can filter for the mime type but not the name of the file. That sounds like would be an easy addition :)
Comment #83
slashrsm commentedSuggested filename filter added and fixed most of the comments from #78.
http://www.thefreedictionary.com/referencer
I used this code, which returned correct elements, but assert still failed.
Comment #84
tim.plunkettThe one in the patch is spelled wrong.
Comment #85
dawehnerMaybe I am stupid but in this spelling it just feels wrong: "Referncers"
Comment number 65 was not adressed yet.
Additional on admin/content/files/usage/1 I think there should be also some breadcrumb/local task going on.
Fixed the test to use xpath, assertFieldByXPath() is just for input fields.
Comment #86
slashrsm commented@tim.plunkett: Thanks for pointing that out.
Comment #87
slashrsm commentedCrosspost :) Our interdiffs combined.
Comment #88
slashrsm commented#65 probably needs hook_menu() entry? Or is there any other way to achieve that? The same with breadcrumbs for file_usage?
Comment #89
tim.plunkettIf you build the view correctly, it will add the hook_menu you need.
Comment #90
slashrsm commentedI configured menu tab (that appears correctly) and configured "Administration" as menu, but link still didn't appear in Administration menu. What am I missing here?
Comment #91
tkoleary commented@slashrsm
Jessebeach will know that. Maybe ping on IRC
Comment #92
tstoecklerThis should be removed from the export, I think.
I see nothing file-specific in here, is there any reason this is not in views.module directly? Or entity.module for that matter?
Comment #93
slashrsm commentedUUIDs removed and field handler generalized and moved to views namespace.
Comment #94
slashrsm commentedIt looks like #65 is currently not achievable due to Views limitation. See #2027043: Allow page displays to be both local tasks and menu items.
I suggest that we commit this when ready and fix #65 as a follow-up once #2027043: Allow page displays to be both local tasks and menu items is fixed.
Comment #95
dawehnerWe don't remove uuids from exports in core anymore.
Comment #96
slashrsm commentedI looked at content view and did not find it there, so I removed it here also. Rerolling.
Comment #97
tstoecklerOops, sorry! I thought we did. Thanks for clearing that up.
Comment #98
yannickoo@dawehner and why we don't remove them anymore?
Comment #99
dawehnerBecause it just makes sense to provide a UUID, so that you always can no that the view you are dealing with is the one coming from the original provided one.
Comment #100
dawehner#96: 2005166_95.patch queued for re-testing.
Comment #102
slashrsm commentedThis should fix 'em...
Comment #103
gábor hojtsyComment #104
dawehnerHere is a final nitpick review before setting to this to be ready! What a huge issue!
Let's document the types.
Should be "Contains \"
"Constructs a EntityLabel object.".
the variable name should be at the end.
Let's use multiple lines for adaption later.
Should be "{@inheritdoc}"
Afaik both should be public.
Nice, simple and performant code.
Comment #105
slashrsm commentedAll things from #104 should be fixed.
Thank you all for your great feedback!
Comment #106
dawehnerThank you very much! I am so glad that people took over these conversions, as this is the only chance to get them done.
Comment #107
yesct commentedThis issue was RTBC and passing tests on July 1, the beginning of API freeze.
Comment #108
damiankloip commentedIt seems strange to have a function called _views_file_status in file.module. Also, it would be nice if this could be replaced with an OO equivalent somehow ....
Comment #109
slashrsm commented@damiankloip: It was already suggested in #78 to fix this in a follow-up. This function was there before, I just moved it from views.inc to .module since it was not autoloading properly.
Comment #110
damiankloip commentedok, if there is a follow up. Great. Is there?
Comment #111
yannickooWhy we don't just return an array like on other (core) modules? It's the same but it's confusing :/
Comment #112
slashrsm commented@damiankloip: #2032893: Deprecate the function _views_file_status()
Comment #112.0
slashrsm commentedUpdate issue summary (file usage view, progress update).
Comment #113
slashrsm commentedReroll against latest HEAD.
@yannickoo: Are you sure this is some kind of a sandard? I can see bot approaches among core modules.
Comment #114
yannickooThe node module needs that because it manipulates the permission but other modules like user, views, tour, toolbar, statistics, aggregator, contextual, language, locale and menu just return an array :)
It's funny to see that that taxonomy module uses
$permissionsinstead of$permsbut we should only put the permissions into a variable if we want to manipulate them.Comment #115
dawehnerIt doesn't really matter from my point of view, so let's do whatever we have now.
Comment #116
wim leers#113/#114: like #115 says, it doesn't matter. There's no standard for this, and it'd be silly to have a standard for it. What matters is what is returned. Many
hook_menu()implementations in D6/D7 also did something like$items = array()first, while they could just as well have donereturn array(…). It really doesn't matter at all.Comment #117
yannickooAs I said above it's just confusing... $perms, $permissions, return array().
Comment #118
wim leers#117: it's not confusing at all. It just requires a very tiny additional amount of code reading.
Comment #119
slashrsm commentedCompletely agree with @dawehner and @Wim Leers.
Comment #120
alexpottLet's put
$uri = $entity->uri();inside the if as we only use $uri there...As we're changing code here... let's not use a variable name $loads... and lets use loadMultiple if $ids is an array of ids...
Comment #121
slashrsm commentedFixed.
Comment #123
gábor hojtsy@slashrsm I think you went a bit overboard with integrating the method call and the array lookup :) @alexpott merely asked to move the method call inside the if(), not to merge it with the array lookup :)
Comment #124
slashrsm commentedOh :)
Comment #125
dawehnerCan we find a variable which refers to entity ids per entity type, but I can't think of a proper one.
Comment #126
slashrsm commented$entity_per_type? :)
Comment #127
klonosNo no: $entity_id_per_type ;)
Comment #128
dawehnerGood idea, so $entity_ids_per_type ?
Comment #129
slashrsm commentedComment #130
gábor hojtsyLooks good, fixes all @alexpott's concerns.
Comment #131
alexpottNice... views really is a great addition to core :)
Committed 8cf7830 and pushed to 8.x. Thanks!
Comment #132
slashrsm commentedGreat! Thank you all for your help!
Comment #133
klonosThis is great! Do we have a follow-up for a menu item (I don't want to file any duplicates)? Back in #65 we started a discussion over this, but never filed an actual issue for it. According to #94 that task is blocked on #2027043: Allow page displays to be both local tasks and menu items.
Comment #134
fubhy commented"Contains of" is that our new standard :P
Let's fix that in a follow-up right away because I bet we would miss that when we convert the left-over "Definition of" stuff with a regex search&replace.
Comment #135
ParisLiakos commentedlets fix those too..should be 644
Comment #136
slashrsm commentedHere we go.
Comment #137
dawehnerSeems legit.
Comment #138
alexpottCommitted 6c117d6 and pushed to 8.x. Thanks!
Comment #139
klonosGreat, thanx! What about my question in #133 about a menu item?
Comment #140
gábor hojtsy@klonos: let's open a followup and postpone on #2027043: Allow page displays to be both local tasks and menu items. We cannot solve it as-is yet without that pre-requisite.
Comment #141
slashrsm commentedThere is a follow-up ticket for that that is still in progress AFAIK.
Comment #142
klonosDo you mean there's a follow-up for adding a menu item, or a follow-up for #2027043? If there's a follow-up for adding the menu item, could you please post a link? Thanx.
Comment #143
slashrsm commentedYou are right. I was thinking about #2027043: Allow page displays to be both local tasks and menu items. There is currently no issue to add that link to this view.
Comment #144
klonos#2047893: Add menu item(s) for the new admin/content/file view
;)
Comment #146
star-szrCross-linking with a tiny followup issue: #2059699: Remove t() from test assertion messages in file.module
Comment #146.0
star-szradded the issue asked for in 109-110, 112
Comment #147
gábor hojtsy