Sandbox link:
https://www.drupal.org/sandbox/fietserwin/2702627
Git clone command:
git clone --branch 7.x-1.x https://git.drupal.org/sandbox/fietserwin/2702627.git duplicate_images
About this module:
The project page tells all there is, but in short this module allows administrators/editors to remove duplicated images (or for that matter any type of document) via a wizard like form that guides the user through the process.
Points that have gotten attention while developing this module:
- Security:
- introduced a separate permission to run the form
- most form fields are checkboxes. There are 2 textfields; 1 is casted to an int (thus safe), the other is used in comparisons only, not outputted (safe as well
- Translation: all texts go through t()
- Duplication: I did search but did not find any module that does the same.
- coding standards: are followed
- Install/uninstall: no tables or variables are stored, so there's no (need for an) install.php.
About me (my profile page):
- D.O. member for over 6 years.
- Co-maintainer for imagecache_actions, (image_effects,) availability_calendar.
- Lots of patches for other contrib modules.
- Patches and many reviews for Drupal core (D7 and D8).
- But still not the right to create projects myself, as I always collaborated on other projects...
Comments
Comment #2
PA robot commentedWe are currently quite busy with all the project applications and we prefer projects with a review bonus. Please help reviewing and put yourself on the high priority list, then we will take a look at your project right away :-)
Also, you should get your friends, colleagues or other community members involved to review this application. Let them go through the review checklist and post a comment that sets this issue to "needs work" (they found some problems with the project) or "reviewed & tested by the community" (they found no major flaws).
I'm a robot and this is an automated message from Project Applications Scraper.
Comment #3
madan879 commentedHi,
Please fix the basic issues with your codes are listed below (http://pareview.sh/pareview/httpsgitdrupalorgsandboxfietserwin2702627git):
EDIT: removed long pareview.sh dump
Comment #4
fietserwinBusy updating the documentation.
Comment #5
fietserwinCorrected the documentation.
Remaining remarks are either:
- incorrect (e.g. split array).
- exceptions to the rule (passing text to l() through t()).
- kind of outdated (I insert PhpStorm ignore tags to get my code warning free (and to explicitly document warnings), but from Drupal I may not insert /** @ignore... */ tags in my code.
- many param names really speak for themselves and don't need another line to repeat that name as documentation. The type info is more important as that, besides the name, also indicates if I pass a image style name or image style object.
Comment #6
REDrupalPlugin commentedUsing Coder I found the following issues (learn more about coder and coding practices at https://www.drupal.org/project/coder
EDIT: removed long Coder dump, see http://pareview.sh/pareview/httpgitdrupalorgsandboxfietserwin2702627git
Comment #7
arulraj commentedAutomated Review
Found more issues on pareview.sh. Follow link: http://pareview.sh/pareview/httpgitdrupalorgsandboxfietserwin2702627git
Manual Review
[Suggestion:
1. Add filter_input for $_GET
2. Add t() function for links.
3. After proceed search duplicate steps am getting list of errors.
a.Warning: scandir() [function.scandir]: Unable to find the wrapper "private" - did you forget to enable it when you configured PHP? in DuplicateImagesSearch->search() (line 192 of sites\all\modules\duplicate_images\DuplicateImagesSearch.php).
b.Warning: scandir(private://,private://) [function.scandir]: The filename, directory name, or volume label syntax is incorrect. (code: 123) in DuplicateImagesSearch->search() (line 192 of \sites\all\modules\duplicate_images\DuplicateImagesSearch.php).
c.Warning: scandir(private://) [function.scandir]: failed to open dir: No such file or directory in DuplicateImagesSearch->search() (line 192 of \sites\all\modules\duplicate_images\DuplicateImagesSearch.php).
d.Warning: scandir() [function.scandir]: (errno 2): No such file or directory in DuplicateImagesSearch->search() (line 192 of \sites\all\modules\duplicate_images\DuplicateImagesSearch.php).
e.Warning: Invalid argument supplied for foreach() in DuplicateImagesSearch->search() (line 197 of sites\all\modules\duplicate_images\DuplicateImagesSearch.php).
4. Implement hook_help in your code.
5. Add image preview for Results updates section also to avoid accident deletions
If added, please don't remove the security tag, we keep that for statistics and to show examples of security problems.
This review uses the Project Application Review Template.
Comment #8
fietserwin#6 and #7: Thanks for your reviews I did not follow up all your points,so I'll explain what I did and why I didn't do some things:
RE #6: Yes, coder does till show a number of warnings and errors, but as I explained in #5 I don't deem these necessary to solve for the reasons outlined there, but I will repeat them here;
RE #7.1: I didn't know that PHP function, but I can only find 1 use of it (in the token contrib module) in the whole of my local dev/test Drupal installation. The step/op as I use it was copied from update.php which also does not filter it. Moreover, there's no need to as the step is never output nor stored, only compared to a predefined list.
RE #7.2: I don't understand. You cannot translate Drupal paths, a translator cannot determine that the path 'admin/config/media/colorbox' should become 'fr/admin/config/media/colorbox' for French admins as the language prefix is site specific configuration (it can even be a different domain name). I could use the menu item name, but a plain 'Colorbox' tells users less because it does not make clear in what submenu it is placed. Note: I did add a check if the user may access the Colorbox settings page before displaying this link.
RE #7.3: Good catch! I did not test without the private path setting defined. I solved this bug.
RE #7.4: hook_help: even ctools, views, imagecache_actions and many of the other mostly used contrib modules do not define this.
RE #7.5: I did add it, though it seems a feature request to me and thus should not necessarily be part of this code review.
I hope you can understand my reasoning and that it suffices for the application review.
Comment #9
skaughtjust to add in see Sanitization functions | D7 The most general way to protect any input ($_GET or $_SERVER etc..) var in best drupal etiquette should be at least passed through
check_plain()return isset($_GET['op']) ? check_plain($_GET['op']) : 'intro';although, there should be a better way than using GET in general.
a quick question/point
duplicate images may exist in drupal because of node revisions or use of fields with other entities -- which a site may have the 'same image' specifically because of this. are you backtacking the type of use and giving notice to your image admin. (perhaps this is a feature request).
Comment #10
skaughtin term of
l()use.l(t('English name'), 'your/path/to-something')it's the $title that needs t() -- yes, it's funny that drupal doesn't automatically do this..but, it doesn't....
as a personal note... Coder module is out of date (and a bit out of fashion) -- http://pareview.sh should be followed, for sure.
and of course, you're module shouldn't have ANY generic php errors. that's just bad work. comment#7 reports stand apart -- enable watchdog if you don't have a generic php console.
Comment #11
skaughtyou should have a
configure=admin/config/media/duplicate-imageson your .info -- as well as hook_help to outline what you module does (and doesn't, ie: monitor revision).. and where to go.and currently the entire module is giving WSOD level errors.
with other errors as well.
please fix these before marking as 'needs review' again.
--head commit: April 16, 2016 8c4afec422ed1a41c38d244b1be3c319835da36e
Comment #12
skaughtnow i see why you're using $_GET...
Please look into Multistep forms in drupal... it would be a better UX flow for your module than it's current direction
https://www.drupal.org/project/examples does have a basic sample... but there are several ways to do it..
maybe even --> hook_menu allows for 'tabs' which also maybe batter for what your doing. http://drupal.stackexchange.com/questions/37262/create-tabs-with-hook-me...
Comment #13
skaughti've spend a bit more time going over the errors, it seems one side of issues comes from images that have already been deleted from the file system, but still exist on the managed_file table. You'll need to introduce some pre-flight checks to make sure those files do actually exist.
As you are using this Task-Lisk theme item as a step wizard it is awkward as the user can not back up on a step, nor reset to beginning.
In your step intro page i think you should be clearly recommending to users to use Backup_migrate to backup their file system before use -- as your module does effectually delete live content items (thinking of 'an average admin/content editor' -- they will never know 'which is the good file' this way).
Comment #14
fietserwin@SKAUGHT: thanks for your in-depth review and remarks. I will respond to them in order here.
re #9:
- Sanitization functions are used on output, not on input. So IMO these are not needed here. And please remember this code is copied from existing Drupal core code , in this case the update.php script.
- The type of usage is shown during the process, ie the fields of the entities that refer to the image or to the managed file record are listed after the usage search process.
re #10:
- As this is an admin only module, I did not make up an 'English name' for paths. But to satisfy pareview in this case, I changed it to 'Colorbox settings' instead of the path to it. The name of a (contrib) module should be passed through t(), so that was indeed an error. All these warnings should be gone now.
- The module does not have any generic php errors, all warnings stem from the fact that the back button indeed (as you found out later) does not work.
re #11:
- I find the configure line in the .info doubtful as it is the main (and only) module's page, not its settings page. But I added it anyway.
- IMO hook_help is highly overrated because, as said in #8, not many modules do implement it, but to get rid of these remarks, I added it.
- All these errors stem from using the back button which, apparently, is not supported by the form caching mechanism, or by just starting at a step. The module does not support going back, so errors may be expected. Furthermore going just to a step in the middle of the process can not be done via the menu, so yes, if anyone wants to abuse the module, errors may be expected. For me the question of which situations to catch is "Can a normal user, via the UI, arrive at this point in code with this state?" If so, the situation should be checked and an clear error message should be given. We are not developing a highly dependable embedded system here, but a use once tool for admins, so error handling can be relatively minimal.
re #12:
- As not being able to use the back button bothered me as well, especially during testing, I will look at the multi-step form example. Expect an update on this in a few days.
re #13:
- I don't get how this can happen. The process starts with looking at duplicate files and only then searches for managed file records that reference those files, so they will exist. In the last step, i use the core API function file_delete(), so it should be the API function that checks for this, not this module.
- Back button: already answered
- Warning added (proudly copied from what B&M does on the update.php form).
Thanks once more.
Comment #15
skaughtre13
file function are failing due to the file not actually being there. When the routine is in the process of deleting a FID.. you need to pre-check to make sure the file is actually there (if not.. routine should then just
break;out of that one delete step..and change to just a removal of the FID from the file table.i might think part of the problem is drupal's File API itself... as there should be an Exception thrown (a good old lack of
try{}catch{}kind of thing), so it's up to "use dev's" to work it through.i've had this happen before myself. (: The FIle API has some annoying holes in it.
just to note a typical reason why that file is missing [an edge case situation]:
a user may have deleted it from a WYSIWYG/File manager that wasn't connected to the Field/managed File that originally uploaded it.
in my situation.. i've testing in a local copy of my own portfolio site. which is a drupal project that's be running for a couple years now. I have deleted various files out of the file system, cuz i know there are dead files...
Comment #16
skaught@arulraj -- hopefully your tracking still. Could you clarify your testing at all to verify any similarity to my #15 examples?
Comment #17
fietserwinre #15: As the process starts by looking for physical files that are duplicates of each other, and only after finding these searches for the corresponding managed file record, it can not (or at least should not) get into the situation that it tries to delete a record in the managed files table for which the actual URI does not (or no longer) exist. What exact messages did you get?
Aside: to me this seems like we are getting into discussions about features and normal/minor bugs. Both should not be part of a Project Application review and should certainly not block applications, also see PAReview: Review Template.
The same holds for the still standing automated review remarks:
And that (the latter part of the quote) is what I, IMO, certainly did.
Comment #18
skaughtwhen error occurs user gets WSOD -- app breaks. no screen message for UI user. logged in watchdog/php console.
Certainly, this could be considered 'an issue'. meaning module could be released 'as is'.. But...it is a big enough issue that will occur for general Users [of course, It won't be my project issues messages to deal with]. Myself, I would release a module with this standing problem.
The UX notes (either multistep or tabs..) are pure suggestion, of course.
as such, i've filed your first issue: https://www.drupal.org/node/2717987
Comment #19
fietserwinHaving fixed #2717987: WSOD level break during delete process, the project is ready for review again.
Comment #20
nico.knaepen commentedThere are still issues when validating through pareview, check http://pareview.sh/pareview/httpgitdrupalorgsandboxfietserwin2702627git
And some other minor coding standards issues:
duplicate_images.info
Add spaces before and after "="-sign
Add package information to have the module shown in a correct group (ex. "media" or "image")
I do like the fact that you are working OOP. However I would recommend to already work out your module partially how Drupal 8 modules are structured. Just to save you some time in the migration process once the module is going to be ported to D8. This can simply be done by using the xautoload module. Then it's only a matter of moving your PHP classes to the src subfolder.
Comment #21
fietserwinRE #20: thanks for reviewing.
1) See quote above (from PAReview: Review Template):
The remaining warnings have a reason behind them to defer from the standard.
2) Done, good catch
3) I prefer not to add another dependency, as i want this module to have a smooth install, run and remove experience. Moreover,I guess that form classes in D8 will be quite different anyway.
Comment #22
ravimane23 commentedI can see there are many errors in PAReview : http://pareview.sh/pareview/httpgitdrupalorgsandboxfietserwin2702627git
Some of them listed below :
/duplicate_images.module: all functions should be prefixed with your module/theme name to avoid name clashes. See https://www.drupal.org/node/318#naming
function duplicate_help($path) {It should befunction duplicate_images_help($path) {Coder Sniffer has found some issues with your code (please check the Drupal coding standards).
Comment #23
ravimane23 commentedComment #24
fietserwin- function duplicate_help($path): good catch: corrected.
- As reviewers seem to be obsessed by pareview, I did some RY (whereas DRY should be the standard). Remaining remarks are due to PHPStorm comments or errors in the review code itself. You can safely ignore these!
So please now review what reveiwers should review: security, duplication with other projects, possible licensing issues (Project Application Review Template).
Comment #25
nikhil banait commentedHi @fietserwin,
This is very nice and useful module.
Manual Review
Individual user account
Yes: Follows the guidelines for individual user accounts.
No duplication
Yes: Does not cause
Master Branch
Yes: Follows the guidelines for master branch.
Licensing
Yes: Follows the licensing requirements.
3rd party assets/code
Yes: Follows the guidelines for 3rd party assets/code.
README.txt/README.md
Yes: Follows the guidelines for in-project documentation and/or the README Template.
Code long/complex enough for review
Yes: Follows the guidelines for project length and complexity.
Secure code
Yes: Meets the security requirements.
Coding style & Drupal API usage
There are still some minor coding standard issues. Please fix.
Pareview status: http://pareview.sh/pareview/httpsgitdrupalorgsandboxfietserwin2702627git
This review uses the Project Application Review Template.
Comment #26
fietserwin@Nikhil Banait: thanks for reviewing.
Minor coding standard issues are not a reason to block an application, see this quote from the (also above mentioned) documentation page:
Moreover, remaining issues are either due to errors in the automated pareview (I don't have multi-valued arrays on 1 line, it is a single valued array in a multi-parameter function call) or due to the fact that I use PHPStorm comments to suppress warnings from that IDE on specific places. So I cannot solve these (non-)problems without giving up my green lights in my (and many other Drupal devs) preferred IDE or forcibly rewriting otherwise correctly formatted code.
Given that all other points are answered with a YES, IMHO, this can be set to RTBC, please.
Comment #27
nikhil banait commentedComment #28
mlncn commentedThanks for your contribution, fietserwin! You are now a vetted Git user. You can promote this to a full project.
When you create new projects (typically as a sandbox to start) you can then promote them to a full project.
Here are some recommended readings to help with excellent maintainership:
You can find lots more contributors chatting on IRC in #drupal-contribute. So, come hang out and stay involved!
Thanks, also, for your patience with the review process, especially my slowness in approving following review. We know it's broken and are trying to fix it.
Comment #30
avpaderno