Paging

@ufku, why won't this feature be added? I'm currently working on a site that needs paging due to the large number of image files.

Patch re-rolled for 2.x with some whitespace cleanup. This was needed for a site with quite a large number of files as well, so I'm re-opening this.

Hi All,

I have worked on patch #974568 for making working it on Drupal 7 with IMCE with version = "7.x-1.9" for adding pagination on pop up window of IMCE for file listing. Now this patch https://www.drupal.org/files/issues/imce_pagination_engage2.patch is working. Please try this.

I am VERY interested in your patch er.pradeep! I will check out your patch, however, why did not you attach the patch directly to this issue? (just curious)

This patch is not written using typical Drupal standards. It should not be written to include your personal file system path "sites/all/modules/contrib/IMCE" instead it should be applied from within IMCE directly. You cannot guarantee that all users will have the same directory structure for their modules, some use /sites/default, some use sites/all/modules (no contrib) etc.

Can you please re-roll the patch with this change so others can also try it easily. Please see this post for tips on patch file creation. :-)

I'll see if I can get your patch to apply to my dev environment and I'll provide some additional feedback for you.

Here is the same patch, re-rolled with the directory paths changed so it will apply properly. I've just started testing it and it has some potential for sure. There's a few small quirks I am noticing (For example the number of files that it lists is a little buggy), but this gives me a good place to start. If I add/fix/change functionality I'll be sure to come back with an updated patch.

Thanks to those who put in some work on this functionality!

Another quick re-roll of the patch so it will cleanly apply on the latest dev version too. (last patch was written for version 1.9).

Patch has not been modified other than the version is applies against.

***EDIT***
This applied, but I think I am missing a file somehow. Will be back with an update

Missing file re-added. Please do NOT use the patch in comment #8, instead use this one on #9 (I don't know how to delete/hide files from old comments, otherwise I would!)

Hmmmm... I'm noticing more quirks with this patch, so it will need some work for sure. The URL it is using to perform the paging is causing problems. (I'm losing my "Select" button once I start using the pager) Again, I appreciate the first attempt from er.pradeep! It just needs a little work and maybe we can get this working better.

Making progress! I believe I've fixed the issue with the links. I'm now loading in the proper URL variables so the select button is functioning properly now.

There are a few small quirks to still be worked out and I'll be trying to fix them as I have time. There are some issues I see that are related to having low numbers in the Maximum number of pages to display (for example, anything lower than 4). I'm also occasionally seeing an extra blank page in the pager as well.

I have a lot more tests to run, but at least this patch is now useable. As I make progress I will continue to update this issue.

I will hide the other patches I've written and make sure only the most current is displayed in the top of this post. (I finally figured out how to do that!)

If anyone else would like to take a look and help contribute please feel free!!

ugh, more issues. Looks like my idea of using the $_GET parameter to fix the link issues is flawed. When changing directories the $_GET array contains ajax information such as:

?jsop=navigate&dir=images/media

Grrrrrr... back to the drawing board.

I'm working out a new solution. Will post again when I make progress

Here is a new patch that addresses the issues I was seeing. I handled the pager URL issue with a session variable and I'm not sure if there would be a better way to access the information I need, but this does work. I also fixed the extra blank page issue, the low number "maximum number of pages to display" problem, and the issue with numbering once you get to the end of the paging. (I know this may sound vague if you haven't tried previous patches, but I want it documented for anyone that may).

This is far from perfect, and still needs more testing, but I believe it's good enough to encourage others to give it a try and provide feedback.

As always if I improve upon it I'll be back with a new update.

FYI, This patch will apply fine to dev and version 1.9 of IMCE

After doing some more testing I realized that sorting by column headers is not working as expected. Although sorting is being performed on click of a header, it's only sorting the current page results, not results across all pages.

Another issue I'm running into is on one of my apache servers when the number of files is extremely large (which is why I need the pager in the first place!) I'm seeing the value of "Transfer-Encoding:Chunked" on any ajax calls. This is causing overall default sorting to break down too because it appears to be unable to sort on all records since the response is being "chunked"

I'm trying to understand how IMCE is performing all of it's sorting tasks (I believe it's all being done within js/imce_extras.js) and seeing if there is any way to tweak it a bit for the way this pager works.

Moving the status back to "Needs Work"

Little progress has been made with this since I realized the issue with sorting. This patch prevents the HTML file rows from being printed out into the HTML so there's no way to sort of rows that aren't output.

I've played around a bit with adding back the removed HTML file rows and then adding functions to .hide() and .show() based on the sorts selected, but the idea really isn't coming together. Not to mention that this pager is coded to reload the browser window with each click of a new page in the pager.

I'm now looking to start over and see what I can do with a custom module to add on the pager functionality, rather than try to change the way IMCE works.

Any help or advice appreciated!!

Creating a custom module to handle the large amount of files in each directory is going OK. I've decided to create a year filter and a cosmetic pager. If anyone is interested in the functionality please feel free to check out the repository here:

https://bitbucket.org/navisitecad/imce-year-pager

Hopefully it will help you out or at least get you started on a path that will prevent IMCE from totally locking up.

I'm open to new ideas and suggestions so feel free to chime in!

Back to provide an update:

The custom module i created with paging and the year filter isn't working out as well as I had hoped. If you use IMCE Tools for searching you'll notice search breaks down once I search, and find something that isn't displayed with my year filter turned on (meaning I find a file from 2013, but my filter is restricting the list to 2016).

I'm also seeing small issues when uploading new files or when creating new copies of files via cropping, thumbnails, resizing.

I ended up abandoning the custom year/pager module for now, there are just too many hurdles to overcome.

For now, I've added a year token to the directory structure for my image fields. I'm hoping this will help reduce overall folder size and will then speed up display. It's not the optimal solution, but it has helped us squeak by for now.

I took the patch from #16 and worked on it.

I'm not that happy with the implementation... but it works.

Since I had the issue with S3 and I can't really count the number of files without iterating thorugh them, I removed the "total pages" calculation.

It's a pager that works... that's all what counts for me right now. You're free to make it better :D

Well, had an issue with directories... fixed that. (They were also skipped... when I only wanted files to be skipped.)

Now they disappear when one is an directory, also the paging isn't perfect with directories. But it works for me (better as the current situation... which results in a 5xx-Error after loading for 5mins).

Just realized, forgot to add the pager-tpl ...

This has been a huge improvement for us, we use S3 and adding a pager + removing the creation of thumbnails (unrelated with this patch) has lowered the burden on the server.

Adding a patch that is the same as #24 but doesn't touch the .info file.

@pcambra : your patch is missing imce-pager.tpl.php ...

@SerkanB : your patch works (fine), but it suffers the flaw pointed out by ladybug_3777 : each individual page is sorted as required, but the pages are sorted by the "natural" order of the file system.

In my test case with 17000+ images, I'm only interested (in IMCE) by the few last downloaded files : it's absolutely impossible to retrieve them !

I made an attempt to sort the directory (by filemtime, descending) (hardcoded!) before extracting the page.

It works for me, and a "perfect" solution would be far more complicated !

of course, I missed the imce-pager.tpl.php too ...

Thanks @gifad for noticing, here's the patch from #25 with imce-pager.tpl.php added back.

Unfortunately the changes from #26 and #27 defeated the purpose of this patch for me, which is having the files loaded from S3 fast and without burden on the server, with the order, it seems to try to order everything and then display it, which is as slow as listing the files for me.

Patch in #28 and #24 do not apply to 7.x-1.x-dev or 7.x-1.11. Terminal output follows. Perhaps dev has changed since. Will review.

patching file css/imce-content.css
patching file imce.module
Hunk #1 succeeded at 83 (offset 14 lines).
Hunk #2 succeeded at 100 (offset 14 lines).
Hunk #3 succeeded at 219 (offset 14 lines).
patching file inc/imce.admin.inc
Hunk #2 FAILED at 197.
Hunk #3 succeeded at 766 (offset 7 lines).
1 out of 3 hunks FAILED -- saving rejects to file inc/imce.admin.inc.rej
patching file inc/imce.page.inc
patching file js/imce.js
Hunk #4 succeeded at 856 (offset 5 lines).
patching file js/imce_set_app.js
patching file tpl/imce-content.tpl.php
patching file tpl/imce-pager.tpl.php

Rerolled patch in #28 against 7.x-1.x-dev (manually applied the HUNK that failed). Works for me.

Special thanks to @ladybug_3777 and @SerkanB, as well as everyone else involved!

Although this pager is great, it becomes almost useless when using it in combination with the imce_tools search module: clicking on a result link does nothing - i.e. not working.

For example, if I have 100 files and Number files per page is set to 20, if I search with keyword john and in the results john-smith.jpg appears in 87th position in the list, clicking on the result link john-smith.jpg does not find and highlight the file as it would normally do because we're on page 1 of 5 and the file is actually on page 5 of 5.

For this to work, it appears that the imce_tools search module would need to check for the existence of a pager and update the results links accordingly.

See #2918425: Update clickable search results for pager patch

Hi,

I have applied the following fixes to the patch in #30

1. Added the page query param to the Ajax URL which will solve an issue where the user can't delete files from 2nd page onward.
2. On files deletion, added some logic to update the files on the current page.

Added another fix: The "insert file" functionality, when going through the pages, this button disappears due to disregarding the query params, this patch will always keep the existing params and just add the `page` param to them and therefore will handle the existing functionality a lot better.

I have tested the patch from #33 and it's working great, but I am not using imce_tools. I had initially tried the one from #30 which also worked fine, but the file deletion on subsequent pages was an issue, and #33 fixed that as far as I can see.

I'm so excited to see you guys have had some time to invest in this! I haven't had time to get back to it but I'm excited to give it a try soon. Keep up the great work!

I just tested this with imce_tools, with its imce_dir_man, imce_file_path and imce_search submodules all enabled (with the patch from #33). I did notice an issue with paging; when I click one of the pager links, the directory structure at the left refreshes.. and it does not list all the subdirectories anymore .. only some of them. I then tested this with imce_tools and all its submodules disabled... same thing. Even if I click back on Root in the left-hand directory list, it won't list all the folders anymore unless I close the IMCE window completely and then click the IMCE icon from the editor again. It seems somehow the pager links are causing the directory navigation to refresh in an incomplete way.

Adding a very minimal change to #33, the imce_showperpage variable should have the same default value everywhere we call it, otherwise the pager won't be displayed.

There's one thing that doesn't work for me, if you have the page in several folders, when you move between them, doesn't seem to reset, this was probably introduced in #32/#33 but the featured added there is far more valuable for us.

Setting to needs review because maybe the integration/search by imce_tools should be addressed separately.

Hi @ufku!
Why not?