Special Characters?

This is probably "Double safety" somewhere. I.e. feeding it through the security functions (check_?) twice. Have you checked if you get the same problem with a normal flag?

BTW: & is also know to cause this behavior.

I think it is this that is the problem #1387718: Link field in view using "Title, as link" formatter outputs "'" in place of apostrophe in this case. However there are many modules that have the same kind of problem.

It is all characters that need to be escaped in order to avoid security problems. It is probably this function:

http://php.net/htmlspecialchars

Unfortunately I see no way of fixing it in this module... :(

It is always better to fix it at the origin. As you see there are more characters that are affected. And also not all views will be fixed. I have however another idea, but that is also a hack... (And it doesn't solve the generic problem)

/Thomas

This was much simpler than I thought... from a security perspective the fix seems safe.

With Flags 7x-3.6 and Flag Lists 7x-3.0-beta1 (in core 7.35), I am not finding that patch ...-9's code solves the problem. (Specifically, how the newly created List appears in the Flag Lists Operations dropdown menu.) Upon first creation, it appears in the dropdown as entered. (I tried making up a name that includes quotes, apostrophes, and other non-alphanumeric characters.) But upon browser refresh, all the non-allowed characters are replaced by codes.

Are reports of the patch success in context of an earlier Flag Lists release?

I notice that in the beta1 version, at line 197, the exact code specified in the patch exists there already – but it's not solving the issue, at least not in the dropdown menu's display. Also I notice that the code being identified for replacement in the patch exists farther up, at line 156 but it's commented out. Can anyone explain what that's about? As an experiment I tried uncommenting it and running the module, but that didn't seem to make a difference.

Hi,

thanks for the report!

The code on line 156 breaks the title on some pages and hence it was removed. I don't think it is related your problem.

Regarding the new bug you have spotted, I am not 100% sure which dropdown menu you are referring to? Can you provide the path to the page (Your host name etc is not needed, only the part after that), please?

/Thomas

Thomas, thanks for the response. The dropdown referred to is the List Operations dropdown that displays created lists. This list operations widget is appearing above a view table which has a column of checkboxes for selecting items (in our case they are videos) for adding to a custom list. The attached screen captures show that context, then how a new list name appears in the dropdown. A test name includes extra characters such as quotes, apostrophe, and others. In the first of those examples, the list name appears as entered. But after browser refresh, some of those are lost when converted to code. From testing so far, it appears only double quotes and apostrophes/single quotes are the issue. Although it's not a huge issue, it's likely that many of our users would want to start with a name like "Bob's Favorites List" or something like that. (meaning mainly, to use a possessive apostrophe) Regarding the URL, it is a main page with the URL [domain-name.com]/westat-video-library so I expect that doesn't help you.

Late-breaking addition: Especially interesting is that if I use the keystrokes for typeset style (a.k.a. "curly quotes") for either the apostrophe or the double quotes, then it works perfectly! (entered characters do not change after refresh) But I doubt that most users (other than former desktop publishers like myself) would normally think to enter them in this way.

Aha,

thanks!

Now I understand! I know that there are many problems / bugs with that part of the code. I did open this issue some weeks ago #2453803: Not all Flag lists operations are working to fix them. I will check for this problem as well. BTW the patch on that page is not in the 7.x-3.0-dev yet.

I am having problems reproducing the bug.

I created a list named

Thomas's "test & list"

which I think should trigger the problem. I have then tried with both F5 and ctrl-F5 (on Firefox 37.0) to refresh the page but after both these refreshes the list name appears correct. Is this the right way to reproduce it?

However the version I am using here is 7.x-3.0-dev PLUS the mentioned fixes #2453803: Not all Flag lists operations are working They are related to this functionality but I have not fixed your problem on purpose... :(

If you confirm the procedure I have used I will revert the version here.

/Thomas

Thomas,
My test of the same list name Thomas's "test & list" is not working here, and actually drops the part of the name past "test." But this environment is not the same as what you recently tested. It sounds like you have found a combination that solves the issue and maybe I should install the same.

To reiterate the exact condition I have now: The Flag Lists version is 7x-3.0-beta1. The changes/patches that had been applied were:

1. minor change described in https://www.drupal.org/node/1929060 to eliminate the error message about an "an illegal choice." Without that, the user had to refresh the browser after creating a new list name and before adding anything to it. (The list-add actually worked despite the error, but of course the error message needed to be eliminated and we didn't want to have to include the awkward extra instructions about refreshing.)
2. Patch in https://www.drupal.org/node/2452549 which solves the problem that an Admin user could add items to a custom list, but other authorized users would get "0 items added to list" regardless of how many items were selected to add to their list.

Does your combination of 7x-3.x-dev, plus the patch, also pass the failure points described above? If so, then it could be the comprehensive solution.

I have now checked the code and tried #2453803: Not all Flag lists operations are working for some time and I can not reproduce your problem :( So I will push #2453803: Not all Flag lists operations are working to the -dev. That is the code I am running here.

Thomas, I have tried the latest (April 15) dev version and here's what I am seeing upon testing the list-creation and adding-to-list process:

1. If the browser is not refreshed after entering a New List name, then adding selected (checkboxed) items still generates the error "An illegal choice has been detected. Please contact the site administrator." But the list is still created and items are added.
2. If I enter the list name Charles's "Test * List!" & Test! and then check the dropdown of list names (but without refreshing), it looks correct up to but not including the "&." Anything from "&" and following is dropped.
3. If I refresh the page after the test name as entered above, the name disappears completely from the dropdown list. However it will be found in the user's "list of lists," even if empty. The name there has not carried the apostrophe and quotes correctly, just the codes for them.
4. If the list name as described above is entered, the browser not refreshed, and items are selected and added to the new list from the dropdown, the "illegal choice" error occurs but it also reports the correct number of items added. Going to the list of lists, the new list has been created and containing the expected items, but again the quotes and ampersand (&) are in code.

Note that the "illegal choice" error was not present when using a patched version of flag_lists-7.x-3.0-beta1. I have not tried applying the same modification to the April 15 dev version. That patch was: https://www.drupal.org/node/1929060#comment-7430798

The trouble I am having with this may be due to particular Drupal environment factors that are different from what you tested. In case it helps now or later on, here are the details I would think could be related:

• Drupal core: 7.36 (although no differences have been noted during >1 year of updates)
• Views: 7x-3.10
• Flag: 7x-3.6
• Flag Actions: 7x-3.6
• Flag Lists: 7x-3.0-beta1+4-dev (April 15)
• Views Flag Refresh: 7x-1.3

Confirmed... :)

A short status update:

1) The problem with special characters I think I have solved. There is one case which is not working yet and that is "Deletion of a flag_list". Here the title is not showing correct yet, but this is more of a cosmetic thing.

2) #1929060: An illegal choice has been detected. Please contact the site administrator. is a little more troublesome :) The patch presented is unfortunately only a quick fix. What it does is that it takes away all verification of the entered value. I have figured out what goes wrong, where it goes wrong and why. But to fix it is a little more problematic. The AJAX routine is missing to set the updated part of the select array on the current page. If the page is refreshed (F5) everything is working nicely. I would like to avoid to reload the page and instead only add the missing info.

Thanks for the update and continued efforts. If you can bring this to the level of a new dev version, let us know.

Here is a first patch that I think solve both the problems. The AJAX handling in 2) is tricky so the only fix I have manage to get to work is to actually reload the page after the submission.

The attached patch should work even though my design base, for my on convenience :) , is the 7.x-3.0-beta1+4-dev + #2465553-4: Feature Requests: Option to customize wording; List Operations as block?

If necessary to ease testing I can push a new -dev.

Sounds encouraging! So in order to get to the level you have tested, would I need to start with 7.x-3.0-beta1 then successively patch with 2465553-4 followed by 2388741-24 ? Or start with 7.x-3.x-dev ? Of course, it would be great if you could push a new -dev to incorporate those changes.

Hmmm, seems like the -dev has a strange name for me. The name 7.x-3.0-beta1+4-dev is the current 7.x-3.x-dev. But I will push everything to the -dev.

Added back a character that I thought I had added for debugging only.

I have tested the April 27 Dev version in the context of our site, and here is what I'm seeing both positive and negative – in reference to the special characters issue and some other recently discussed issues:

1. The apostrophe/single quote, double quote, exclamation point and ampersand are being preserved and not changing to codes, including after page refresh.
2. The feature for applying a custom name to replace the default "list" or "lists" is working well throughout all displays derived from Flag Lists, except: the create list button (currently: "New List?"), and the dialogue displayed upon clicking that button (several instances there, see attached screen capture). I think it was mentioned before that this would be more tricky to change. Just including it on the wish-list.
3. However, on the critically non-functional side: A newly created list does not appear in the "Add to a list" dropdown. Refreshing the page does not help. The new list does appear in the user's table of lists. But since it doesn't appear in the dropdown, there's no way to test for the existence of the previous error "An illegal operation has occured..." when selected items were added to a newly created list

Discussion on last issue: At an earlier step in this evolution, there was a point where the failure of a newly created list name to display was limited to non-Admin users. But I have tried both Admin and non-Admin in the latest test, and cannot get a display of the list name in the dropdown.

1) That is good. There is one place where it doesn't work and that is when you delete lists.

2) That is as expected. I know that where Javascript is included it is not changing the list. But this is in line with what I pointed out... :( I now have an idea how to fix that. :)

3) This is disappointing... I found a small problem which I corrected in the 7.x-3.0-beta1+7-dev release. This is in the 27 April release of -dev which you are using. I did do a re-install here after fixing it and it is working for me. So we need to check a few things:

1. Have you cleared the cache?
2. Did you remove all old files before you installed the new ones?
3. Which browser are you using?

After checking your pictures: (This part is updated)

1. When you have entered the name of a new list and created it, do you see that the page is updated? For me it takes a short while for this to happen.
2. Do you see the already existing lists in the dropdown list?
3. What are you briefly doing with these modules?
   • Flag Actions
   • Views Flag Refresh

   Are they involved with this functionallity?

4. Maybe I should created a specially crafted flag_lists/js/flag_lists_ops.js file for you with some extra debug info in...?

4) Your fourth item: The non-admin vs admin was due to a logical problem which, if I remember right..., is fixed with #2453803-2: Not all Flag lists operations are working the part -1517,16 +1520,18. So that you shouldn't see any longer.

/Thomas

Some answers which I hope will help in the diagnosis:
First group of questions:

1. The cache has been cleared during the testing process, after updating the module. (Note: Site cacheing as controlled in config>development>performance is turned off in this dev site.) But I have now found a cache-clearing relationship to the problem: If all caches are cleared (using Admin login from one browser), then the missing newly-created lists will then appear for the non-Admin user (logged in from a different browser).
2. Updates of Flag Lists have been done by overwriting the existing Flag Lists module via direct server access. I have not typically disabled and uninstalled it before doing the update.
3. I have tested in Mac Firefox, Safari, and Chrome. No differences noticed in regard to the new list name showing in the dropdown list.

Second group of questions:

1. (your 1. and 2.): In all tests, the dropdown list of lists has always had at least one existing list, and the tests were intended to add new list names. I am seeing occasional but inconsistent updating of the dropdown list contents. Sometimes it almost immediately adds the new item, but most times it does not. (Note: Seems to work a bit better in Firefox, where there is a brief page "blanking" that tells me it's being updated.) More specifically: It seems it always works just after the cache was cleared. That first newly-added item shows in the list. But subsequent attempts to add new list names usually don't work, until after the cache is cleared again. Also, if lists are deleted (on the page having a table of all lists and a widget for deleting selected ones), that is not updated in the list creation widget's dropdown. A cache clearing, not just page refresh, is required to update it.
2. (your 3.): Regarding Flag Action and Views Flag Refresh: At some earlier stage of development I had added those modules to solve some behavior issues. Frankly I can't remember, or identify by testing, what those issues were. They may not be needed now. However, in disabling them I do not find a difference in the behavior described in 1. above. I do get a "fatal error" when disabling Views Flag Refresh but it goes away upon refresh:
   Fatal error: views_plugin_display::destroy(): The script tried to execute a method or access a property of an incomplete object. Please ensure that the class definition "views_flag_refresh_plugin_display_extender" of the object you are trying to operate on was loaded _before_ unserialize() gets called or provide a __autoload() function to load the class definition in D:\Websites\Showcaselibrarydev\docs\sites\all\modules\views\plugins\views_plugin_display.inc on line 272
3. (your 4.): Regarding the suggestion about a custom .js file: Of course, I would be greatly appreciative of that, if it may help in the diagnosis.

Would it help to see a series of screen captures showing exactly how the Flag Lists system is being used in this site?

Thanks for your comments!

Your 1) It seems definitely like the caching is involved. I will see if I can find something based on your findings sofar. On my case I have some of the caching activated. (But normally not the js and css caching on my development site.) I will see if changing anything here make a difference...

Your 2) I had exactly the same problems after adding those modules and trying to remove them. It was very tricky to do it. I can't describe exactly how I did it, but it involved removing all of their settings, clearing the cache several times and deactivating the module etc. (Maybe #2297647-5: Fatal error: views_plugin_display::destroy() or #2297647-6: Fatal error: views_plugin_display::destroy() is the ultimate trick...)

3) Please provide the suggested pictures! That would certainly help to understand how you are using it! If needed I can give you a direct email-address? Finally let me check a few more things before I create the js file I have in mind.

I am only aware of one place where special character are not handle correctly and that is in the "Deletion of a flag_list" as mentioned above. However this seems to me to happen external to this module... All other places works as of my knowledge.

We found a related issue with Views at https://www.drupal.org/node/1387718

Temporary fix at https://www.drupal.org/node/1387718#comment-11387379