Originally identified in #2941710-9: UI/UX - Improve SQL dump config form:
PHP's maximum max_input_vars variable had stopped my settings form from saving at all. I wonder if the form structure could be changed to reduce that
(For reference, I have 1864 columns, across 252 tables in the local DB I happened to be testing against. I use search_api_db and I have a quite a number of fields across several entity types, including paragraphs, but that's not exactly rare.)
We did improve the situation quite a bit in that ticket, by removing a superfluous checkbox for each column (amongst other things), but it's still far too many really.
mhavelant suggested:
Yeah, the form needs lots of work. Adding to your request, we also need to update it for supporting a large number of tables, too.
Maybe adding AJAX, pagination or some other form of optimization would help with both issues.
| Comment | File | Size | Author |
|---|---|---|---|
| #7 | interdiff-2973031-4-7.txt | 1.44 KB | james.williams |
| #7 | gdpr-2973031-7-dump-form-more-tables.patch | 6.18 KB | james.williams |
| #4 | more_tables.png | 107.18 KB | james.williams |
Comments
Comment #2
james.williamsHere's a suggestion: instead of listing all tables, we only list those that have any configuration set for them, and have some kind of 'add another table' option near the bottom?
The reality is that only a few tables normally need any anonymization, i.e. tables relating to users, contact messages, etc. We could even default to showing the known user ones (core's user_* tables), and then let the administrator select which additional tables they want to add, one (or more) at a time.
I'm thinking something a bit like Search API's 'add related fields' button at the bottom of its fields config page. (Admittedly, I might be thinking of what it did in D7!)
AJAX might be nice, but may just be an unnecessary complication, in my opinion.
Comment #3
mhavelant commentedI like the idea of having a default set of tables visible and adding more on-demand!
Comment #4
james.williamsWell, here you go then :-D
* Shows all tables that have already been configured for anonymization/emptying
* Also shows all user & contact tables by default (there may be others we should by default? Is it worth adding a hook so modules can declare theirs as likely to include PII data? Are there already APIs around for doing that? I'd love it if there could be a default suggested set of config, e.g. stuff like user emails, contact details on commerce orders, etc, could all be preconfigured or at least shown here immediately by default? Maybe that's something for another ticket...)
* Adds a new details pane that lists all other tables in a 'tableselect' widget, which includes the table columns. A 'refresh' button is at the bottom, which adds the tables in full to the form so that the actual settings can be configured. (Without javascript!) It could be seen as a slightly odd workflow, but I don't think there's much benefit to building it with AJAX really.
I also snuck these two changes in as part of it, since they made sense to do alongside these changes, I hope you don't mind:
* Added table descriptions to all details panes (where supported), because that's nice :-)
* Tweaked the description across the top of the page, since it was out of date anyway.
This drastically reduces the number of form elements on the page! (To just over 200 on my reference site.) There is still plenty of potential for there to be too many, so pagination etc might be worthwhile in future, or at least a warning, since the issue should be detectable. Maybe even just a note in the Readme file suggesting to change the setting in php.ini?
Comment #5
ricovandevin commented@james.williams Thanks for the patch. It is already a great improvement.
A hook that allows modules to declare their data is being 'personal' might be a great addition but as you already suggest yourself that can be part of another issue. What I'm missing at this moment is a way to remove a table from the view once you decide that you do need to anonimize its data (anymore). That might be a good feature to add within this issue.
Comment #6
james.williamsThanks for the appreciation :-)
To remove a table, ensure all its settings are unticked / set to ‘no’, and it will go away after saving the configuration or pressing the button to refresh the form. The exception is the tables that I thought were worth ‘forcing’ to display, i.e. user & contact tables.
Are you suggesting it’s worth being able to remove those user & contact tables from the form though?
Comment #7
james.williamsI thought of a few more modules' tables that really ought to be shown by default on this form even now: comment & webform. Updated patch attached, plus interdiff.
Comment #8
mhavelant commentedComment #10
baluertlMoving issue state manually from Fixed 🠂 to Closed (fixed) in the hope that issue queue gets cleaner a bit.