Multistep forms fill sessions table

Unecessary hunk pulled out, unecessary date calculations replaced with a simple time value.

The conversation with Dopry raised some questions for me, too. Long-term, we may want to think through ways of making up-to-the-moment clearing more straightforward for modules. For now, though, in the final hours of beta 1, this solves the emergency.

RTBC it shall be.

Dries, any form that has #multistep set to true will store its initialization params in the session. Each instance of each page in each form with that flag will have its seed data stored. One by one it's not really a problem. But that means those unused seed values will accumulate in saved sessions. And if the users log on regularly, their sessions won't be garbage collected. On a site with a few hundred users, doing something like creating polls regularly, or using any new module that instantiates a multi-step form, the endless accumulation could be a serious problem.

OK. To demonstrate, I set up a basic site, visited the modules page, turned on poll module, and made a poll. I previewed once, then saved the values. Then I checked the contents of $_SESSION.

Array
(
    [form] => Array
        (
            [e4a171783b07279dca279496088672d4] => Array
                (
                    [0] => system_modules
                )
            [feba265b479eac97a574ffb3a1341d65] => Array
                (
                    [0] => system_modules
                )
            [0275691da1ebf1a266baf1058823b75a] => Array
                (
                    [0] => poll_node_form
                    [1] => Array
                        (
                             [uid] => 1
                             [name] => admin
                             [type] => poll
                        )
                )
            [617d62ef5e0e6d5c20ab4f5fcf900724] => Array
                (
                    [0] => poll_node_form
                    [1] => Array
                        (
                            [uid] => 1
                            [name] => admin
                            [type] => poll
                        )
                    [2] => Array
                        (
                            [title] => This is my sample poll
                            [changed] => 
                            [choices] => 5
                            [choice] => Array
                                (
                                    [0] => Array
                                        (
                                            [chtext] => Question 1
                                            [chvotes] => 0
                                        )
                                    [1] => Array
                                        (
                                            [chtext] => Question 2
                                            [chvotes] => 0
                                        )
                                    [2] => Array
                                        (
                                            [chtext] => Question 3
                                            [chvotes] => 0
                                        )
                                    [3] => Array
                                        (
                                            [chtext] => 
                                            [chvotes] => 0
                                        )
                                    [4] => Array
                                        (
                                            [chtext] => 
                                            [chvotes] => 0
                                        )
                                )
                            [morechoices] => 1
                            [active] => 1
                            [runtime] => 0
                            [form_build_id] => 0275691da1ebf1a266baf1058823b75a
                            [form_token] => d8408eeeba2594b40bac0704b26bf690
                            [form_id] => poll_node_form
                            [log] => 
                            [comment] => 2
                            [menu] => Array
                                (
                                    [title] => 
                                    [description] => 
                                    [pid] => 1
                                    [path] => 
                                    [weight] => 0
                                    [mid] => 0
                                    [type] => 86
                                )
                            [name] => admin
                            [date] => 
                            [status] => 1
                            [promote] => 1
                            [op] => Preview
                        )
                )
            [8a5f9625b4bfe1e7df793d9932b96053] => Array
                (
                    [0] => poll_node_form
                    [1] => Array
                        (
                            [uid] => 1
                            [name] => admin
                            [type] => poll
                        )
                )
[node_overview_filter] => Array
        (
        )
)

Pay special attention to the entry for node_poll_form -- the *full form_values* array is persisted. This is necessary because the form values become a seed for the next step in many multistep forms. And we're lucky, because core only tends to use it on forms with a very small footprint. But persisting all that data is a BIG, big problem.

If I add something like ecommerce, which will use extensive multistep forms for *every purchase made by every user* the results will be pretty grim. It boils down to this: the information we store in the session is transient, and is not intended for long term persisting. There's no reason to keep it, and keeping it can become a problem in the future because of the eternal accumulation of data under some circumstances. My big question with this patch was whether waiting a day left too *much* opportunity for accumulating session bloat. The more I look at it, something like an hour might be much better. Leaving it to eventual session expiration is just bad. Very, very bad

drumm: to expand on chx' statement, there's not really any way to reduce the amount of information we store without making a lot of assumtions about what subsequent steps in the multipart forms will require. In 6.0, we might consider being able to mark certain fields as explicitly multistep, and only having THOSE fields appear in the value collection that's passed on to the next step. That, though, is prone to its own set of weird hassles.

I'll take a closer look today to see if there are any other opportunities to automatically clear stuff, but I don't think we can do too much more.

Er, more specifically, I don't see how we can do too much more without adding a LOT of code VERY late in the cycle.

That's one possibility, although it would rule out any possibility of using multistep forms in pre-bootstrap scenerios like the installer, and would lead to a few more DB hits each time a multistep is displayed. I'd have to look, too, to ensure that a cache_clear_all() by user a wouldn't destroy an in-progress form by user b.

It might be an acceptable solution, but it's not without drawbacks. chx, any thoughts?

This still raises difficult questions: I'd prefer that we NOT use a table, at least at this point, to store the data. It will make multistep forms incompatible with the pre-db installation process, something that closes off future enhancements to us. If we do do this, I'd prefer a separate cache table just for transient form data. What's the 'process' for generating a new cache table?

The problem with using the cache table is that is know to be 'nukable'. That is, at any point in time, one might nuke the cache without excepting data loss, or causing trouble. The old cache data is always properly regenerated. The session data, on the other hand, is not supposed to get lost and can't be regenerated. So maybe using the cache isn't a good idea after all?

Precisely. The session *does* do exactly what we need; it's just a matter of clearing it out in a more timely fashion to prevent evil stuff like, 'five million people filled out my survey form, now the sessions are filling my database..."

We seem to have come full circle. I still feel the solution in the patch is the best one we have until 6 rolls around and we can look at more comprehensive/agressive/api-breaking approaches if necessary. For 5, though, this s a great boon.