Let's say I have two custom entities: Contact Form Type and Form Submission. So Contact Form Type is akin to a content type and Form Submission is akin to a node of that type. As the names imply, these entities are used to submit contact forms. Without going too much into why Webforms or EntityForms are not being used, I'll just say that these two solutions did not exactly fit our needs (thought EntityForms came close). Besides, the inherent problem of data loss still exists with both EntityForms and Webforms.
Anyway, so the user creates a new Form Submission (of a particular Contact Form Type) by filling out the requisite fields. The contents of the fields are sent via email and stored as usual with Field API. Great! Contact Form Types however are highly volatile. Fields frequently change, are deleted, new ones are added. In fact, Contact Form Types themselves may be deleted, new ones added, etc. The problem is that when the fields are deleted, the stored data is deleted with them. This data however needs to persist. While it won't ever be edited (no reason to modify a submitted contact form), field data may need to be queried. For example: "give me the names and phone numbers of everyone who has submitted a contact form". It will also need to be displayed in it's entirety (e.g., "show me all the contact form submissions I sent/received")
The way I see it, I have 3 options:
1. Store the fully rendered version of the Form Submission entity in a separate table (effectively perma-caching it). This would allow me to easily display it in it's original state even if the fields are later deleted. However, this approach would prevent me from easily interrogating the separate fields of the form (e.g., build a view with email addresses of everyone who submitted a contact form).
2. Store contents of the Form Submission in a separate table as something like a serialized object of key/value pairs (corresponding to the form fields). This would, at least in theory, allow me to get access to discrete field values. But it almost forces me to write my own storage mechanism for fields, effectively re-writing Field API.
3. A compromise where a certain fields are deemed "immutable" and are stored via normal Field API (e.g., First Name, Last Name, Email), while others are stored using one of the above methods.
So my question is this: How can freely add/remove/update the fields of an entity, but still retain (read) access to the data that was originally created using those fields?
I've been beating my head against the wall for a while now and am starting to feel like I'm "doing it wrong". Should I be looking at a NoSQL solution here or am I just overlooking something obvious? Hoping someone can provide fresh perspective or at leasts nudge me in the right direction.
Thanks!
Comments
The only way to do this
The only way to do this without ultra heavy modding of Drupal would be to just say, "No, you cannot delete any fields. At all. You can only hide them, restore them, or add new ones."
Since querying is a serious requirement, then, if the above solution will not suffice, I propose that you implement a NoSQL solution to hold all results, since this is exactly the type of problem that NoSQL was designed to handle. It would actually be easy to implement (just hooking on to an insert or update hook), but displaying the data might take a little bit of coding. Basically, you can no longer trust what Drupal tells you is in the node, because fields may have changed.
These are your only two solutions, as far as I can see.
Great, thank you for your
Great, thank you for your feedback. I'm thinking along the same lines of essentially trying to mitigate the changes to the data structure in the short term and looking at a NoSQL field storage in the long run.
Seems like these two modules would be helpful with the latter:
https://drupal.org/project/mongodb
https://drupal.org/project/efq_views