I have now spent several hours just trying to get the D7 version of the Date module working as well as it was a month or two ago, with no success. I haven't been able to figure out all the things that changed in core that require corresponding changes in Date.
1) It is now impossible to create a datetime field. It fails immediately with errors.
2) I can add a datestamp field, but it fails if I try to create content.
3) I can add a date field, but it no longer shows up on the node edit form.
All of these things worked a couple months ago.
Just a FYI, I'm running out of more time to dig into all the new failures.
Comments
Comment #1
karens commentedI spent six or seven hours on this today and can't even get it working as well as it was in August. I've got to pull off for now, no more time to work on it.
Comment #2
the greenman commentedMost of the issues I have found are due to the new nested value functions. (form.inc 5724) The element value will sometimes be a sting time, while the form is expecting an array of date and time. ($element[#parents]).
I am a little too new to fields to know where to start debugging, but will add notes if I get further.
...
Looking a little further. Date_popup_input_value is one of the places where I hit this issue first. At the point where this function is called, I have my date as a string - but this is looking for an array.
...
It looks like the initial code to process form values (in the widget form) can no longer access the item values. (I am going round and round now, so may be confused. I was working with a partially rolled-back install, so may have been barking up the wrong tree.)
Comment #3
yhahn commentedI did a little digging here on #2 here and it appears that the issue is like @greenman pointed out -- there are changes in the FormAPI input processing that are choking on Date form elements, in particular ones that are broken out into sub-fields at the process level.
For example, on a node form Form API is setting the field value to a string:
While after processing the field is being expanded out to a set of select boxes
etc.
It looks like the solution here is to set a
#value_callbackin_date_field_widget_form(), but this will require some more work as the same widget logic used to expand date fields out into separate form API elements will need to be used here to take the field value and expand it out into an array matching the widgets.Expect a first stab patch soon.
Comment #4
yhahn commentedHere's an initial patch that adds value callbacks to each of the date element types. There are some changes here that need closer review -- I'm in foreign territory here and the lack of tests is definitely hurting my ability to know just how much functionality/corner cases may be affected by these changes. I'll look into getting time to write tests if possible.
FYI the related core issue to this change is here: #913528: Create new boolean field "Cannot create references to/from string offsets nor overloaded objects".
Comment #5
karens commentedThanks so much! Yes, we definitely need tests, it's on the long list of things I need to get around to doing :(
Comment #6
yhahn commentedFYI I think we may want to consult a D7 form API expert in regards to this issue. You'll see in the patch that any time a date form API element is used it needs to also be accompanied by an explicitly declared value callback:
Unfortunately the value callback cannot be declared by the element
processcallback based on the way values are being checked and set in the FormAPI. This seems wrong to me though as it means that to use any form element that expands a string value into a nested array for its processing/UI you'll also need to know what its corresponding value callback is to handle the value expansion.I'm not a FormAPI expert though so I may be completely missing something obvious that could let us get around this.
Comment #7
yhahn commentedI talked to Adrian and he pointed out that the
#value_callbackcan be defaulted for the element types in question inhook_element_info(). I've rerolled the patch to do this which takes care of my concern in #6. Also added value callback fordate_timezoneelement which I missed first time around.Comment #8
jurgenhaasI'm having the same issue as in item 1) of the original post: can't create a field of type "datetime" bit a field of type "timestamp" can easily be created.
Note: this has been tested against the D7 beta 1
Comment #9
karens commentedYou can't create a datetime field because core broke it. There is nothing I can do about that until core gets fixed.
Comment #10
danny englanderSubscribe
Comment #11
jurgenhaas@KarenS, is there an issue filed with core for this?
Comment #12
karens commentedThe datetime field support was broken by #866340: Remove support for date and time types originally. I got that fixed once by adding a schema_alter to the date module, but that no longer works. The latest bug seems to be #927828: Some schema code incorrectly rely on the generic type instead of the engine-specific type, which has only a 'needs work' patch. I can't tell if that will fix it or not. I have links on the Date project page to several core issues that are now blocking this module from being ported.
Comment #13
Anonymous (not verified) commentedI took a look at the datestamp issue. On the surface it seems like a pretty straight forward issue. The problem is that it's trying to insert a string but the database expects an integer. So I attempted to format and convert the string date to an integer timestamp using date_make_date() and date_format_date() in date_field_update(). Another issue I found here is the date isn't being added to the correct index of the $entity object.
This is what I changed date_field_update() to. This seems fine, I have the date formatted as it should be and it's in the $entity object as it should be but there seems to be a further problem. Whenever the form is submitted the date reverts from the date set to the current time. I've tried to trace what's going on but it's not apparent to me where this is happening. I've attached my incomplete patch of the work I've accomplished.
Comment #14
karens commentedI tried out the code in #4 and it works but has lost the part of the code that does timezone adjustments, so you enter a date and it is saved without converting it back to UTC, so the next time you load it it is off by 5 hours (or whatever your timezone adjustment is).
I have also decided I am going to have to just remove support for datetime fields. Core no longer supports it, they weren't interested in adding it back even though it was available in D6, and the patch that might make it work again doesn't work and isn't even marked as critical, so it may never get fixed. I added that field type when core added it so I guess I will remove it now that core no longer supports it.
Comment #15
karens commentedAfter spending several hours reading through a bunch of recently-committed FAPI issues I think that @yhahn is right that a value_callback is now going to be necessary to keep the new drupal_array_get_nested_value() and drupal_array_set_nested_value() functions from blowing up when the pattern of the processed, nested arrays does not match the original element. One of the elements that needs this is the combo_field element, which has a very complex pattern and which handles the timezone conversion that was missing in the earlier patch.
I never in everything I read found anything that documented that a value_callback is now required or why. And the work to create the correct array in the date field now is not happening at the time the value_callback is called, but much later, during validation. So fixing this correctly will require a total rewrite of the way that date fields are processed :(
I see that Adrian said we can default the value_callback, but you still need to know the entirety of all the sub-elements that might be present. For the combo_field element this is really complicated -- it is an array that joins the 'from date' (with all its sub-elements) to the 'to date' (with all its sub-elements), and potentially adds a timezone element and a repeating date element (with another complicated array of sub-elements). I have no idea how to move all that into a value_callback that is invoked before we know anything about the nested elements.
Pondering this further....
Comment #16
karens commentedI've made a bunch of changes and incorported the patch at #7, adapting it to the new DateObject class from Dave Reid that has been added to the D7 version. Still lots more work to do, but I can at least created an ISO date field without errors. And without validation, since the validation isn't working right yet.
Comment #17
effulgentsia commentedSubscribing. I'll try to make some time to help with the FAPI part of this. Clearly, if core makes it impossible to have a date field, we need to ensure those are critical issues in the core queue. Also, cross-linking to #763376-145: Not validated form values appear in $form_state.
Comment #18
yched commentedNot really helpful, but see my #913528-54: Create new boolean field "Cannot create references to/from string offsets nor overloaded objects" :-(
Comment #19
an.droid commentedI can't look at the code of date.module now, but as I understand the problem is in splitting/merging field item values before saving. If my understanding of Fields workflow is correct, there is hook_field_attach_submit() which can be used for this purposes. It allows to fetch needed values from
$form_stateand pass them to corresponding$entity.This
$entitywill be later passed to the hook_field_insert() (or _update() ) where additional modifications to the field values an be made before saving.UPD: Oops, this is more about FAPI element, rather then Fields API field. Sorry, misunderstood.
Comment #20
das-peter commentedSubscribe
Comment #21
fagoI agree that the value_callback isn't very helpful for elements expanding as it is now as it is invoked before the expansion process. However, one can workaround that rather easily. I've done so for the duration element I've written for Rules.
Perhaps the approach taken could help you too, so I shortly describe it here:
* The duration element has a process callback expanding into 2 child elements.
* The value callback is used to assign an array() as value. It must be an array if you have child elements.
* Use an after_build callback (which is called after processing) to set the real value of the element.
Relevant callbacks:
Comment #22
karens commented@fago, that works, just setting an empty array and populating it in the afterbuild? This looks like an approach that fits into the way that the Date module works. I ripped out the combo_field element in my last round of changes because it totally blew up, maybe this would allow me to add it back. I used yhahn's patch which just removed the element altogether to get the field working again.
Comment #23
scotwith1tsubscribe. it's a shame that date field types didn't get much consideration (or God forbid, be included!) when D7 fields API was locked down. seems like with as many event-related sites as there are using drupal that date field considerations would have been an extremely important consideration.
Comment #24
fagoad #22: yep, that seems to work very well :)
Comment #25
scuba_flySubscribe
Comment #26
karens commentedOK, this is interesting. I've been trying out ways to get my old workflow working again. The old process was:
- Skip the value_callback
- Use #process, without a value callback the $element['#value'] would contain whatever was in the original '#default_value' (i.e. #default_value had been 'processed').
- In the #process function, take the value of $element['#value'] and explode it into sub elements
The new process is this:
- Cannot skip the value callback, so just return an empty array(), as per fago's suggestion. Return an empty array prevents the error messages you get if you skip that step.
- Use #process, but in this case $element['#value'] is empty.
- In the #process function, take the value of $element['#default_value'] and explode it into sub elements.
I'm still working through all this, but it looks like this may work. But it is definitely a different FAPI workflow that should be documented somewhere.
Comment #27
pivica commentedsubscribe
Comment #28
karens commentedI have a simple ISO date working correctly now, using @fago's idea of returning and empty array for the value_callbacks. I can create a select widget date and the timezone is correctly converted. I committed that much, next need to get other types working and get tests added.
If anyone wants to supply tests, I'd like to have tests for the basic Date API (create a custom form element of the type '#date_select' and process it correctly), the field API (create a date field using the Field API and process it correctly), and the field FAPI processing (create a date field using the UI and create a node using the UI and process it correctly).
I have some old tests in the D6 version of the Date Repeat API, I want to get them updated so they work in D7, too.
Comment #29
Anonymous (not verified) commentedI had written some test both for Drupal 6 and Drupal 7. They can be found here. http://drupal.org/node/935076 It provides some basic date CRUD testing.
Comment #30
karens commented@tristanoneil, yes, thanks for those, I'm picking those up. They basically test the UI rather than the underlying API. But I'm trying to get lots more functional tests.
For instance, the Date API has some basic date functions that can be tested, independent of the way fields and forms work. Create date objects of various types, alter their granularity, timezone, or other settings, and test that the right values get created. The Date API has its own date elements that can be tested independent of anything about the way that Fields work -- create a custom (non-Field) date form element and process it to be sure it returns the right result. The field API can be tested independently of the Field UI by creating a date field using the Field CRUD functions and then populating it with a node_save() to be sure the right value. Finally the Field UI can be tested. And the Date Repeat API can be tested. And the Date Popup element can be tested. And the date sql functions can be tested. And the Date iCal functions can be tested.
So those are the kinds of tests I'm looking for.
Comment #31
karens commentedI've commit tristanoneil's test and a couple others and I'm moving all the discussion about tests to #935076: Comprehensive tests.
Comment #32
karens commentedI think I have most of the recent brokenness fixed again, except for the fact that you can't create a datetime field, which won't be fixed until core is fixed.
This is not to say that Date is fully ported, but I think after many many hours of work I am back to the level of functionality it had a couple months ago.
Comment #33
moshe weitzman commentedThanks for your hard work Karen. It is very much appreciated.
Comment #34
effulgentsia commentedDitto.
#927828: Some schema code incorrectly rely on the generic type instead of the engine-specific type is critical, so therefore, presumably will be fixed. Is there anything else in core that will make it impossible to have a fully functioning Date module? If so, we need to make sure those issues are critical, so please link to them from here, so we can all lobby as needed.