Update to 7.x-2.11 "broke" repeating all-day dates

Oh no, I'm so sorry.

Do you have a backup you can restore while we work on fixing the update script?

FYI I unpublished 2.11 so we can fix this problem and release 2.12 ASAP.

@seattlehimay
please provide us with a backup of the date table prior to upgrade and also post upgrade , you can dump single table data

to dump a single table with mysql:

If your db is local:
mysqldump db_name table_name > table_name.sql

if your db is remote:
mysqldump -u <db_username> -h <db_host> -p db_name table_name > table_name.sql

example, if your date field has the machine name field_event_date then the table name would be field_date_field_event_date

The first thing we'll need to do is work out some example data to add to date_all_day_update_7200() that shows the problems, then we can refine the update script to make sure it works correctly.

FYI there's a separate task in #3201878: Provide update script to update all-day fields to extend the update script to use a sandbox, to avoid problems with too much data timing out the update script.

@seattlehimay: Thanks for providing those data dumps.

I see two problems with the data as it's stored:

• The data is stored with consideration of timezones. This means that the end-of-day calculations get more complicated and can't be based on the raw values.
• The update script didn't just modify the time, it changed the date by a whole day too. Dangit.

In hindsight I wish the field schema had been modified to store the "all day" option as a separate checkbox, that'd have solved a huge amount of problems.

@seattlehimay: Can you please confirm that field_event_date has the increment set to 15 minutes?

Thinking through it.. I wonder if that'd be the best option long-term anyway, to add an option to store whether the "all day" option is checked? Then we'd need update scripts to a) pre-fill all values to FALSE, b) process all records to see if they might be considered "all day" and set those values to TRUE, and update all internal APIs accordingly.

This is really bad.

This is not just repeating dates, I'm seeing old events that have the end date shifted ahead by a day and 15 minutes back.

What was previously:

Monday, August 6, 2018 - 9:30am to Monday, August 6, 2018 - 1:00pm

...with no all day or repeating settings is now being shown as:

Monday, August 6, 2018 - 9:30am to Tuesday, August 7, 2018 - 12:45pm

Also looks like it updated date fields to select the "Show End Date" checkbox for any event that previously had "All Day" selected and "Show End Date" deselected. No repeat settings, only "All Day" chosen.

Edit: This seems to be an issue with 7.x-2.11-rc1 too. I updated from 7.x-2.10, and never had a problem disabling "Show End Date". Now there's no way for me to deselect it.

Researching a bit further... 7.x-2.11-rc1 is also a problem for anyone who uses date timepicker. Timepicker uses time increments in its selector (typically 15 minutes), so it's never going to be a configuration where value2 is exactly one minute prior to the previous day.

In an existing node in 7.x-2.11-rc1, it is showing value = 2017-12-04 00:00:00 and value2 = 2017-12-04 23:45:59 as default element values for a full day event. This is not correct.

The reason why the "Show End Date" checkbox cannot be deselected is due to how the code works in the date_combo_element_process function:

  if (!$date_is_default && $field['settings']['todate'] != 'required'
      && is_array($element['#default_value'])
      && !empty($element['#default_value'][$to_field])
      && $element['#default_value'][$to_field] == $element['#default_value'][$from_field]) {
    unset($element['#default_value'][$to_field]);
  }

The $element['#default_value'][$to_field] value never reaches a state where it can be unset, so the checkbox *always* has a default value of TRUE.

We're going to have to roll back to 7.x-2.10 until some of these issues are fixed.

after update from 7.x-2.10 to 7.x-2.11，I find out the date field is disappeeared...

then I roll back to 7.x-2.10，but the field is still not showing，how do I fix this problem?

@yalyboy: Please open a new issue for that, thank you.

@yalyboy, the update to 7.x-2.11 changed how data is stored in datetime fields. Just rolling back to 7.x-2.10 is not going to fix the issue if you're using the pop-up calendar. The popup assumes dates are in defined increments (:00, :15, :30, :45, etc.), but the update sets the end date to one second before the end of the timeframe, which is not going to be valid.

The way to fix is to roll back to an older copy of your database, re-write the install script code to reformat the dates to how they were prior to 7.x-2.11, or wait until a new release is available.

We decided to do a combination of writing code to reformat dates to the old format and manually updating nodes after downgrading to 7.x-2.10. Really big mess and very glad we didn't roll this out to test or production environments.

-----

@DamienMcKenna, sorry, had just typed this response during the time you commented. Can certainly take it into a new issue.

@DamienMcKenna, ok I've open a new issue.

@ron_s, Thank for your response, I'll wait new release.

FYI the first step will be #3206100: Disable date_all_day_update_7200().

We had a site affected by this change as well, same as what was outlined by @ron_s. We were able to roll back the _value2 fields with the following SQL, though this may not be appropriate for anyone else. Our field machine name is field_date_event. We use a date picker and 15 minute increments for this field.

This works for us because we assume we do not have any events starting at midnight that aren't all day events.

UPDATE `field_data_field_date_event`
SET
    `field_date_event_value2` = DATE_SUB(`field_date_event_value2`, INTERVAL '23:45:59' HOUR_SECOND)
WHERE
    `field_date_event_value2` LIKE '%:59'
  AND `field_date_event_value` NOT LIKE '%00:00:00';

Basically, this query rolls back the "End" date for all values set to 59 seconds but that are not all day events starting at midnight.

As for a fix to the update script, should the select query on line 57 of date_all_day.install be changed?

From:

->condition('f.' . $field_to_change, '%:00:00', 'LIKE')

To:

->condition('f.' . $field_to_change, '%00:00:00', 'LIKE')

Can we safely continue to use version 7.x-2.11 or will the change we made break how all day events are validated elsewhere in the module code?

@DamienMcKenna I think you're on the right track to move towards having separate storage for "all day" in the schema. That seems to make a lot of sense, though I recognize it's a significant change.

Making some changes on a client site last night, saw this update available and installed it. This morning the client reported to me:

" the dates and times on all of our events shifted 15 minutes and a day last night. Our event coordinator caught it this morning and corrected them all - so please do not reset anything"

So, the client asked me to NOT roll back the data.
1) Will this cause problems with next update?
2) Is there anything else I should be checking on this site?
3) If a fix is not available yet, should this update be removed so that we're not still encouraging people to install an update that might break their site?

(EDIT: I now the warning on the release notes.)

Due to the amount of changes needed for the all-day fix, 7.x-2.12 has been changed to 7.x-3.0.

Moving this to 7.x-3.0-beta2 so that we can get the core restructuring into beta1.

Without knowing about these issues, we updated to 7.x-2.11 last week. Now we're finding all of these issues and rolled back to 7.x-2.10 but it doesn't fix the issues. What to do? Roll back to a previous backup, or wait for 7.x-3.0? And why is 7.x-2.11 still the recommended release if it causes known bugs?

7.x-2.11 ensures new dates have the correct all day values

if you're affected by the upgrade and don't want to work through them then restore to backup and roll back to 2.10

going forward however , 2.11 is best , certainly if you can simply just update your date tables manually and fix the glitch.

DamienMcKenna has worked very hard on this issue as have many others so we appologize for inconvenience. It would be great if you could help us improve the 3.0 release.

Thank you so much for the reply and to everyone working on fixes. We're trying to figure out the best path forward for our situation given that the site had the Date All Day submodule enabled and the release note (Warning: Do not install this update if the site has the "Date All Day" submodule installed, please wait until 7.x-2.12 is released.) was overlooked on our end. We would appreciate some advice.

if you're affected by the upgrade and don't want to work through them then restore to backup and roll back to 2.10

We are affected by the upgrade and have over 3000 events that are affected. Our most recent backup is a week behind and rolling back would have implications for other content of the site. Presently, we're evaluating whether to

1. roll back to 7.x-2.10 and then manually change some of the affected events and just wait for the 3.0 release that should handle date repeats and all day dates,

or

2. stick with 2.11 and make manual changes to affected events, and update to 3.0 when it is ready

going forward however , 2.11 is best , certainly if you can simply just update your date tables manually and fix the glitch.

We don't have anyone who can update the data tables directly in the database. Corrections would have to be made manually via the UI.

Given the release notes and our usage of Date All Day and Repeating Date Field submodules, should we go back to 2.10 and simply wait for 3.0? Or, does the Date All Day and Repeating Date functionality work in 2.11 for new events created after the update?

@joseph.olstad stated:

7.x-2.11 ensures new dates have the correct all day values

On our install, this does not appear to be an accurate statement. After Date is updated to 7.x-2.11, new "All Day" events are broken as well as the existing ones. Trying to create a node with an "All Day" checkbox checked gives the error "Time field is required." The update does not appear to ensure that new nodes will have dates with correct all day values.

So, even if we fix all the past dates broken by the update, it looks like there is no way to utilize 7.x-2.11 as-is moving forward, and we must roll back to 7.x-2.10.

Can you please test the current 7.x-2.x dev snapshot, let us know if it solves the problems or if they still exist. Thank you.

Tested "All Day" with the current 7.x-2.x-dev - no luck! Still the same error: In edit mode of the node I check "All day", it's not respected in the view mode and editing it again the checkbox is not marked anymore.

Tested "All Day" with the current 7.x-3.0-alpha1, ran updatedb to add the 'all_day' column to all tables with the date field and YES, it works like expected.

Moving this to the 7.x-3.x branch as it relates to "all day" values.

Does #34 confirm that the original problem is now fixed? Or are there still problems?

@DamienMckenna, so is the plan to merge 7.x-3.x that has the fix , back into 7.x-2.x?

#34 indicated that the all day bug is fixed .

Or ya maybe just tag and set 3.x as the recommended release ?

The intention is to have 7.x-3.x a separate thing due to the API changes from the all-day functionality.

Sounds good yes