This has been present for a while in the D6 version of date (#523218: Default value for To date: Blank should stay blank).
This bug is still present in the D7-1.x-dev version of Date.

Steps to reproduce (in D7):
Add Date field to content type (Type of Data to store: Date)
Choose "Optional" for To Date
Chose 'Minute" for Granularity
Chose "Site's Timezone" for Time zone handling
Click Save field settings
Keep all the defaults on next page (this page includes forms for setting 'To Date' to Never, Optional, or Required as well as 'Time zone handling' options... is this duplicated from previous page?... but that is for a different issue).
Click "Save Settings"
Add new content and fill out ONLY the From date.
Click "Save"
Edit the saved content - To Date is now filled in with From date. It is not blank as it should be.

Comments

arlinsandbulte’s picture

Note: also verified for the Datestamp field type.
I was unable to create a Datetime field, but that issue is being addressed here: #866340: Remove support for date and time types

KarenS’s picture

I'm not too worried about fixing things like this until the D7 version is working more or less the same as the D6 version, so this is a low priority for me.

It is actually not a bug but by design, but obviously lots of people think it's a bad design.

KarenS’s picture

Priority: Major » Normal

I'm going to downgrade the importance. Critical and major tasks are getting the D7 version working in the same way as the D6 version.

arlinsandbulte’s picture

Hmmm, I could have sworn that the D6 version used to work this way (blank 'to dates' would stay blank), but I just tried each point release and I guess I was wrong. That certainly justifies the normal priority and really makes this more of a feature request than a bug report, but I will keep it as such for now just to keep it near the top of the list. If you disagree, I'm OK with that.

Also, this got me thinking about how a blank To Date could make other things simpler, more intuitive, and simplify code. In particular, relating to the way "All Day" is handled in Date. So, as long as I am on my soapbox, here are some of my thoughts:

The way it is now:
For a date field with no To Date (To Date set to Never), it is labeled as "All Day" when the time equals 12:00AM.
For a date field with a To Date ('To Date' is set to Optional or Required), it is labeled as "All Day" ONLY when both the From Date & To Date times are on the same day AND both set to 12:00AM.
This can cause problems when I WANT to have an event set to occur at 12:00AM, and not have it show "All Day."

My Preferred Proposal:
For a date field with no To Date (To Date set to Never), it is labeled as "All Day" only when the time is absent (e.g. value is '2010-08-11'). It is not labeled as "All Day" if there is a time specified (e.g. value is '2010-08-11T05:12') thus becoming a milestone event.
For a date field with a To Date (To Date is set to Optional or Required), it is labeled as "All Day" when the time is absent in the From Date (e.g. From Date value is '2010-08-11'). When time is absent in the From Date, it is also automatically absent in the To Date. A milestone event is specified by setting the From Date (including time) and To Date equal.
This proposal might require pretty extensive code changes because, right now, the time is ALWAYS included in the stored date. This might be similar to the "Fuzzy Granularity" proposal, but applied to time.

Maybe an easier Proposal (does not involve dates without time):
For a date field with no To Date (To Date set to Never), it is labeled as "All Day" when time equals 12:00AM (this is the same as it currently works). Unfortunately, this makes it impossible to create an event at 12:00AM. But for that you could use a date field WITH a To Date...
For a date field with a To Date (To Date is set to Optional or Required), it is labeled as "All Day" ONLY when both the From Date & To Date times are on the same day AND both set to 12:00AM. This is the way it works right now, BUT the difference is that if the To Date is blank (or NULL?) it is not labeled as "All Day" but simply occurring at the time of the From Date (a milestone event). Not sure what to do when the From Date and To Date are equal and not set to 12:00AM. Perhaps that could just be another way to specify a milestone event.

Note: In the above, I am mostly trying to describe support for 3 different type of date events:
Time slot event: This is the typical event that occurs over a specified period of time. For instance Calculus class is on 2010-08-12 from 1:00PM-2:00PM.
Milestone event: This is only a single point in time. There is no duration associated with it. For instance, meet at 12:00PM on 2010-08-13 to go to the mall, or this item was posted at 1:53PM, 2010-08-16.
All Day event: This is an event that spans an entire day. For instance, Christmas in on December 25, 2010 (All Day).

I can think of lots of other ways to handle blank To Date values and how to differentiate All Day, Milestone, & Time Slot events. This definitely needs more thought and discussion... 7.x-2.x here we come! :-)

KarenS’s picture

Interesting ideas, maybe you are right that we should think about leaving the to date NULL in the database. I think there is code there now that will barf on that so it will take some work.

tim.plunkett’s picture

Subscribe as a placeholder until I have time to post something intelligent.

YK85’s picture

subscribing

basicmagic.net’s picture

subscribe

arlinsandbulte’s picture

Version: 7.x-1.x-dev » 7.x-2.x-dev
designcontext’s picture

I think this is very important!
Now what is happening: I set a from date 12.04.2011 - 16:30 and no to date (which can occur because some events are open end).
I make a view with from and to date and I see: to date 12.04.2011 - 16:30 - thats bad usability!
Hope it will be fixed!

HnLn’s picture

sub

mansspams’s picture

very good arguments @ #4, subscribing

klonos’s picture

...subscribing.

Kolibri’s picture

Hello.
It would be wonderful if you could possible set an empty to-date to NULL in the column field_work_date_value2 of field_data_field_work_date, just like it is done in the content_type-table.
Because if this takes place, the filter of the views module would recognize an empty field as empty.
Merci

BrightBold’s picture

Subscribing. I love @arlisandbulte's suggestions in #4 — I often encounter all day and "milestone" events (for the latter, often deadlines, e.g., forms due at 5:00pm) and the current system tends to confuse my content administrators so I'd love some new ways of handling these.

mansspams’s picture

Priority: Normal » Major

Bumping up, with your permission...

arlinsandbulte’s picture

Title: To Date & All Day Date Handling (was: Default value for To date: Blank should stay blank) » Default value for To date: Blank should stay blank

OK, I have put a bunch more thought and experimentation into my ideas in #4.
I now believe my suggestions in #4 could be better (my suggestions in #4 might not have even been possible to implement).

First of all, storing a date into the database seems to ALWAYS result in a complete date WITH time, no matter what date type I use in the database. So, I don't think storing a date without time is practical (it might be possible, but not practical from a performance POV).

Thus, here is my latest and greatest suggestion:

The way it is now:
For a date field with no To Date (To Date set to Never), it is labeled as "All Day" when the time equals 12:00AM.
For a date field with a To Date ('To Date' is set to Optional or Required), it is labeled as "All Day" ONLY when both the From Date & To Date times are on the same day AND both set to 12:00AM. If no information is entered in the 'To Date,' it is automatically populated with the same values as the 'From Date' when the node is saved.
This can cause problems when I WANT to have an event set to occur at 12:00AM, and not have it show "All Day."

My Proposal:
First of all, EVERY date field will have 3 values stored in the database:
1.) Start Date & Time. This is just the way it is now.
2.) End Date & Time. Similar to how it is now, but even dates with 'no end date' still include this in the database table. The 'no end date' option simply tells date processing whether to use this value or not. (this makes editing/changing the 'no end date' option easier after the field is created).
3.) All Day flag (true or false). This is new and is used to specify if an event occupies the entire day. IMPORTANT: if All Day is TRUE, no time zone processing should be applied to the date values. There might be some people who want time zone processing to occur on their all day event. If so, they should not set the All Day flag = TRUE and use Start = 2011-01-01T00:00:00 & End = 2011-01-01T23:59:59.

This allows the end date to remain always populated as Date module does now.
To specify a Time Slot Event: Enter Start and End data as usual.
To specify a Milestone Event: Enter Start and End data, making sure they are both equal (if end is left blank, it is populated with the same value as the start when saved, just as it does now). Note: Time could be set to 12:00AM and the event will NOT be designated 'All Day'.
To specify an All Day Event: Enter Start and End data as usual. Check the 'All Day' checkbox option. Javascript might be used to disable the time entry boxes as they are irrelevant when 'All Day' is TRUE.

As I understand it, the biggest challenge for this plan is adding the 'All Day' flag to the database tables. Once a field's tables are established, Drupal makes it VERY difficult modify that schema.
As Date 7.x-2.x does not yet have an official release, it might be possible to do this, but it might break all existing sites that are using Date 7.x-2.x-dev. (~14,000 sites atm using 7.x-2.x-dev or 7.x-2.x-alpha3)

Next, of course, is a whole bunch of rewriting the date module... essentially a complete rewrite. This also has big implications for anything that uses date field data (such as calendar module).

Lastly, the 'All Day' checkbox must be added to each date field, probably along with some javascript as noted above.

arlinsandbulte’s picture

Title: Default value for To date: Blank should stay blank » To Date & All Day Date Handling (was: Default value for To date: Blank should stay blank)

This issue has evolved a long way from "To date: Blank should stay blank".

In fact, my latest proposal eliminates 'blank' To Dates (or end date, we use the term interchangeably) in favor of keeping the To Date always populated.

mansspams’s picture

all day checkbox could be separate module that alters widget, display and views results. then it is just matter of removing current flawed all day system.

BrightBold’s picture

To specify a Milestone Event: Enter Start and End data, making sure they are both equal. Note: Time could be set to 12:00AM and the event will NOT be designated 'All Day'.

I like your thinking, @arlisandbute, but I want to make sure we are taking content creators into account as we make this change. It's not intuitive for people to enter an end date/time for something that doesn't have an end date, (just as it's not intuitive to enter 12:00am to indicate an all-day event.)

So while it may be difficult to store a NULL end date (in that as KarenS points out in #523218-37: Default value for To date: Blank should stay blank it may break the behavior of Views, arguments, filters, etc.) it seems to me that asking the content creator to enter both a start and end date is worse than letting them leave the end date blank and then filling it in with the start date. I'd rather see that behavior continue if we find an empty end date breaks too many things. BUT... wait... I am testing this now and it appears that Date 7.x-2.0-alpha3 is successfully leaving the end date blank; not only did a blank end date stay blank, but when the start and end date/time were set to be equal, the end date actually became blank after I saved it. So it appears that the Milestone Event portion of this may already be solved. (Can anyone else confirm?)

So perhaps the only piece of this that's left is the All-Day handling, which in my mind is the most important piece. It would be great to see this handled as part of Date core, but in the meantime maybe people can help port the Date Tweaks module, which includes this functionality, to D7?

arlinsandbulte’s picture

@BrightBold: Yes, I think we are on the same page. I updated #17 to add "if end is left blank, it is populated with the same value as the start when saved, just as it does now".

HOWEVER: About your keen observation:

BUT... wait... I am testing this now and it appears that Date 7.x-2.0-alpha3 is successfully leaving the end date blank; not only did a blank end date stay blank, but when the start and end date/time were set to be equal, the end date actually became blank after I saved it.

You are right! (I tested with latest -dev.) I was not aware of this...
*BUT* the End date value IS still being set to the same value as the Start in the database. So, I am guessing Date module is really still doing the same thing, but on display, it checks to see if the values are equal. If they are equal, Date does not populate the End Date form so it appears to be 'blank' even though it really is not. Pretty clever on Karen's part if this is intentional.
@KarenS, can you verify this?

Anyhoo, This really does not change my above proposal in #17.

KarenS’s picture

Title: Default value for To date: Blank should stay blank » To Date & All Day Date Handling (was: Default value for To date: Blank should stay blank)
Status: Active » Fixed

I actually don't know when I made this change, but obviously somewhere along the line I did. Yes, the storage is left alone and the 'To' date is left blank if it is an exact match for the 'From' date. That keeps my code from breaking but keeps the display clean and simple.

So I think that addresses the original issue. There is a separate issue about setting an 'All day' flag, or at least there was at one time. Either way I'd like to keep that discussion in a separate issue.

The last thing that might be a nice feature here would be to expose a checkbox to the user for dates with optional end dates where a little javascript would hide the end date if empty or unless the user checks that box. I am not enough of a javascript guru to write that, but if someone wants to create an issue for that and someone makes a patch, I would be open to that idea.

KarenS’s picture

Title: To Date & All Day Date Handling (was: Default value for To date: Blank should stay blank) » To Date & All Day Date Handling
Category: bug » feature
Status: Fixed » Active

I probably marked this fixed prematurely.

I added some code to let the user of a date with an optional from/to field show/hide the To date. If they hide it and submit the form that way, the validation replaces whatever is in the To date with whatever is in the From date.

I added some additional code to let the user show/hide the time with an 'All day' flag. If they choose the option to hide it and submit the form that way, the validation sets the time for both the from and to date to 00:00:00.

The above two changes need some real world testing from anyone who can.

There is already code that makes the 'To date' appear blank if it matches the 'From date'.

The last question/task is whether we need an 'All day' flag in the database. I am starting to lean in that direction. My hesitation is that I have to write an update hook that will be very very very messy.

arlinsandbulte’s picture

Great work Karen!
Testing your changes mentioned in #23, all seems to work OK, but it is throwing some error messages. I think I will put them in separate issues and refer back to this one:
#1234090: "Undefined variable: all_day in date_combo_validate()" upon node save.
#1234092: Notice: Undefined index: value2 in date_combo_element_process()

About the 'All Day' flag
I was once of the opinion 'no, we don't need an all day flag value' as you can tell from #4 above. However, the more I dug into the issue and the more I thought about it, the more I became convinced that adding an 'All day' flag is the right way to go:

  1. The MySQL database datetime & timestamp data types ALWAYS include a time value. So, we cannot use the absence of time as an 'all day' indicator as I proposed in #4. We might use separate Date & Time fields, but that would increase processing complexity, IMO.
  2. Using time = 00:00:00 to indicate 'All day' makes it impossible to assign a milestone event at time = 00:00:00 in some use cases.
  3. By using an 'all day' flag, we can force timezone correction to NOT APPLY to 'all day' events. Thus, we can assign Christmas to ALWAYS fall on 2011-12-25(all day). Without this, I can store Christmas as 2011-12-25T00:00:00 which makes it all day. But if another user with different timezone settings views the event, the event time is changed, and it becomes a milestone event at their timezone's hour offset. The opposite can also happen. I can specify a milestone event. If another user views the event with timezone settings such that the milestone time falls on midnight for them, it will be shown as 'all day.'

In the short term, we might want to continue as-is with the improvements you started in #23.

I think changing the database schema to add an 'All Day' flag should be done in Date3 (7.x-3.x) and made a top priority for that release.
Yes, an update hook might be messy. And, no matter what, I think some users will need to manually configure 'All Day' settings for each of their Date fields after they make the switch... or do some sort of custom database migration... ugh...
Another option might be to make a field creation setting of 'Enable All Day Dates.' Thus, the user would have to choose this option when creating the field, and the flag would be added to the field definition. However, it could not be changed later, and all legacy data would be stuck without 'All Day' support... ugh & ugh. (D7's limitation of no allowable field edits that can change database schema really sucks... is there an issue somewhere?)

So in summary, here is what I think we should do:

  • Continue improving Date 7.x-2.x WITHOUT adding an 'All Day' flag.
  • In the next version of Date (7.x-3.x), make adding an 'All Day' flag top priority. This will require a lot of work, both in module code logic & an update hook to migrate date data from earlier versions to 7.x-3.x. Dependent modules, such as Calendar might also require a lot of work.

Adding the 'All Day' flag is certainly not the easiest path to take. But, I think it is the best option. Long term, it will make Date module more robust and easier to maintain & troubleshoot, IMHO.

KarenS’s picture

I'm not going to carry two versions of Date :( My plan is to finish the current work, cut a release of Date 7.2 (which will be the last 7.2 release), then release Calendar 7.3 and deprecate Calendar 7.2, then immediately create a Date 7.3 branch and deprecated Date 7.2. I can then remove a bunch of Date code that is no longer needed for the new Calendar version. As a part of that change I want to add in any architectural changes, and the only one I anticipate is adding an all day flag.

The update hook would be really really really messy -- it will be a batch that could take a long time to run for sites that have a lot of dates and it is hard to write and test updates. You only get one chance and if you don't do it right it's really hard to fix. I'm leaning in the direction of adding the flag, but still not sure I want to do the hook.

If the flag is going to get added it needs to get added now, before we have tens of thousands of sites using this version.

arlinsandbulte’s picture

Ok, I think I understand.

About the required update hook, yeah, I know.... really messy, but it probably has to be done.
here is a quick plan on what I think the update would need to accomplish:

  1. Add an "All Day" Boolean TRUE/FALSE field column to each date field table in the database
  2. For date fields where "Collect an end date" setting is not enabled:
  • IF timezone handling = "Site's time zone" & convert_to_site_time(time_value) = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
  • IF timezone handling = "Date's time zone" & time_value = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
  • IF timezone handling = "User's time zone" & convert_to_site_time(time_value) = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
  • IF timezone handling = "UTC" & time_value = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
  • IF timezone handling = "No time zone conversion" & time_value = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
  • For date fields where "Collect an end date" setting is enabled (required or not):
    • IF timezone handling = "Site's time zone" & convert_to_site_time(time_value1) & convert_to_site_time(time_value2) = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
    • IF timezone handling = "Date's time zone" & time_value1 = 00:00:00 & time_value2 = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
    • IF timezone handling = "User's time zone" & convert_to_site_time(time_value1) = 00:00:00 & convert_to_site_time(time_value2) = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
    • IF timezone handling = "UTC" & time_value1 = 00:00:00 & time_value2 = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE
    • IF timezone handling = "No time zone conversion" & time_value1 = 00:00:00 & time_value2 = 00:00:00 THEN "All Day" flag = TRUE ELSE "All Day flag = FALSE

    If I understand everything correctly, this should properly set the "All Day" flag for ~99.9% of all sites.
    There may be some sites that desire a different condition to set "All Day" flag properly for their uses, but I don't think we can realistically account for all those possibilities and options.

    Also (on a somewhat related note):
    Because D7 does not allow changes to a field's database schema once that field has data, what if we would just include ALL database fields by default (like To Date, rrule, date's timezone, etc.) and only use them if needed? That way, users could edit field settings more like they could in D6...
    Not sure if that is a good or bad idea... just throwing it out there.

    paulgemini’s picture

    I tested this and when I save the "To" date of an "all day" event, the my content deletes the to date and brings me back to the edit node interface. If that makes sense.

    KarenS’s picture

    I found the problem in #27, it only affects the Popup date, we have to let it pass validation with the time missing if it's using the all day flag.

    paulgemini’s picture

    Right - that makes sense - it's a validation issue. Any ideas on a patch?

    paulgemini’s picture

    I tested it using text input, and that works. HOWEVER - the dropdown field selector also doesn't work.

    All Day is an awesome addition, BTW. Once this is working, I can stop putting Google Calendar in a stupid iframe and start using Drupal!!:)

    jastraat’s picture

    Unfortunately, the commit this morning (August 1st) hasn't entirely fixed the problem regarding the all day checkbox + popup date. On creating a node, I get the following error:
    The value input for field Date Start date is invalid:
    The value 08/25/2011 does not match the expected format.

    This can be avoided by entering midnight for the time and not checking the all day checkbox. The error is not present when saving an existing node however. The all day checkbox can be left checked (as it is automatically if midnight was entered), and the node saved without a time.

    KarenS’s picture

    jastraat, I suspect you do not have the fix. The behavior you are describing is what it was doing before I fixed it. The commit was just made this morning. If you are using the tarball you don't have it. The bug affects only the Date Popup widget.

    jastraat’s picture

    Karen,
    That was my initial thought as well, but before posting my feedback, I checked the diff from Commit 6c7f313 and the new code is in the latest tarball (and the version that I'm testing).

    paulgemini’s picture

    Are we sure the change was committed? I've replaced my Date directory multiple times and update manager is still showing that I have Date 7.x-2.x-dev from August 31st...

    KarenS’s picture

    The update manager uses the tarball. The tarball is only updated twice a day, so the tarball is often missing the current day's commits.

    paulgemini’s picture

    Ohhh, I see. Interesting. Didn't realize that. Thanks. Now it's updated. Appreciate it!

    atolson’s picture

    subscribing

    DamienMcKenna’s picture

    KarenS, I updated from git but am still getting the error. My field allows a "to" date and I entered the following in the node form:

    • All day: unchecked
    • Show end date: unchecked
    • Date: Aug 23 2011
    • Time: [blank]

    Hit Save and I get the following:

    • An error stating:
      The value input for field Date Start date is invalid:
      The value Aug 23 2011 does not match the expected format.
    • The Date field is blank.

    What I was expecting:

    • Each subelement within the field should individually indicate whether it is required, i.e. both the Date and the Time fields should show they are required.
    • If there is an error with the data, e.g. the Time was missing when required, they should still display the incorrect current values when the form is displayed again rather than blanking them out.
    • If the time field is required but missing, the error message should say e.g. "A valid time is required for [fieldname]."

    Update: the "An illegal choice has been detected" message I got was from another field on the form, sorry about that.

    DamienMcKenna’s picture

    Also, having the "All day" checkbox checked and having a valid date still results in the "invalid" message.

    bjlewis2’s picture

    Using the latest dev: Date 7.x-2.x-dev (2011-Aug-01)

    I am also getting the following error when I try to use the "All day" check box.

    • The value input for field Date Start date is invalid:
      The value 08/17/2011 does not match the expected format.

    I am also getting these errors, in case they are related.

    • Notice: Undefined offset: 0 in date_combo_validate() (line 359 of C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\website\d7\sites\all\modules\date\date_elements.inc).
    • Notice: Undefined offset: 0 in date_combo_validate() (line 360 of C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\website\d7\sites\all\modules\date\date_elements.inc).
    • Warning: implode() [function.implode]: Invalid arguments passed in form_error() (line 1598 of C:\Program Files\Apache Software Foundation\Apache2.2\htdocs\website\d7\includes\form.inc).
    KarenS’s picture

    I just fixed #1241576: 'All day' dates are always invalid. Let's don't make this huge issue a bucket for every bug with this functionality. This was a feature request. The feature was added. The only remaining issue on the feature request is whether to add an all-day flag to the database. If there are bugs with the committed code, let's file them separately. At a minimum I need to know what kind of field and widget you are using when you say it is broken.

    There is already another issue about the fact that dates that fail validation get blanked out. I have no clue how to fix that.

    arlinsandbulte’s picture

    Ok, regarding the only remaining discussion point for this issue: Adding an all-day flag to the database.

    There are 2 reasons in favor of adding this flag to the database:

    1. Proper support of (milestone) events that occur at 12:00am. Right now, if a date with no end date falls falls on 12:00am (=00:00), it is assumed to be all day. If a date WITH an end date has both times set to 12:00am, it is assume to be all day. Thus, it is impossible to create a milestone event that occurs at 12:00am.... doing so always results in making the date 'all day.'
    2. Time zone conversions to 'all day' dates. Now, a date might be 'all day' in one time zone, but if it is adjusted for a different time zone, the date no longer occurs at 12:00am (=00:00), and it is no longer 'all day.' IMO, in 99.99% of all cases, time zone conversion should NOT be applied to 'all day' dates.
    KarenS’s picture

    It is not going to be possible to selectively apply timezone conversion or not to dates that use the same date field. If we did, our Views queries would not work. I should say it is perhaps technically *possible* to write a query to do that, but it would be an enormously complex query that would be highly likely to break (it would have to have some sort of if/then logic to do different queries and retrieve different values based on the setting of that flag). Not only that but it would be very difficult, if even possible, to write a query that would sort correctly a combination of dates that need timezone adjustment and those that don't.

    So an 'all day' flag for a field that has any dates that might need timezone conversion will only mean that the time should be ignored when displaying the date. Internally we will still have to store it as midnight and do the timezone adjustment. By doing that we can still sort the dates correctly (all day dates will sort to the top of the list for any given day) and sanely query a combination of datetime and all-day dates to see if they are <> a local datetime value.

    Going any farther than that will take us down the path of figuring out how to do 'fuzzy' dates, something that I am not even going to attempt yet because I'm not sure at all how (or if) it could be done.

    arlinsandbulte’s picture

    Ugh, you sure know how to burst someone's bubble, KarenS :-P, lol, :-)
    Like Larry (aka Crell) once said: Dating is complicated....

    So, then I suppose the only (practical) reason to add the 'all day' flag to the database is the first one in #42 above....Proper support of (milestone) events that occur at 12:00am.

    P.S.
    Just a thought: perhaps the 'all day' database flag should be added for the proper support of events that occur at 12:00am now. Then, if we (and by we, I mean Karen ;-) decide to tackle the complex query thing, or we have a revelation on how to easily & efficiently disable timezone conversion for 'all day' dates, that could happen later.

    The improvements made thus far in this issue are GREAT and go a long way (95%?) to resolving this issue (which I forked almost exactly one year ago... this discussion has been going on since July 18, 2009!
    Furthermore, all the changes up to this point should be portable to the D6 version of Date, I think.
    Thus, I would not be entirely opposed to postponing the database 'all day' flag until D8....

    I'm certainly no expert in databases or queries.
    I just try to provide an unbiased end-user's perspective. I hope you are not bothered will all my naive questions & suggestions, KarenS.

    thebuckst0p’s picture

    KarenS, I have the all-day/to-date problem on a Drupal 6 site, and I've complained publicly about the bloat of the Date module in 6 and 7, so I'd like to help simplify it if I can, and specifically backport the simplifications to Drupal 6. What's the best way to get involved with that process?

    Thanks,
    Ben

    Cyberwolf’s picture

    Subscribing.

    wusel’s picture

    Subscribing for D7.

    arlinsandbulte’s picture

    This might be D8 material, but I am tagging 'D7 stable release blocker' anyway for further discussion/review.

    Joel MMCC’s picture

    Another wrinkle here is that I can foresee needing a few more additional kinds of Date/Time handling of Events beyond the three that arlinsandbulte described:

    • Time TBD: The Date is known and has been announced, but the Time has not been finalized yet, or has not been officially announced yet. This is not an All Day event. It’s either Milestone or Time Slot, but with a Time that is currently unknown or not to be revealed just yet.
    • Midnight/Noon: Date and Times are entered normally, and we would want the Time displayed normally unless it is either “00:00:00” or “12:00:00”, in which cases it should display as simply “Midnight” or “Noon,” respectively.

    How should these be handled in a way to distinguish from All Day and Midnight Milestone events? Do we need yet another Flag?

    Or, how about replace the “All Day” Boolean flag with a special TinyInt Enum value (I believe MySQL already basically treats Booleans as if they were TinyInts with values 0 or 1), such that 0 = Display Time as Normal, 1 = All Day (that should work for backwards compatibility with existing development code), 2 = Time TBD, 3 = Midnight/Noon, etc.? This TinyInt could be called “Type” or some such.

    This would leave us 250 other values for future functionality (e.g. 4 = Custom Time Display String [that one would of course require an additional VarChar field to hold the string], etc.).

    These could be displayed in the UI as a drop-down menu or (if short) horizontal Radio Button group (less mouse action required than with a drop-down).

    Perhaps instead of a sequential Enum, it could be a bitmapped flags type (still TinyInt for now), perhaps something like this:

    • Bit 0 (1): All Day (should be backwards compatible with Boolean)
    • Bit 1 (2): From Date TBD
    • Bit 2 (4): From Date Midnight/Noon
    • Bit 3 (8): From Date Custom
    • Bit 4 (16) (reserved for future feature use)
    • Bit 5 (32): To Date TBD
    • Bit 6 (64): To Date Midnight/Noon
    • Bit 7 (128): To Date Custom

    Any possible combination would be unique. Simple bitwise AND masking can detect them, and OR (or simple addition) can build them.

    Of course, we also need to make sure that strings such as “All Day,” “Midnight,” and “Noon” are properly localized on display.

    arlinsandbulte’s picture

    Regarding TBD (I Propose WON'T FIX):
    I think I can safely say this WILL NOT be addressed anytime soon if ever in the Date/Calendar Modules. I understand the desire, but, right now, this just adds too much complexity to an already complex module. This is essentially the same thing as #259308: Allow "fuzzy" granularity, which Karen has decided not to pursue in the Date/Calendar modules. It may happen someday, but not in the forseeable future, IMO.
    Furthermore, mainstream date/calendar software has little or no support for TBD either. MS Outlook, Lotus Notes, Google Calendar, etc. do not support TBD time. So, I don't think this is a 'must have' feature.
    The date module DOES support optional dates, so that feature could be used in leu of a specific TBD feature.

    Regarding Midnight/Noon (I Propose POSTPONED):
    Theoretically, this is perfectly feasable now using theme functions. I see no reason why date needs to do anything special for Midnight/Noon times. The date/time value does not need to change or have anything special attached to it. Only the way it is rendered and displayed needs to be tweaked. Thus, theme functions sounds like the best way to accomplish this IMO.

    Joel MMCC’s picture

    I disagree about TBD being like “Allow ‘fuzzy’ granularity.” I’ve been following and even posting in that thread for quite some time (I’m the one who pointed out that MySQL DateTime types have this ability already, but that relying on that for Drupal would’ve been problematic even for D6 due to its support of PostGreSQL, let alone D7’s PDO support of most any database), and this is not at all the same thing.

    This is simply allowing a Time to be entered or left blank and defaulting to 00:00, but telling it to display as “(Time TBD)” (translated if need be) for the time being. This would be for, for instance, sports games that will have a definite starting time, but we just don’t know what it is yet. We do know the date, and will know the time, but that has not been finalized yet so we don’t want to display any time, nor do we want to falsely state that it’s an “(All Day)” event as happens now.

    Fuzzy granularity goes way beyond that, and allows for things such as “December, 2011” without specifying a day, or “March 15” without specifying a year (handy for an annual Ides of March event without having to resort to Repeating Dates), or even “Twentieth Century” without specifying anything more than that. That’s not what I’m suggesting.

    I’m not familiar with Theme Functions, and will take your word for it that they can do the Noon/Midnight thing. But how many casual non-programming Drupal site builders would know how to do that? I assume we’re talking getting down and dirty with PHP code?

    KarenS’s picture

    My initial reaction was the same as arlinsandbulte's about the TBD, but I see now you are just talking about different ways to display a time stored as 00:00:00. I get the idea and see what you're suggesting, but the problem in my mind is that each of these will need a way for the user to indicate what they want, so we not only need a way to store the value, we also need a way for them to select it. And that is going to make the UI quite messy. I already have people creating issues saying the all day checkbox is too much clutter.

    Let's leave the current task to the current task -- get the all day functioning working completely. Adding things like TBD can be feature requests for a future version. I need to get a release out and don't have time to add more new features or it will never get done.

    KarenS’s picture

    And if you open up a feature request for TBD, I want to suggest that you propose it as some sort of a hook or plugin, where Date would just provide a way for other modules to insert other display options into the UI, and then actually put that functionality into a separate module that can be enabled for people who want it. We might even want to rework the all day option in the same way. I want to get these kinds of things more modularized to avoid adding more complexity to the basic date functionality.

    arlinsandbulte’s picture

    Please correct me if I am wrong, but 00:00:00 a perfectly valid time that signifies midnight.
    But, 24:00:00 in MySQL datetime gives me an out of range error. We cannot store a time of 24:00:00.

    Thus, we CANNOT use a time of 00:00:00 to signify TBD event. Right now, 00:00:00 is used to signify an ALL DAY event, and that causes problems because that makes it impossible to create an event @ midnight (00:00:00). If we did the same for TBD, we would have the same issue.

    KarenS’s picture

    If we switch to using a flag to indicate what 00:00:00 means it can mean either all day (or something else, if we later add that). The idea, as I understand it, is to have a variety of ways we might want to use a date that is set to 00:00:00. The actual storage in all these cases will be the same, they will all have the 00:00:00 value, the flag(s) will indicate what it actually means. This will work only if the meaning is just having different ways of displaying the value (like to have the option to call it 'All day' or 'TBD'). And for now I don't want to get that complicated -- the only possible meanings will be 'midnight' or 'All day'.

    Joel MMCC’s picture

    What KarenS said. I was talking about adding a bitmapped or enum-type TinyInt flag that would indicate what “00:00:00” time would actually mean, on a per-Event basis.

    The particular case that this has come up for is a sports commission website for a multi-city metropolitan area. They want a list of upcoming sports-related events in the area. Some of those truly are All-Day events (e.g. tournaments, sports education seminars, etc.). Some others have dates that have been determined, but times that have not yet been determined. Some others may actually start at midnight (none yet, but it could happen: a midnight weekend basketball game for inner-city youth, for instance). All three of those would have a time of “00:00:00” — we need the capability of specifying which one, and we need all three to be available choices.

    Right now, all such Events display as “(All Day).” The only way (short of mucking around with the theme, which has ramifications if the theme gets an official update) to keep that from displaying is to disable the display of time entirely, which would of course harm the display of those events that actually do have a set time.

    Another possibility would be to add a short Char or VarChar field (say, VarChar(16)) that would allow the user to specify, on a per-Event basis, what text to display for a zero time. Default, if empty string, would be normal time (12:00 AM). Any text could be put there: “(All Day)” or “(Time TBD)” or “Midnight” or “Whenever” or whatever the user wants (translation may be an issue here). If an actual non-zero time were entered, that field would be overridden. This is basically what I meant by “custom string.”

    The best solution, IMHO, is to do both: the enum or bitmapped TinyInt would handle the most common cases (All Day, TBD, Midnight), allowing those to appear as a radio button group or drop-down menu on the Add Event form. The Midnight choice would also tell a time of “12:00:00” to display as “Noon.” The Custom choice (which would appear as an “Other” radio button or menu item) would then use the Custom String field (which the user could type into, and which would be enabled only if “Other” were selected).

    KarenS’s picture

    I don't know that the enum or bitmapped options are cross-database compatible. We are supporting a number of databases other than MYSQL. I have headaches enough just trying to support a datetime field that core won't handle, I'm not diving down that rabbit hole again. It looks like the only option I can be sure will be supported is straight up int:tiny. And again, I can't visualize any way to make the UI for this nice. You would need more than the All day checkbox, and that's already too cluttered for some people's taste. So all of this has to go into its own feature issue that will be considered at another time, perhaps only if there's at least a proof of concept patch to look at.

    For now it will probably be a tinyint field where 0 means 'midnight' (the standard meaning of the value) and 1 means 'all day'.

    DamienMcKenna’s picture

    ... how about just building a way to customize the output when the time is NULL?

    arlinsandbulte’s picture

    The time cannot be stored as NULL, MySQL datetime field type will not allow it.

    KarenS’s picture

    And there is no way to represent a NULL time in a unix timestamp either.

    wusel’s picture

    like http://drupal.org/node/523218#comment-5292012:

    Can you add e.g. an NaD-value ("Not a Date") to the date module? NaD may be stored as the first second of the technical possible values of the date-field (in the past) as an constant.
    And the next second of the technical possible values of the date-field (in the past) could be unsed as an constant for "allday" and so on.

    This or a blank value or something else is neded to show, that a birthday-field has no value, if we don't know the date of the birthday of a member or user.

    Thank you very much.

    KarenS’s picture

    We don't need a flag to indicate the whole field is empty, that would be NULL value and we already can handle that just fine. This issue is just about handling an empty time. I don't want to explode this task into every other possible feature anyone might want to add. This is just about handling dates that should be marked as being 'all day'.

    Joel MMCC’s picture

    I wasn’t talking about an actual enum or bitmapped field, but rather a TinyInt (supported by all SQL database engines) that your code interprets as an enum or bitmapped field. Likewise, not all engines support Booleans, or at least not implemented the same way (MS SQL treats them as bit fields, combining up to eight Boolean columns into one byte, up to sixteen into two bytes, and so on; while MySQL treats them as TinyInts that are either 0 or 1 [newer versions of MySQL do have a bit type, but Boolean remains TinyInt]).

    “For now it will probably be a tinyint field where 0 means 'midnight' (the standard meaning of the value) and 1 means 'all day'.”

    That leaves 2 through 255 for future expansion without having to add yet another field. :-)

    As for user interface, as I said, a drop-down or horizontal radio button group should suffice. The latter would be a single click on the user’s part, and then only if s/he’s using a non-default option.

    Example:

    “From” Time of Zero means: ◉ Normal Time (00:00, 12:00am)   ○ All-Day Event   ○ Time TBD   ○ Normal (display as “Midnight”)
      “To” Time of Zero means: ◉ Normal Time   ○ Time TBD ○ Normal (display as “Midnight”)

    Those would render as “<input type="radio" name="zero_from_time_type" value="#" />” tags where “#” is 0, 1, 2, or 3, respectively, and your code could just stuff that value directly into the field.

    KarenS’s picture

    The above example changes a single checkbox that is floated next to the 'End time' checkbox into two lines with multiple options. That is quite a bit of additional clutter, more than I personally like. But, as noted before, adding anything more than All day handling should go into a different issue.

    On the All day issue, I have pulled all the All day handling out into its own module by injecting a number of drupal_alters in strategic places that the new module can implement. This cleans up the code, turns the feature into something that can be enabled or disabled, and provides an illustration and a proof of concept for ways that other modules can inject processing steps into Date fields. I'm doing a bit more testing and will commit that soon.

    The next step after that is to add a field to store the All day data. Since I want this to be something that other modules can plug into and I don't want different modules coming up with different ideas about what a value represents, my new idea is to add a 'data' column to the date fields. This would contain a serialized array of key/value pairs. The All Day module can add a field that uses the 'date_all_day' key and other modules can add other items. This seems like the easiest way to keep everyone from colliding with one another. Another advantage of that is that I can also move the RRULE into that array and drop the RRULE column, since the RRULE is only used on repeating fields, and even there the value is only needed by the edit form and the formatter.

    KarenS’s picture

    Another thought on data storage, in addition to the data array I could add a tinyint field called 'Has Time'. That field could be set to 1 to indicate the date contains a valid time or 0 to indicate the time in this field should be ignored, basically the 0 would the same as saying the time is NULL (since we can't actually store a NULL value for time in all our date field types). That give us one generic, queryable field, and the key/value pairs in the data array added by other modules could be used to store settings, like how to display that value. The idea is that the core Date module will not do anything with this field except create it (and perhaps mark 0 for the obvious case of fields that don't have any time granularity), then other modules, like Date All Day, can manipulate it for dates that would otherwise have time but have been flagged as all day.

    KarenS’s picture

    The new Date All Day module has been pulled out in this commit:

    http://drupalcode.org/project/date.git/commit/b650e47

    tim.plunkett’s picture

    Ouch, fullcalendar was using date_field_all_day(). I guess I need a hook_update_N to enable the new module for them?

    tim.plunkett’s picture

    Upon further reflection, this is going to be even harder to work around, since the function name changed.

    Can date_field_all_day be reintroduced to date.module, with @deprecated in it, and removed in 7.x-2.1 or later?
    If you remember, I put in several patches to fix the logic in this function, and fullcalendar is completely reliant on it.

    KarenS’s picture

    Yes, provide a patch if you can and I'll add it back.

    KarenS’s picture

    Never mind the patch. I added back the old function for backwards compatibility and left the new one alone. We need to think about where this really belongs because the concept of what is 'All Day' is going to be affected by how we handle the All Day question going forward.

    KarenS’s picture

    The more I think about this the more I like the change of having an All day or No time flag and moving a lot of other values into a serialized array. But that is more of a change than I originally planned, so I am going to put that off to do in a new 7.3 branch, to indicate that the changes are more major.

    I'm going to take this issue off the release blocker list since it doesn't have to be fixed before release. I did get the All Day and Date Repeat Fields into their own modules, which cleans up a lot of stuff and makes it easier for other modules to alter the processing. That's enough for this release and this branch.

    Joel MMCC’s picture

    How about this as a simple UI addition to the Date All Day sub-module?

    Right now, you have it display the pre-set string “(All Day)” in place of the Time if that Module is activated and the Time is 00:00:00, right?

    How about simply letting us change that string on a per-entry basis? No need for a TinyInt flag, just a short VarChar (say, 16 characters max) to store the text to display. It would be like my “Custom” suggestion above. It would only appear in the UI if the Date All Day Module is Enabled, thus not cluttering the UI for those who have no need for this.

    It would default to “(All Day)” to mimic current behavior for backwards compatibility. But, if users need to, they can type in, on a per-entry basis, “Midnight,” “(Time TBD),” or even “12:00 AM” if they so choose. Or even “Yeeehawww!” if they really want. Whatever.

    You could also have a drop-down or radio button group that would insert preset common strings into that with a mouse click (again, these would only show up if the Date All Day module is Enabled, thus no additional UI clutter for those who don’t want this functionality), rather than setting a TinyInt flag. Or you could perhaps make it have “combo box” functionality, in which it appears as a drop-down menu with preset choices available, but the user can override it and type in any desired string of no more than the maximum field length. This would also handle localization / globalization.

    rbrownell’s picture

    Hello;

    I have stumbled up on this issue while I was attempting to configure a View to show nodes based on a To Date in the future or if there is a Null To Date. Having select a Null To Date does not yield the views results that I am looking for. As a result I decided to poke around in the database in my site to discover that to select Null is not going to work because if unspecified the To Date field stores the same value as the From Date field.

    Having the To Date match the From Date conflicts with View's ability to filter based on a field being Null. Should I create this as a separate issue to suggest finding a solution for things like this in Views or does it belong here?

    Thanks;

    -Ryan

    ymeiner’s picture

    I kind of found a solution, not clean and need to be checked but here are the changes i did to make it work in drupal 7.

    in date.module, function date_formatter_process - there is a condition that checks if there is value2 so i created a flag in it.
    original:

      if (empty($dates['value2'])) {
        $dates['value2'] = $dates['value'];
      }
    

    after change:

      if (empty($dates['value2'])) {
        $dates['value2'] = $dates['value'];
    	$dates['value2']['is_empty']=true;
      }
      else {
    	$dates['value2']['is_empty']=false;
      }
    

    In date theme, function theme_date_display_combination - i added another case after:

      if (empty($date1) && empty($date2)) {
        $output .= '';
      }
    

    Here is the edition:

     elseif ($date1 == $date2 && $options['fromto']=='value2'&& $dates['value2']['is_empty']==true) {
        $output .= '';
      }
    

    So if the dates are the same (because there is only one value) and the option fromto is set on the second value and the second value is empty, it displays nothing.

    lpalgarvio’s picture

    das-peter’s picture

    I just posted a patch for the views filter handler provided by the date views module here: #1635810-3: Views filtering of optional end dates

    tmsimont’s picture

    I'm looking at all these comments and see a lot of posts about how to handle 00:00:00 time and all these crazy ideas about what that represents...

    It appears that right now the date_is_all_day() function treats 00:00:00 time as an "All day" value.

    Isn't it kind of counter-intuitive to try to redefine the meaning of 00:00:00 time? AFAIK 00:00:00 in a database means 12:00 a.m. -- nothing more, nothing less. Why are we calling that "All Day" or anything other than just "midnight"

    What if you have an event with no "to" time that starts at Midnight? If you have this, and you also want "All Day" support then you're boned. As KarenS said, adding a selection for how to handle 00:00:00 introduces too much clutter. I totally agree -- I don't think we should ever treat 00:00:00 as anything other than midnight, otherwise you confuse and clutter the UI trying to provide ways around the ambiguity.

    KarenS has an interesting idea in #65 of a "has time" flag, but I can see how that could severely complicate views.

    In my opinion, there's only 2 logical ways to move with this:

    1) Create a storage mechanism for Dates without time. This would be equally as useful and closely related to times without dates: #104287: Time without a date feature. Then in separate db fields or tables, store settings for how to handle Dates without time. You could potentially have different field types or field-specific settings for what to do with timeless dates.

    2) Set "All day" times as 00:00:00 TO 23:59:59, because that is truly "All day" rather than trying to trick out and redefine a 00:00:00 value. "All day" time does have a from and a to -- it's from the start of the day to the end of it. If a person's opinion is that all day is some other definition, like 8 a.m. to 5 p.m., that person should be setting a from and to time. Although I suppose you could potentially have an option to define what time range defines "All day"

    Those are my thoughts, just want to put them out there because it seems like discussion here has kind of stalled and the current 7.x branch still uses some odd logic to redfine midnight into "All day," which really doesn't work...

    vijaycs85’s picture

    Issue summary: View changes
    Issue tags: +sprint

    Lets add to current sprint.

    GreenSkunk’s picture

    @tmsimont a hybrid of #77.

    Option 2 ... What is All Day?

    User, global or field setting for what times the system should consider "All day".
    All Day = Start 07:30:00; End 15:30;
    Editing a date time field
    Click the "All Day" checkbox and the fields are populated with the configuration setting.
    Maybe this already exists ... I skimmed this issue but it is still open for a reason.
    More digging.

    bwoods’s picture

    I found this thread after reviewing the current Date module logic and came to the same conclusion as @tmsimont in #77. It seems weird to me to define "all day" as anything other than spanning an entire period of whatever a day is. I do agree with @GreenSkunk in #79 that perhaps a setting to define the span (while I think of "all day" as a 24-hour period, it's possible someone else might consider just daylight, etc.) would be best. I realize what's currently in the Date module is the cleanest, as there are no lookups, but it doesn't work if you have to consider people entering/viewing content from multiple timezones.

    Charles Belov’s picture

    @bwoods: Probably an edge case, but the San Francisco Municipal Transportation Agency's bus service works on a 5:00 am to 5:00 am day. This might be true of any company that has a 3-shift day with the overnight shift not starting at midnight.

    Liam Morland’s picture

    It seems to me that the thing that makes an event "all day" is the fact that its exact time is not known or not clearly defined. If you know that a conference lasts from 09:00 to 17:00, say so. But if the start and end has not been decided or varies or for other reasons, it may make more sense to just say that the conference is on a certain day, non-specific about its time. That doesn't mean that it starts at 00:00 and runs until 24:00.

    steinmb’s picture

    Category: Feature request » Bug report
    Priority: Major » Normal
    Issue tags: -sprint

    Found old "all day issue" - Perhaps this can be closed as outdated? If not the IS at least should be updated. Reading quickly though it and to me they discuss different problems though there could be all day edge cases in some of the comments.