Custom characters intended as separators within custom formatted date/time are not displayed in the correct location. [#1538420]

This is a deliberate sort-of duplicate of the 2¾-year-old #486862: “@” as Custom Format Date/Time separator character appears AFTER the Time, even if placed BEFORE! for 6.x-2.x, which KarenS said (in #6) had to be made into a separate issue for the 7.x version, so here it is. It’s still happening, even in the latest 7.x-2.x!

I made a Custom Format that looks like this:
“l, F j<\s\u\p></\s\u\p> @ g:ia”
and assigned it as my Long Date Format.

It’s supposed to display like this:
“Tuesday, June 9^th @ 12:34pm”

Instead, we see this:
“Tuesday, June 9^th @ 12:34pm @”

Note the second “@” that appears after the whole thing.

If I leave off the HTML that was added to make the ordinal suffix into a superscript (as it should be for best appearance), it gets even worse: the first (intended) “@” disappears, leaving only the second (incorrect) one, like so:
“Tuesday, June 9th 12:34pm @”

I triedc learing all caches, updating to dev releases, etc.

This happens wherever Long Dates are shown, regardless of whether it’s in Calendar Views (e.g. Upcoming Events), Event CCK or Fieldable Entity Details, etc.

It happens with all 6.x. and 7.x versions to date that I’ve tried.

I had also tried backslash-escaping the “@” and HTML tag angle brackets (basically every character that was not a PHP date/time placeholder or standard date punctuation [space, comma, colon, etc.]), like so:
“l, F j\<\s\u\p\>S\<\/\s\u\p\> \@ g:ia”
but to no avail.

I even tried using the word “at” (escaped as “\a\t”) instead of “@”! Same result.

But then I tried using characters other than an “@” — standard PHP date/time-separator characters such as colon (“:”) and comma (“,”) work fine (sort of — they display fine, but still generate undesirable extraneous HTML — see below). They’re not really what I wanted, but I suppose the comma will do for now.

Weird thing is, all of these look just fine, as intended, in the previews shown in the “Configuration → Regional and language → Date and time → Date formats” (…admin/config/regional/date-time/formats) page! It’s only when anything involving Date API displays them that they mess up!

In looking at the actual generated HTML code, it turns out that more than just the “@” is being duplicated in my case:

“l, F j\<\s\u\p\>S\<\/\s\u\p\> @ g:ia”
gives, for instance (in HTML, substituting single angle quotes for angle brackets here to prevent them from being interpreted as HTML tags):
“Tuesday, June 9‹sup›th‹/sup› @ 12:34pm ‹sup›‹/sup› @”

As you can see, the “‹sup›‹/sup›” tags and an extra space are being duplicated as well as the “@”! Those tags also wind up duplicated even if I use a standard separator character such as comma or colon or hyphen, but at least in those cases they’re empty and so have no visual effect (would still potentially mess things up for visually impaired accessibility methods, HTML validation and thus SEO rankings, etc.)

Others in that 6.x Issue thread reported that other non-standard separator characters also wind up duplicated, including (#2) bullets (“•”), (#8) surrounding the time portion in parentheses (causes empty parentheses to appear after the time, like this: “2/25/12 (7:00pm EST) ( )”), etc.

One @mkeiser there (#9) seems to have tracked down the cause to some RegExs in the date_limit_format() function (date_api.module file in the 6.x) and even proposed a potential fix which worked for him/her, but this has still not been fixed.

Comment	File	Size	Author
#14	Screen Shot 2015-06-18 at 1.44.46 PM.png	15.85 KB	akalata
#14	Screen Shot 2015-06-18 at 1.42.46 PM.png	9.13 KB	akalata
#14	Screen Shot 2015-06-18 at 1.42.37 PM.png	17.08 KB	akalata
#14	Screen Shot 2015-06-18 at 1.42.30 PM.png	17.37 KB	akalata

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

Joel MMCC CreditAttribution: Joel MMCC commented 18 April 2012 at 18:05

Note: Apparently the CSS on this site (line 58 of the aggregated cached .css file) negates the visual effect of (and for that matter) tags (Why!? Why allow those tags in the Issue Post Editor if you’re going to deliberately suppress their effect!? I know it wasn’t that way when I Created the old 6.x-2.x issue, — I would’ve noticed that! It was apparently done as part of the site redesign a year or so back), but they are there in my examples. The “th”s should indeed be superscripted in all of the display examples.

Comment #2

KarenS CreditAttribution: KarenS commented 18 April 2012 at 18:15

Status:

Active

» Postponed (maintainer needs more info)

Everything that is not a date format string must be escaped. That includes '@', '<', 's', 'u', 'p', '>', etc. When doing that everything works fine for me.

Date uses formats in additional ways -- to do things like figure out if there is a 'time' component in the string, so it is less forgiving than a simple date format string. But it ignores anything that is escaped, which is why anything that is not to be interpreted as a date part has to be escaped.

I have no idea what the comment in #1 means.

Comment #3

Joel MMCC CreditAttribution: Joel MMCC commented 18 April 2012 at 19:07

Status:

Postponed (maintainer needs more info)

» Active

#1 was simply explaining why the“th” doesn’t appear superscripted in the Issue in the display examples. The Drupal.org redesign .CSS file negated the effects of those tags. I’ll post a comment about that on a more relevant forum for the Drupal.org design team, if I can find one. It wasn’t addressed to you.

I did escape everything (and I do mean every single character other than actual PHP date formatting indicators) in my various trials, and it always still messed up, no matter what I escaped or didn’t escape.

As I said in about the middle paragraph of my post (slightly edited to replace angle-quote characters with actual angle brackets, as those are allowed now if done as HTML Named Entities):

I had also tried backslash-escaping the “@” and HTML tag angle brackets (basically every character that was not a PHP date/time placeholder or standard date punctuation [space, comma, colon, etc.]), like so:
“l, F j\<\s\u\p\>S\<\/\s\u\p\> \@ g:ia”
but to no avail.

Also, a bit further down:

In looking at the actual generated HTML code, it turns out that more than just the “@” is being duplicated in my case:
“l, F j\<\s\u\p\>S\<\/\s\u\p\> @ g:ia”
gives, for instance:
“Tuesday, June 9th @ 12:34pm @”

Note that that particular example didn’t escape the “@”, but I tried it with escaping the “@” as well, and it still did the exact same thing, including the duplicated empty set of “” tags.

Comment #4

KarenS CreditAttribution: KarenS commented 19 April 2012 at 09:26

Priority:	Major	» Normal
Status:	Active	» Postponed (maintainer needs more info)

It is not a major bug that makes the code unusable if you have a complicated date format that isn't working. And as I said, I can't replicate the problem. I have tried dozens of messy, complicated formats like this and they work for me. I just tried 'l, F j\<\s\u\p\>S\<\/\s\u\p\> \@ g:ia' and it also works fine.

Since I can't replicate the problem using the latest code there is nothing I can do with this. Maybe you have some other module interfering. Maybe you're not testing the latest dev code. See if you can reproduce the problem on a clean install with the latest Date module and nothing else installed, because I can't.

Comment #5

gagarine CreditAttribution: gagarine commented 31 May 2012 at 12:27

Comment #6

gagarine CreditAttribution: gagarine commented 31 May 2012 at 12:30

Status:

Postponed (maintainer needs more info)

» Active

KarenS, please look at the #1606900: Date format with special caracteres are printed two times perhaps you will find the info you need.

Comment #7

KarenS CreditAttribution: KarenS commented 23 July 2012 at 14:23

Status:

Active

» Closed (works as designed)

Your example in the related issue does not appear to be escaping special characters, which you must do. It doesn't matter if it works without escaping in format_date(), the Date API is doing much more complex things and requires the escaping.

If you can provide an example of a way that escaped characters is not working, that would be a bug.

Comment #8

Joel MMCC CreditAttribution: Joel MMCC commented 23 July 2012 at 16:03

Just wanted to add that the effect that I was trying to get, while it still doesn’t work properly, isn’t proper modern U.S. English style for dates and times anyway.

For U.S. English medium or long dates, you are not supposed to add the ordinal suffix (whether superscripted or plain), even though we do often add them when speaking the dates verbally. This is because we place the day number after the month name (whether full or abbreviated).

So, today would be “Monday, July 23, 2012” and not “Monday, July 23rd, 2012”.

As for times, the “AM/PM” must have a space separating it from the time. “12:43 PM” is okay, “12:43PM” is not. The preferred style format, if your system can handle it, is to use small caps for the am/pm indicator, but I don’t think that I can actually show that here with the limited HTML available for these comments.

In either case, you’d almost certainly want a non-breaking space between the minutes (“i”) and the AM/pm indicator (“A” for uppercase, “a” for lowercase), so that you don’t wind up with the minutes at the end of a line and the am/PM indicator at the beginning of the next line in a word-wrapped paragraph (you’d also want one at least between the month name and day, if not everywhere there is a space).

There is simply not enough room in the format strings to do this properly (with the small caps effect), especially if we have to escape each and every non-date/time-part-specifying character. Without modifying the CSS files and using only inline CSS, it’d have to be something like this:

l, F\&\n\b\s\p\;j \@ g:i\&\n\b\s\p\;<\s\p\a\n \s\t\y\l\e\=\"\f\o\n\t\-\v\a\r\i\a\n\t\:\s\m\a\l\l\-\c\a\p\s\"\>a\<\/\s\p\a\n\>

That’s 125 characters by my count. The “format” column of the “date_formats” table is of type varchar(100). It won’t fit.

The same effect could be had by modifying the relevant CSS file and adding a short class name such as “sc” for “small caps” and putting the “font-variant: small-caps;” declaration there. That would still be a pretty tight fit, though, depending on how long your class name is (keep it short!).

A similar visual effect, though not strictly the proper method, can be had by using “A” instead of “a” and surrounding it with a “” tag (properly escaped, of course).

I used that latter method on a site and it seems to work okay except that the original problem remains: the date/time separator character appears after the time as well as before.

Comment #9

KarenS CreditAttribution: KarenS commented 23 July 2012 at 18:14

You don't need to escape spaces. If you insist on adding html to the format string it is going to get long. All of this is done to be able to do smart analysis of the types of date parts that are in the format string, which is used in lots of ways. Extra, unexpected characters in the format string make that impossible. My response to the issue has been repeating that you have to escape non-format characters. I can't tell if you are saying it still won't work that way or if you are just complaining about this. This system was not designed as a way to add html, which is totally outside the way date format strings were intended to be used. They were never ever ever designed as a place to add html. If you want to do that you should be doing it in the theme where you have total control over the output, IMO. The date module has tons of themes you can override just so you can do whatever you want.

Comment #10

Joel MMCC CreditAttribution: Joel MMCC commented 23 July 2012 at 20:00

My post was intended as general information for US English users, to correct the erroneous format I had given before.

No, I’m not complaining about HTML. The problem happens even if I don’t use HTML, as others have also indicated. The problem has apparently been traced (in other related Issue threads such as gagarine’s #1606900: Date format with special caracteres are printed two times) to a specific RegEx in your code.

I’m not familiar with editing theme files. That requires PHP knowledge, right? I do have basic PHP know-how, but have never delved into Drupal-specific PHP coding (other than manually applying patches). Which date module theme file should I try editing first? What would happen to the edits to Date Module’s theme files when the Date module is updated? Is there a safe way to do such edits so that updates wouldn’t override them?

Comment #11

tiflopin CreditAttribution: tiflopin commented 7 November 2012 at 09:28

Version:	7.x-2.3	» 7.x-2.6
Priority:	Normal	» Major
Status:	Closed (works as designed)	» Active

With a custom date format like

l j F Y [G\hi]

The output of the date field is : Mercredi 7 Novembre 2012 [10h20] [h]

'[' 'h' and ']' are duplicated :-(

Is there a workaround ?

Comment #12

sassafrass CreditAttribution: sassafrass commented 29 November 2012 at 21:08

I have the same issue in D6 and D7. I use the following string in the date format field (note: the dash is an em dash character, not a regular dash) and the em dash ends up at the end of the date string.

l \— F j, Y prints as: Thursday November 29, 2012 —

Comment #13

Itangalo CreditAttribution: Itangalo commented 1 March 2013 at 08:46

For information: This is most likely an issue that can be solved in the same way as #752550: Week number gets printed twice.
(It's still a bug, and it should be solved, but there is a way for solving it.)

Comment #14

akalata CreditAttribution: akalata as a volunteer commented 18 June 2015 at 18:45

Version:	7.x-2.6	» 7.x-2.x-dev
Issue summary:	View changes

File	Size
Screen Shot 2015-06-18 at 1.42.30 PM.png	17.37 KB
Screen Shot 2015-06-18 at 1.42.37 PM.png	17.08 KB
Screen Shot 2015-06-18 at 1.42.46 PM.png	9.13 KB
Screen Shot 2015-06-18 at 1.44.46 PM.png	15.85 KB

I'm also having this issue, despite escaping as @KarenS describes. However, I'm only seeing it when field content is rendered, not in the admin Date/Time settings.

String format: M j, Y \@ g:i A

Settings display (with or without escaping) :

Rendered field:

Comment #15

akalata CreditAttribution: akalata as a volunteer commented 18 June 2015 at 20:05

Title:	Many Custom Format Date/Time separators appear AFTER the Time as WELL as before, even if placed BEFORE!	» Custom characters intended as separators within custom formatted date/time are not displayed in the correct location.
Priority:	Major	» Normal

Investigating the issue, the source appears to be when date_limit_format() tries to automatically split out the formatting into its parts (date, time, timezone). Since the @ is not part of the regex of symbols to deal with, it gets passed through -- the result of date_formatter_process() on a particular value is

$dates['value'] = array(
  'formatted' => 'Sep 24, 2013 @ 12:19 AM',
  'formatted_iso' => '2013-09-24T00:19:00-05:00',
  'formatted_date' => 'Sep 24, 2013 @',
  'formatted_time' => '@ 12:19 AM',
  'formatted_timezone' => '@',
);

It's interesting to note at the above point, $dates['value']['formatted'] is still giving us what we want. The real issue is with $dates['value']['formatted_timezone'] thinking it has a valid value -- which theme_date_display_combination() picks up and moves to the end of the string (beginning around line 137 of date.theme in current dev).

So, while this issue should probably be FIXED in date_limit_format(), we can use the following WORKAROUND by adding a custom preprocess on theme_date_display_combination(), removing the incorrect formatted_timezone (assuming that you do not need/want it):

/**
 * Workaround: A custom preprocess until https://www.drupal.org/node/1538420
 * is fixed
 */
function mythemename_preprocess_date_display_combination(&$variables) {
  $variables['dates']['value']['formatted_timezone'] = '';
  $variables['dates']['value2']['formatted_timezone'] = '';
}

Comment #16

simonyeldon CreditAttribution: simonyeldon commented 4 February 2016 at 21:10

We are having the same issue with a few of our sites, that are all multi-lingual, but is presenting itself in another way.

We have events, that when the start and end date are on the same day, but different times, when the field is rendered it was only presenting the start time, and not the end time.

Lots of investigation showed the issue is indeed in date_limit_format(), with it being unable to isolate the time correctly due to there being extra characters in the date format.

Example format, from our Japanese website:

$formats[] = array(
    'type' => 'long',
    'format' => 'Y\年 m\月 d\日 (l) H:i',
    'locales' => array('jp'),
  );

When date_limit_format() isolated the time, you were left with the following format: \年\月\日 H:i. It was unable to remove the Japanese characters from the string.

When the theme kicks in, the system does a str_replace to replace the start and end time over the top of the time in the formatted value of the $dates array. As the erroneous characters are in there, it never gets replaced, and hence the end date is not there. (line 208 of date.theme.inc)

We have been able to workaround it using a hook as we are fortunate enough to have a time format that we can manually inject into the $dates array, but this is an issue.

Comment #17

ultrabob CreditAttribution: ultrabob commented 5 August 2016 at 11:03

I've just run into this:

Format: Y\年m\月
Preview in Date and Time area: 2016年08月
Actual output: 2016年08月　年月

Has anyone found a workaround? @simonyeldon could you elaborate on what you're doing?

Comment #18

akalata CreditAttribution: akalata as a volunteer commented 11 August 2016 at 19:34

@ultrabob did you see the workaround I posted in #15?

Comment #19

ultrabob CreditAttribution: ultrabob commented 30 August 2016 at 04:28

@akalata Thanks, I did, but I actually did manage to get it working without code, I needed to add the escape characters and then go back in and reassign the format to the type. Just changing the format didn't solve the problem.

Comment #20

aitala CreditAttribution: aitala commented 1 May 2017 at 20:23

Adding on to the escaped characters... I am trying to display years like '18, '20, etc.

Even when I escape the single quote, i.e. \'y , for the date format, the quote does not render.

Any ideas?

Thanks,
Eric

Comment #21

Joel MMCC CreditAttribution: Joel MMCC commented 2 May 2017 at 16:21

@aitala #10, single quotes are problematic in SQL queries, since they can be used to malicious ends (SQL injection attacks). Most modules will strip them out unless escaped in a safe way, and some go overboard and simply strip them all out entirely just to be safe.

Besides, a straight quote is kind of unprofessional-looking. You want this to be an apostrophe, right? Try using the Right/Closing Single Quote, “’” (as opposed to “'”). That shouldn’t get stripped out. and it would actually look better and more professional.

Here it is larger so you can compare:
"Vote in '18!" vs.
“Vote in ’18!”

See how much more professional the second one looks? Note the “double quotes” as well.

To access the right single quote aka true “curly” typographic apostrophe:

On Windows, hold [Alt] and, on the numeric keypad, press and release in sequence while still holding down [Alt]: [0 Ins], [1 End], [4 ←], and [6 →], then release the [Alt] key.
On macOS, hold [Option] then press and release the right/closing square bracket (]}) key.
On Android with Gboard and some other on-screen keyboards, go to the symbols keyboard and hold the straight single quote key, and alternates should appear in a pop-up. Select the right/closing single quote aka apostrophe.
Not sure how to do it on iOS.
You may also be able to use the HTML Named Entity (“’” for right single quote) or HTML Numeric Entity (“’”) directly in the pattern, though you may need to escape most of the characters.

(The above should be an unordered [“bulleted”] instead of an ordered [numbered] list, but I had to use an ordered instead of unordered list there because this Drupal Issues forum’s CSS negates the effect of the unordered list (a “list-style: none;” appears in the aggregated CSS for the “ul” [and “ol” as well, but it gets overridden by “list-style: decimal outside;”] tag selector, despite there being a button for it in the WYSIWYG toolbar.)