Problem/Motivation

Creating a Smart Date Range with a start and end date 16 years or more apart throws a WSOD and a SQL Exception:

Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'field_rights_window_duration' at row 1: INSERT INTO "node__field_rights_window" ("entity_id", "revision_id", "bundle", "delta", "langcode", "field_rights_window_value", "field_rights_window_end_value", "field_rights_window_duration", "field_rights_window_rrule", "field_rights_window_rrule_index", "field_rights_window_timezone") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10); Array ( [:db_insert_placeholder_0] => 6423 [:db_insert_placeholder_1] => 759511 [:db_insert_placeholder_2] => episode [:db_insert_placeholder_3] => 0 [:db_insert_placeholder_4] => und [:db_insert_placeholder_5] => 1275375600 [:db_insert_placeholder_6] => 1851404340 [:db_insert_placeholder_7] => 9600479 [:db_insert_placeholder_8] => [:db_insert_placeholder_9] => [:db_insert_placeholder_10] => ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->saveToDedicatedTables() (line 1393 of /app/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

Steps to reproduce

Create a Smart date range field on a content type
Create content
Put dates 16 years or more apart (e.g. 2010-06-01 to 2026-06-01).
Save node.

Proposed resolution

The duration field in node__field_rights_window should be extended from a mediumint to an int.

A signed MEDIUMINT has a max value of 16777215, and I'm guessing that that's recorded as minutes. (16777215/2) minutes is 15.9 years.

#3304395: Far-future dates cause WSOD, "Numeric value out of range" appears to have had a similar issue, similarly resolved by extending the field sizes.

Note I originally found this issue on Smart Date 3.7.2, but upgrading to 4.0.3 or 4.1.0-rc2 does not resolve the issue.

Issue fork smart_date-3390714

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

TrevorBradley created an issue. See original summary.

mandclu’s picture

When you upgraded to 4.1.0-rc2, did you run database updates?

trevorbradley’s picture

@mandclu. Yes. That updated the date fields.

This is explicitly about the duration field, which I believe maxes out at signed mediumint.

Here's the DB table post 4.1.0-rc2 update:

mysql> describe node__field_rights_window;
+---------------------------------+--------------+------+-----+---------+-------+
| Field                           | Type         | Null | Key | Default | Extra |
+---------------------------------+--------------+------+-----+---------+-------+
| bundle                          | varchar(128) | NO   | MUL |         |       |
| deleted                         | tinyint      | NO   | PRI | 0       |       |
| entity_id                       | int unsigned | NO   | PRI | NULL    |       |
| revision_id                     | int unsigned | NO   | MUL | NULL    |       |
| langcode                        | varchar(32)  | NO   | PRI |         |       |
| delta                           | int unsigned | NO   | PRI | NULL    |       |
| field_rights_window_value       | bigint       | YES  | MUL | NULL    |       |
| field_rights_window_end_value   | bigint       | YES  | MUL | NULL    |       |
| field_rights_window_duration    | mediumint    | YES  |     | NULL    |       |
| field_rights_window_rrule       | int          | YES  | MUL | NULL    |       |
| field_rights_window_rrule_index | int          | YES  | MUL | NULL    |       |
| field_rights_window_timezone    | varchar(32)  | YES  |     | NULL    |       |
+---------------------------------+--------------+------+-----+---------+-------+
12 rows in set (0.00 sec)

Note field_rights_window_duration is still a mediumint.

mandclu’s picture

Hmmm I did miss that this was about the duration. I suppose there isn't any value having it signed, since the validation will check that the start is before the date. Why do you need to store events with such large durations?

trevorbradley’s picture

> Why do you need to store events with such large durations?

In this case, it's genuinely an 18 year date range, just negotiated up from a 14 year date range. The client wants to store the date range for which rights have been granted for a particular media object, and wants to store that date range authoritatively (not fake it and say the rights started today).

I dunno, copyright licensing is weird.

Regardless of the duration, the module should throw an error on validation rather than WSOD on an invalid SQL insertion, regardless of the data being entered into the UI.

mandclu’s picture

I'm tempted to say that for edge cases like this it makes sense to just use separate fields, but in truth this has come up before, and the storage difference is not huge between mediumint and integer, so this is worth consideration. Also, Smart Date uses integers for storing numbers that if anything are likely to be smaller.

From my quick calculations, moving to integer would allow for storage of ~68 years, not quite enough for an average human lifetime, which seems like it should be a goal to accommodate if making this change. Making the column unsigned would double this, making the duration field able to accommodate even exceptionally long lives. I'm not sure if changing from a signed to an unsigned column creates higher risk for existing data, but we can try it.

trevorbradley’s picture

Something's not quite right about that math. I swear the duration data is in minutes, not seconds. Signed mediumint is 2^23 = 8.38M minutes = 15.9 years, and the system fails on 16 years but works on 14 years.

So a signed integer (2^31 minutes) would be 4085 years.

(If duration isn't minutes but seconds, something else odd is going on)

Honestly, I think it's more important to put in validation on the date range field to prevent a WSOD.

mandclu’s picture

You're correct. Not sure what NyQuil-induced math I was working, but you are correct that even with a signed integer, 4085 years is the maximum duration, which should be sufficient for anything but the rarest of use cases.

That said, I also agree that the validation is important either way, and probably even more so than the field length change. Would it make sense to create a patch in this issue for the validation, and then open another issue for extending the field length? Or vice versa, I don't mind.

trevorbradley’s picture

I see WSOD, I open a ticket. No judgement on how fast it gets fixed.

Either 1 ticket or two works for me. I see the justification for both, but if they're two separate commits, they'll definitely need to be sequenced.

trevorbradley’s picture

Just some context - after a bit more investigation my client was trying to enter in a digital rights window of 99 years, apparently quite common in legal contexts.

https://en.wikipedia.org/wiki/99-year_lease

In history, 99 years was initially a placeholder for "forever" but apparently as the century passed, the law got kind of prickly about being literally 99 years and now it's commonplace.

This is in a field where rights windows are quite frequently just a few weeks, but they want to be able to use the field the same way for both instances.

Aditi Saraf made their first commit to this issue’s fork.

mandclu’s picture

FYI I have created #3406232: Add validation checks for start, end, and duration to handle the validation checks as a separate issue.

mandclu’s picture

There's now an MR up in #3406232: Add validation checks for start, end, and duration, but based on the expanded values that will be implemented in this issue and #3386133: node.field_[name] field needs to be updated. I suppose theoretically we could merge in the constraints now with the smaller values, and then make updating the constraints part of the work in each issue meant to expand column size.

It's worth noting that this issue and #3386133: node.field_[name] field needs to be updated are blocked by a requirement for a new setting for the SqlContentEntityStorageSchema class proposed by #2885413: Timestamp field items are affected by 2038 bug, which now will require a larger set of work to get merged in. As such, I have created #3406172: Provide a flag to allow updates to stored schema for fields as a smaller issue to get the new setting adopted, and hopefully merged in sooner.

mandclu’s picture

Version: 4.0.3 » 4.1.x-dev
Status: Active » Needs review

There is now an MR with code to expand the duration column to a normal integer.

  • mandclu committed 21d0960c on 4.1.x
    Issue #3390714 by mandclu: Smart Date Range: Duration of >=16 years...
mandclu’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.