Feel free to close this issue if we feel it's a duplicate, but there seems to be little movement in these other related issues:

#1815886: Drupal core cannot manage files larger than 4 GB
#1651096: Adding file metadata to {amazons3_file} fails with filesize > 2GB, use BIGINT instead
#1003692: PDOException: SQLSTATE[22003] after entering large value in integer and decimal field

The third issue, however, appears to be geared towards number fields only with regard to the Field API; therefore I believe we'll need to either work on getting the Drupal core issue solved, or find a workaround.

TLDR: it does not seem to matter what value we enter into the "max filesize" field of a Commerce File product field; the {file_managed} table remains an INT(10) which fails on values inserted for files larger than 2GB. For our site, we have MPG and ISO files that are generally over this limit, reaching 4GB (or 8GB for a dual-layer DVD image). This means Drupal cannot serve as a file management system for video production websites like ours in its current form.

My proposal would be to hook the Commerce File (and Amazons3) fields into a function that makes sure the requisite tables' fields are the correct size. For instance, using Commerce File with AmazonS3 and my AmazonS3 (Filefield Sources) Reference integration, I would alter the module to look for an update to the "max filesize" attribute, and when executed, would alter the newly-introduced {amazons3_file} cache table for that module. Otherwise, attempting to migrate (or upload) files larger than 2GB results in this error:

SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column 'filesize' at row 1: INSERT INTO {file_managed} (uid, filename, uri, filemime, filesize, status, [error]
timestamp, type) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5,
:db_insert_placeholder_6, :db_insert_placeholder_7); 

Any ideas are welcome, as right now I am simply altering our schema, hoping this doesn't result in any future conflicts.

Comments

torgosPizza’s picture

Issue summary: View changes

Added the Core issue

Dustin@PI’s picture

@torgosPizza:

Thanks for the great write up ... I hit this problem an hour ago as the first of our videos we sell via Commerce Field exceeded 2GB.

A quick question: how reliable has it been for you to just alter the schema?

torgosPizza’s picture

Hi Dustin,

It's been reliable for our needs at this point. The only caveat is that you'd want to pay attention to core updates and what they do. I don't really think any update hooks for core would "reset" the schema of {file_managed} (and likewise for Commerce File), but I guess you never know - better to be safe than sorry.

Feel free to follow the other Issues, including the core one which just recently got some attention. It seems like an important issue for using Drupal to create a media site!

bojanz’s picture

Version: 7.x-2.x-dev » 7.x-1.x-dev

I've looked into this.
The limit of the filesize column in the database is 4GB (unsigned integer).
However, if you have a 32bit PHP installation, the filesize() function stops at 2GB (see docs).

So, the plan of action:
1) Fix D8 and D7 core schema to use a bigint for the filesize column (patch posted in #1815886: Drupal core cannot manage files larger than 4 GB).
2) Alter the schema of the column on commerce_file install, changing it to a bigint.
3) Add a warning for 32bit PHP installations.

bojanz’s picture

Version: 7.x-1.x-dev » 7.x-2.x-dev

Stuff happens in 7.x-2.x nowadays.

bojanz’s picture

Title: Commerce File doesn't update file table schemas for large file sizes (2GB+) » Ensure support for files larger than 2GB
Version: 7.x-1.x-dev » 7.x-2.x-dev
Status: Active » Fixed

Committed: http://drupalcode.org/project/commerce_file.git/commitdiff/728e143?hp=a2...

I took the 64-bit check from S3 (awssdk).

bojanz’s picture

Issue tags: +possible-backport

Tagging, could be ported to 7.x-1.x.

torgosPizza’s picture

@bojanz, you are my new hero. Thanks for taking the initiative and writing a patch for the Core issue. I'd love to see this get resolved there.

Dustin@PI’s picture

@bojanz and @torgosPizza, Thanks for the great work in the issue and the related ones. I've applied the patches etc. and I almost have it working ("just" have to upgrade to a 64-bit server now ... thank goodness for VMs).

Thanks Again!
Dustin.

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Wrong issue used