Inside Info: Huge Files and Zip Uploads

Motivation

Per Step 6 of the cms.gov requirements, the hospital prices list must be contained in a single file. Depending on the number of services, the comprehensive pricing file can be huge. For instance, one site hosted on Acquia Cloud Next has an xml file that is over 500MB. This is problematic because Acquia Cloud Next (at the time of this writing) caps file uploads at 256MB. Luckily, the Hospital Price Transparency module supports some zip uploads, allowing an elegant workaround.

Details on Zip Support

There is no indication in the cms.gov guidelines that suggest serving a zip file is allowed or advisable. Therefore, the module does not support serving a zip file to end users. Instead, it puts specific requirements (i.e. validation) on any uploaded zip file. Then it extracts the archived file and serves it.

The specific requirements for uploading a zip file are:

1. The Hospital Price Transparency module must be configured to allow zip uploads on the file field. (See Inside Info: Module Configuration)
2. The uploaded zip file must contain exactly one archived file. Note that some operating systems (e.g. OSX) may add hidden files to the zip. We attempt to ignore these per #3380046: Allow hospital price transparency to ignore hidden files in zip . Still, it is advised that files are archived using the command line or other archiver tool to ensure no extra files are added.
3. The archived file must have one of the other allowed file types (likely xml, csv, or json).

Upon saving an HPT entity with a zip upload, the archived file will be extracted and saved as a new File entity. Then the HPT will be updated such that it references the extracted file rather than the zip file. At this point the zip file is left as a temporary file and could be cleaned up on a cron run. The (potentially huge) extracted file will be served to end users.