Problem/Motivation

When using amazon s3fs as storage for your images the image exif data is not stored in your database when uploading your images.
This patch checks if the image is stored not in public path and then copies the image to a local temp file for the exif data.
When finished the temp file is deleted.

The problem is valid for all remote stream wrappers. If a file isn't available locally reading the metadata might fail.

Proposed resolution

Create a local copy of the file in case the file is handled by a non-local stream wrapper.
Remove the local copy once the processing of the file is finished.

I wonder if ExifContent::get_data_from_file_uri() shouldn't implement a metadata cache based on the provided URI.
The method is used in some loops which causes lots of reads and while reads most likely aren't as performance intensive as the remote download and unlink handling it seems to be excessive to read the same thing from the filesystem over and over again.

Remaining tasks

Reviews needed.

User interface changes

None.

API changes

None.

Data model changes

None.

Comments

Edwin.ewise created an issue. See original summary.

Edwin.ewise’s picture

StatusFileSize
new2.92 KB

When uploading a png image or when uploading an image (ex:jpg) which contains exif data to S3, in both cases the title field is not filled with exif metadata file_filename. This updated patch '02-fix-for-exif-images-on-s3fs-1.patch' solves this issue.

Edwin.ewise’s picture

StatusFileSize
new3.13 KB

When uploading a png image or when uploading an image (ex:jpg) which contains exif data to S3, in both cases the title field is not filled with exif metadata file_filename. This updated patch '02-fix-for-exif-images-on-s3fs-1.patch' solves this issue.

das-peter’s picture

Title: Fix for image exif data when stored on s3fs » Fix for image exif data when stored on s3fs / remote stream wrappers
Issue summary: View changes
StatusFileSize
new2.74 KB

@Edwin.ewise Thanks for the patch. I've patched the module but unfortunately the patch doesn't work when S3FS is used a the public stream wrapper.

I've created an overhauled patch:

  1. Added generic check to see if the used stream wrapper is local or not
  2. Create temporary file only once per ExifContent instance. ExifContent::get_data_from_file_uri() is used in loops would lead to lots of download / delete operations. Now it's downloaded once and cleaned up on instance destruction.

I wonder if ExifContent::get_data_from_file_uri() shouldn't implement a metadata cache based on the provided URI.

  • das-peter authored 9347682 on 8.x-1.x
    Issue #2981560: Fix for image exif data when stored on s3fs / remote...

  • das-peter authored 9347682 on 8.x-2.x
    Issue #2981560: Fix for image exif data when stored on s3fs / remote...
jphautin’s picture

Status: Needs review » Fixed

merged. thanks. very useful :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.