We regularly use uuid_features to export Panel panes, and such. We often find ourselves including images in the export using the inline base64 file export mode. However, we've had to start bumping up our PHP memory limit, just to load up the Features page. This solution just won't scale to our needs.

At first, I tried the patch in #1229670: Support File entities and file, image and media fields, re-rolling it in the process. It doesn't appear to address the issue we're facing though. Nor does the "Remote file export, URL" file export mode.

In experimenting with the other file export modes, I can generate a feature with the file included as a separate file. That is, using the 'Local file export' option, I specified the path to the site's custom features module (e.g., sites/example.com/modules/custom/example_feature/assets). This very nicely creates an asset/ dir in the exported Feature's module directory, copies the file into it, and sets the uuid_features_file_path to point to it. But obviously this isn't very portable or useful, since we're hard-coding both the site path, and a single feature's name.

It seems like a fourth file export mode is required here; one that copies the file(s) in question into the exported Features module.

It's entirely possible that this functionality is already available, but I just haven't been able to figure out how to do it. If that's the case, I'll gladly re-categorize this issue as a 'support request'.

CommentFileSizeAuthor
#11 include_file_field-2411869-11.patch12.87 KBergonlogic
#10 include_file_field-2411869-10.patch10.44 KBergonlogic
#9 include_file_field-2411869-6.patch10.18 KBergonlogic
#5 include_file_field-2411869-5.patch10.22 KBergonlogic
#4 include_file_field-2411869-4.patch7.54 KBergonlogic

Comments

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

Reading through features_export_build_form_submit(), there don't appear to be any hooks that would allow us to inject our own files into the export. Worse still, we can't even name files as we'd like, since the file extensions are hard-coded.

Patching features to add a hook after all the module's files are generated seems like the best solution (see: #2412013: Add hook to allow injecting files into export) The only alternative that I can see is to replace this submit handler when we wish to inject our own files. Unfortunately, this submit handler is huge, and we'd have to basically copy it wholesale into our replacement handler.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

Discovered #1064340: Allow export of additional files/assets., which should make this much simpler.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

Also #2292813: Store the files in the feature instead of encoded data is the equivalent feature request against Default Content.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented
StatusFileSize
new include_file_field-2411869-4.patch7.54 KB

Here's a first pass patch that adds a 'packaged' file export mode. So far, it properly copies packaged files into exported Features. Next I'll work out the import mechanism. It supports multiple and multi-value file fields. Also, it requires the patch in https://www.drupal.org/node/1064340#comment-8918659

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented
Status: Active » Needs review
StatusFileSize
new include_file_field-2411869-5.patch10.22 KB

This patch makes it possible to import the packaged files. Still to do:

  1. check whether the packaged file already exists on the site
  2. make the feature not appear to constantly 'overridden'

(1) would be greatly simplified by moving some of the check at the top of _uuid_features_file_field_import_file() into helper functions, and re-using it for our purposes.

Still, this patch has satisfied the basics of this feature request, and is ready for some initial testing.

das-peter’s picture

das-peter commented

Didn't test it but a visual review looks pretty promising :)

MatthewHager’s picture

MatthewHager commented

I just tried this out and somehow I ended up with:

/**
 * Implements hook_assets/about-camden.jpg().
 */
function camden_about_camden_page_assets/about-camden.jpg() {

}

at the bottom of my uuid_node.inc file in my exported feature.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

I suspect that's due to the patch from #1064340: Allow export of additional files/assets..

Is it benign, or does it impede the intended operation of this feature?

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented
StatusFileSize
new include_file_field-2411869-6.patch10.18 KB

Re-roll due to a white-screening bug identified by @MatthewHager. Thanks!

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented
StatusFileSize
new include_file_field-2411869-10.patch10.44 KB

It looks like it wasn't setting the _files array for multiple nodes exported in a single feature. Try the attached patch.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented
StatusFileSize
new include_file_field-2411869-11.patch12.87 KB

field_collections weren't packaging files. Along the way, I cleaned up uuid_field_collection_features_export_render(), where we were calling features_var_export() directly on our FieldCollection instance, which, in turn called FieldCollectionEntity::export(). This was why we needed to strip stuff out, json_decode it, etc. Instead, we now just call get_object_vars(), and avoid all the extra overhead.

One issue I'm still seeing is with files containing spaces. They export just fine, but never get imported. Otherwise, this appears to work fairly nicely.

MatthewHager’s picture

MatthewHager commented

I just tested this and it works!

MatthewHager’s picture

MatthewHager commented
Status: Needs review » Reviewed & tested by the community
das-peter’s picture

das-peter commented
Status: Reviewed & tested by the community » Fixed

Awesome! Looks good to me. Committed & pushed.
Thanks!

  • das-peter committed 166fb9f on 7.x-1.x authored by ergonlogic 
    Issue #2411869 by ergonlogic: Include file_field files in exported...
ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented
Status: Fixed » Active

Sorry to re-open this fixed issue, but I wanted to stress the fact that this change is dependent on a patch that hasn't landed in Features itself yet: #1064340: Allow export of additional files/assets.. I haven't actually bothered to try running it against an un-patched Features module, but I can imagine that it could break pretty badly. We should probably make a point of getting that patch committed into Features or, at least, document it's requirement for this to work.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

Also, for this feature to be complete, it should really flag a feature as overridden if the packaged file changes. Testing the file's size should be sufficient for this, but I suppose we could do some sort of diff to be really sure.

jurriaanroelofs’s picture

jurriaanroelofs commented

Great work! but not working well (yet) for me.

Export: At first it failed silently, no files were exported. Then found out why in Watchdog log:
Notice: Undefined index: in features_get_storage() (line 658 of /Library/WebServer/Documents/cms/sites/all/modules/features/features.export.inc).
After setting the file export configuration to use ' Package files with generated module. ' it worked OK.

Import:


    Unknown error occurred attempting to import file: public://glazed-header-images/3539107466_4a57c4abde_o-300_0.jpg
    Unknown error occurred attempting to import file: public://blog-images/6ec033df-smush-3904401799_5bfe586e29_o.jpg
    Unknown error occurred attempting to import file: public://blog-images/6ec033df-smush-8260117875_5ab9373bce_o.jpg
    Unknown error occurred attempting to import file: public://glazed-header-images/1920x300blurrscape.jpg
    Unknown error occurred attempting to import file: public://blog-images/6ec033df-smush-2535937844_87a8cd3ebe_b.jpg

The other feature modules that still used base64 import worked ok.
It's not a permissions problem, I tried putting chmod -R 777 on the entire Drupal installation.

It seems that all changes that are made to $field = &$export->{$field_instance['field_name']}; do not persist into the export data in example.features.uuid_node.inc.

ergonlogic’s picture

ergonlogic
he/him
Primary language English
Location Montréal, Québec 🇨🇦
commented

@JurriaanRoelofs: please file a new bug-report, as the patch in this issue has already been committed. FWIW, I suspect that additional integration will be required, if using a field other than filefield.

jurriaanroelofs’s picture

jurriaanroelofs commented

Ok, new issue here: https://www.drupal.org/node/2428617

I tested with a regular file field and the code showed identical behavior.

kenorb’s picture

kenorb commented
Related issues: +#2488804: Undefined property: uri in _uuid_features_file_field_import_file (when running during Drupal installation)
andypost’s picture

andypost
he/him
Primary language Russian
commented
Related issues: +#2498905: File export mode "Package with generated module" creates invalid php function name