Okay, I've no idea what's going on here. This module has worked for me before but for some reason it's stopped. I've reinstalled Drupal from scratch and only enabled the required modules, etc.

When I import a directory image_import shows the names of all the images as complete but the images do not get copied properly, or scaled. Importing a single image with image.module works perfectly.

e.g.:
files files_12 files_142 files_37 files_6 files_82

cat files shows the basic directory structure of the files directory. It looks like somewhere along the way the files *directory* is getting read and written to a file called files (hence the duplicates).

The log shows lots of:

page not found 2005-05-29 04:43 files/s/files_139 not found.
for the images as they are "imported".

No idea what's going on at all.

CommentFileSizeAuthor
#6 files_21 KBlewiz

Comments

cedpetite’s picture

cedpetite commented

Hi syscrusher,

I have the same probs :

getimagesize(): Read error! à la ligne 88 du fichier /var/www/free.fr/e/b/cedpetite/includes/image.inc.

Thanks

cedpetite’s picture

cedpetite commented
Priority: Normal » Critical

The log show :

lot of "getimagesize(): Read error! à la ligne 88 du fichier /var/www/free.fr/e/b/cedpetite/includes/image.inc."
lot of "system/files not found"

and i have a lot of files called files_XX in files/images.

My images was correctly deleted from my import folder.

lewiz’s picture

lewiz commented

Hmm, I don't think I got any warnings about being unable to resize images. I'll have another go and see if I can tell what is going on. It's really annoying me now -- image_import will be so great when I can get it working :)

syscrusher’s picture

syscrusher commented
Assigned: Unassigned » syscrusher

Good evening

Sorry you're having problems. I just got back from an out of town trip, and have only now seen this bug report.

I haven't been able to duplicate this yet, but will keep trying.

As to the comment about reliability, please do note that this module is still considered a BETA and isn't yet released to production. I know people are clamoring for it, and I'm working on it quite actively, but I can't promise bug-free code quite yet. :-)

Please do post any further information about your configuration here, in particular the settings of your options for this module and for image.module. A helpful thing to send me, if you are able, is the result of the following SQL query:

SELECT * FROM variable WHERE name LIKE 'image_import%';

(Prefix "variable" if needed for your installation.)

The other very helpful bit of info is this:

Upload some images as if you were going to import them. Just *prior* to running my module, go to the directory *one level up* from the import directory, and run:

ls -lR import_directory

(replacing import_directory with the actual name of yours, of course). After doing this, run the import from image_import.module, then run the same command again. Comparing the before and after snapshots has proven quite helpful in isolating problems.

Most of the problems people have had so far have turned out to be system config issues, particularly with permissions. I can't "fix" those in the code, because they're external, but I'm gradually adding more and more logic in the code to detect the problems and report them to the user so they can be fixed manually.

This one may actually be a bug in the code, but it's not happening here, so I need to try to replicate your settings closely so I can see the bug in action. As usual, fixing the bug will be 5% of the effort and finding it will be the other 95%. :-)

Scott

syscrusher’s picture

syscrusher commented

One other thing to check, please:

Try to do a "create content....image" to add an image using that module natively. I'm not disputing that the bug is in my module -- it probably is -- but let's eliminate other possible causes first, since this is something easy to test. :-)

Thanks!

Scott

lewiz’s picture

lewiz commented
StatusFileSize
new files_21 KB

Hi,

Thanks for getting back to me. Regular image creation with image.module works perfectly, so do file uploads.

I have tried setting up a whole new installation of Drupal 4.6 with just image.module, upload.module and image_import.module enabled. The problem still occurred (which ruled out any weirdness with any modules in my other installation). Variable settings are:

image_import_default_import_path = s:15:"image_import/%U";
image_import_default_upload_path = s:0:"";
image_import_force_delete = s:1:"1";
image_import_system_nou = s:1:"0";

I am using suPHP with Apache which means all PHP code runs as my user, not the webserver, hence my permissions are a little more restrictive than most. This *should not* affect anything:

% ls -lR image_import
total 2
drwx------  3 lewiz  users  512 May 29 04:45 lewiz/

image_import/lewiz:
total 2
drwx------  2 lewiz  users  1536 May 30 12:39 yamaha_ns-1000m/

image_import/lewiz/yamaha_ns-1000m:
total 886
-rw-------  1 lewiz  users  33347 Jun 15  2004 p6140148.jpg
-rw-------  1 lewiz  users  43001 Jun 15  2004 p6140149.jpg
-rw-------  1 lewiz  users  35019 Apr  8 00:52 p6140150.jpg
-rw-------  1 lewiz  users  39121 Apr  8 00:52 p6140151.jpg

run image_import.module. After:

%ls -lR image_import
total 2
drwx------  3 lewiz  users  512 May 29 04:45 lewiz/

image_import/lewiz:
total 2
drwx------  2 lewiz  users  1536 May 30 12:42 yamaha_ns-1000m/

image_import/lewiz/yamaha_ns-1000m:
total 0

Sorry, I forgot to ls -lR files/images before that but I'll just say the files weren't there. I now have a bunch of files_N with permissions 0644. I've attached a single file so you can take a peak... it is the contents of the files directory.

I tried to track it down before posting this in the first place but didn't get very far. You're right -- people are clamouring for this because it is a great piece of code that provides a very much required function! Good luck!

syscrusher’s picture

syscrusher commented

Okay, I'm still looking into this, but I see one bad problem right away. The module doesn't allow import from a subdirectory underneath your user import directory (in other words, the "yamaha..." directory in your example run is the problem level).

I admit this isn't documented or checked in the module. To be honest, it never occurred to me that anyone would do it. I'll fix that in the module and its documentation, and will also try duplicating the situation in my test environment to see what happens.

I theoretically could do recursive import, but I'm really skittish about the notion of doing what amounts to an "rm -rf" on a directory after the import.

More work at my end, but I wanted to let you know what I've found so far.

Thanks again for helping me to pin down this bug. I'm sorry for the inconvenience to you, and I really appreciate the test reports.

Scott

syscrusher’s picture

syscrusher commented

{sigh} I'm still unable to duplicate this particular bug, but I'm seeing some other really weird behavior from the core functions that are being used to import the files. Worst of all, these functions aren't giving me meaningful diagnostics -- for instance, they report a "read error" but not the path that they were trying to read. I need to insert a bunch of trace stuff into core to be able to find this problem.

The subdirectory import *does* work if you have permission to override the import path in image_import.module *and* you put that full path into the import dialog. In other words, as long as you don't expect image_import.module to do the recursion internally, this should work, and it does work in my test environment. Which is not to deny your bug -- I just am not currently able to duplicate it, but I haven't given up yet.

All things considered, I am seriously considering a total rewrite of my import loop, and in fact I'm also considering not using the Drupal core functions to move the files into the file import. I hate doing it that way, but the core functions are heavily oriented toward importing files that are part of a POST request and not pre-existing on the server, and there are a whole lot of extra tests and operations that are extraneous to what we're doing here.

I'm sorry this is taking so long to find. Please be patient, and remember that this is a pre-release version. I am working on it, and will try to get this worked out ASAP.

Scott

lewiz’s picture

lewiz commented

Hi,

I tried importing without the subdirectory but that made no difference. I've been doing all of this stuff with the main admin user anyway, so I can override the path.

Also, I just wanted to stay stop apologising! I fully appreciate you taking the time to look into this (and the time to write it in the first place). Very much appreciated.

adrinux’s picture

adrinux commented

I now have a bunch of files_N with permissions 0644.

Not sure how relevant this is lewiz but I was unable to import with 0644 and had to 0666 the image files.

syscrusher’s picture

syscrusher commented

Okay, everyone -- the main import functionality has been totally rewritten. The loop logic is much different, and there are diagnostics embedded all over the place.

Currently, the DEBUG MODE is forced to be ON. You'll see a lot of extra messages. Later, I'll make that an admin-configurable setting, but right now I need the added diagnostics to troubleshoot whatever problems you report. :-)

Please give this thing a test run on a non-production site, and let me know how things work for you.

Some notes:

  1. I've finished the rewrite and some basic confidence tests at about 0200 local time -- so I'm fried mentally and physically, and I'm not promising this new version is clean code yet. I cannot over-emphasize that this is not production code yet.
  2. The old version may have left some cruft in your files directory and your database. I'm sorry about that; in my own defense, all I can say is that I did specify this as an ALPHA version. Ideally, to test this version you should start with a fresh Drupal installation. If you see errors like failures of getimagesize(), and you didn't start over with a new Drupal, you probably are suffering from cruft from the old version. Deleting the offending nodes will probably clear up the problem.
  3. The testing of the new version has been done entirely with Drupal 4.6.1, patched for Morbus Iff's free tag taxonomy. That patch is not required, however, and indeed Morbus doesn't yet officially support 4.6.1.

I hope this version works better than the previous one did. Please post test results to this issue or to new issues for this project.

Scott

lewiz’s picture

lewiz commented

Well done that man! :)

Just done a very quick test with three images (the same three I was testing with the old version) and it has worked brilliantly.

I'll try importing a whole pile more stuff and let you know if I run into any trouble.

Thanks very much for your hard work!

syscrusher’s picture

syscrusher commented

lewiz, you just made my day! I was so crispy last night after I finished that coding session, I thought to myself, "Won't it be just my luck if I've actually made it worse instead of better?!"

I'm glad it is working for someone other than me. :-)

I still consider the rewrite a work-in-progress; there are definitely a few things I want to add for better error handling, and adrinux's throttle feature (actually, a variant thereof) will be added soon.

Please keep up the test reports, and thanks for getting back to me so soon.

Scott

lewiz’s picture

lewiz commented

Just imported about 500 images without any obvious problems.

I did spot two system/files errors in the log but they might be residual errors from the last time (once I tried it on a new installation I did it on my live site). I'll try and track down what caused those and get back to you if there are any problems.

Thanks again.

syscrusher’s picture

syscrusher commented

If the errors you see in the log have to do with getimagesize(), then they are probably residual. I saw a lot of those last night until I deleted the partially-created nodes from the error runs in the old version.

Scott

syscrusher’s picture

syscrusher commented

Changing status to "fixed". So far, the test reports indicate that the rewrite of the import function took care of this problem.

There is an apparent bug in image.module itself, that causes getimagesize() errors to be reported for empty albums. I'll see about getting a patch to "walkah" to address this.

Scott

lewiz’s picture

lewiz commented

Thanks very much -- I've imported even more galleries since then without any problems :)

syscrusher’s picture

syscrusher commented

Here's a followup on the getimagesize() error...

This is in fact a minor bug in image.module. There is an issue open on the bug, and user "robrechtj" has supplied a patch. I have tested the patch, and it works correctly for me.

Scott

Anonymous’s picture

(not verified) commented