Problem/Motivation
Using MigrateSourceCSV with a file with 3000 rows, csv.inc file is raising an error due to exceed the memory limit.
The problem could be fixed increasing the memory limit dynamically but I think the way to fix the problem is allow the user decide how get the number of rows using another way.
Reading the code I found a comment just in the line that was raising the error that said:
"// TODO. If this takes too much time/memory, use exec('wc -l')"
So I think we should take it into account.
Proposed resolution
I propose use exec ('wc -l') command as is described in the comment but I think we should also allow decide use the current way to get the total row count. I'm not sure about how it but maybe passing a new parameter in the constructor to allow the chose which one.
Remaining tasks
- Create patch
- Create tests
Comment | File | Size | Author |
---|---|---|---|
#4 | migrate-cvs_file-lines_count-2868377-04.patch | 588 bytes | voleger |
| |||
#2 | 2868377-2_number_rows.patch | 721 bytes | fran seva |
|
Comments
Comment #2
fran seva CreditAttribution: fran seva as a volunteer and at Bluespark commentedI have attached a first approach to fix the problem.
Comment #3
pifagorneed test community
Comment #4
volegerThis one should be more efficient.
SplFileObject presented in PHP 5.1 and it fit PHP requirements of Drupal 7 https://www.drupal.org/docs/7/system-requirements/drupal-7-php-requirements
Comparing count methods here - https://stackoverflow.com/questions/21447329/how-can-i-get-the-total-num...
Comment #5
pifagor+1
Comment #7
pifagor