Problem/Motivation

Using MigrateSourceCSV with a file with 3000 rows, csv.inc file is raising an error due to exceed the memory limit.
The problem could be fixed increasing the memory limit dynamically but I think the way to fix the problem is allow the user decide how get the number of rows using another way.
Reading the code I found a comment just in the line that was raising the error that said:
"// TODO. If this takes too much time/memory, use exec('wc -l')"
So I think we should take it into account.

Proposed resolution

I propose use exec ('wc -l') command as is described in the comment but I think we should also allow decide use the current way to get the total row count. I'm not sure about how it but maybe passing a new parameter in the constructor to allow the chose which one.

Remaining tasks

  • Create patch
  • Create tests
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

fran seva created an issue. See original summary.

fran seva’s picture

Assigned: fran seva » Unassigned
Status: Active » Needs review
FileSize
721 bytes

I have attached a first approach to fix the problem.

pifagor’s picture

need test community

voleger’s picture

This one should be more efficient.
SplFileObject presented in PHP 5.1 and it fit PHP requirements of Drupal 7 https://www.drupal.org/docs/7/system-requirements/drupal-7-php-requirements

Comparing count methods here - https://stackoverflow.com/questions/21447329/how-can-i-get-the-total-num...

pifagor’s picture

Status: Needs review » Reviewed & tested by the community

+1

  • pifagor committed 4cd4acf on 7.x-2.x authored by voleger
    Issue #2868377 by fran seva, voleger, pifagor: MigrateSourceCSV count...
pifagor’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.