I installed Drupal 7.14 on a Debian testing box (wheezy by this date), enabled Simpletest module, installed the examples module and when I run the simpletest_example test I got a fail on the testing batch AJAX call. Latter I found out that the fail is produced because the log-in page is printed on the response, after the expected JSON string.

Then I went to the test results and it said that the log-in form could not be read.

After debugging I found that the call to curl_setopt_array on DrupalWebTestCase::curlInitialize() is failing because of the option CURLOPT_COOKIEJAR set to NULL. curl_setopt_array stops processing options when an option fails and since CURLOPT_RETURNTRANSFER is after CURLOPT_COOKIEJAR in the array it was not set and curl_exec is echoing the downloaded page.

PHP's curl_setopt manual page (http://co.php.net/manual/en/function.curl-setopt.php) claims that by setting CURLOPT_COOKIEFILE to an empty string "no cookies are loaded, but cookie handling is still enabled". So I tried by setting CURLOPT_COOKIEFILE to '' but it still fails, also tried by setting both CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR to '' and then to NULL but it always fails no matter how I mix it. This is not what the manual says so I filed a bug report (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=680260).

I also tried removing CURLOPT_COOKIEJAR and CURLOPT_COOKIEFILE from the options array but then cookies were disabled and userLogin() failed.

Finally I set $this->cookieFile to a temporary file and now everything is working fine. This is how the method looks now:

protected function curlInitialize() {
    global $base_url;

    if (!isset($this->curlHandle)) {
      $this->curlHandle = curl_init();
      $this->cookieFile = tempnam(NULL, "drupal-simpletest-cookiejar-");
      $curl_options = array(
        CURLOPT_COOKIEJAR => $this->cookieFile,
        CURLOPT_URL => $base_url,
        CURLOPT_FOLLOWLOCATION => FALSE,
        CURLOPT_RETURNTRANSFER => TRUE,
        CURLOPT_SSL_VERIFYPEER => FALSE, // Required to make the tests run on https.
        CURLOPT_SSL_VERIFYHOST => FALSE, // Required to make the tests run on https.
        CURLOPT_HEADERFUNCTION => array(&$this, 'curlHeaderCallback'),
        CURLOPT_USERAGENT => $this->databasePrefix,
      );
      if (isset($this->httpauth_credentials)) {
        $curl_options[CURLOPT_HTTPAUTH] = $this->httpauth_method;
        $curl_options[CURLOPT_USERPWD] = $this->httpauth_credentials;
      }
      curl_setopt_array($this->curlHandle, $this->additionalCurlOptions + $curl_options);

      // By default, the child session name should be the same as the parent.
      $this->session_name = session_name();
    }
    // We set the user agent header on each request so as to use the current
    // time and a new uniqid.
    if (preg_match('/simpletest\d+/', $this->databasePrefix, $matches)) {
      curl_setopt($this->curlHandle, CURLOPT_USERAGENT, drupal_generate_test_ua($matches[0]));
    }
}
Files: 
CommentFileSizeAuthor
#33 drupal.simpletest-curl-cookiejar.33.patch1.49 KBsun
PASSED: [[SimpleTest]]: [MySQL] 39,299 pass(es). View
#28 1671200_28.patch1.52 KBchx
FAILED: [[SimpleTest]]: [MySQL] 37,019 pass(es), 28 fail(s), and 3 exception(s). View
#18 core-simpletest-null-cookiejar-1671200_14.patch1.56 KBchx
PASSED: [[SimpleTest]]: [MySQL] 37,289 pass(es). View
#14 core-simpletest-null-cookiejar-1671200_14.patch1.56 KBjaimealsilva
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed during invocation of run-tests.sh. View
#10 1671200_10.patch1.56 KBchx
PASSED: [[SimpleTest]]: [MySQL] 37,288 pass(es). View
#9 1671200_9.patch1.78 KBchx
PASSED: [[SimpleTest]]: [MySQL] 37,291 pass(es). View
#4 drupal.simpletest-curl-cookiejar.D7-do-not-test.patch2.19 KBsun
#3 drupal8.simpletest-curl-cookiejar.3.patch1.91 KBsun
FAILED: [[SimpleTest]]: [MySQL] 37,283 pass(es), 3 fail(s), and 0 exception(s). View
drupal-curlopt_cookiejar-debian-do-not-test.patch572 bytesjaimealsilva
snapshot50.png45.33 KBjaimealsilva
snapshot49.png148.5 KBjaimealsilva

Comments

jaimealsilva’s picture

Drupal status, in case is needed as reference:

Info
Drupal 7.14
Info
Install profile Minimal (minimal-7.14)
OK
Access to update.php Protected
OK
Configuration file Protected
OK
Cron maintenance tasks Last run 2 hours 26 min ago
You can run cron manually.
To run cron from outside the site, go to ...
OK
Database system MySQL, MariaDB, or equivalent
OK
Database system version 5.5.24-4
OK
Database updates Up to date
OK
Drupal core update status Up to date
OK
File system Writable (public download method)
OK
Module and theme update status Up to date
OK
Node Access Permissions Disabled
If the site is experiencing problems with permissions to content, you may have to rebuild the permissions cache. Rebuilding will remove all privileges to content and replace them with permissions based on the current modules and settings. Rebuilding may take some time if there is a lot of content or complex permission settings. After rebuilding has completed, content will automatically use the new permissions. Rebuild permissions
OK
PHP 5.4.4-2 (more information)
OK
PHP DOMDocument class Enabled
OK
PHP extensions Enabled
OK
PHP memory limit 512M
OK
PHP open_basedir restriction Disabled
OK
PHP register globals Disabled
Warning
Translation update status No update data available
No information is available about potential new and updated translations for currently installed modules and themes. To check for updates, you may need to run cron or you can check manually. Please note that checking for available updates can take a long time, so please be patient.
OK
Unicode library PHP Mbstring Extension
OK
Update notifications Enabled
OK
Web server Apache/2.2.22 (Debian)
OK
cURL Enabled
OK
hash Enabled
sun’s picture

Version: 7.14 » 7.x-dev
sun’s picture

Title: CURLOPT_COOKIEJAR can not be NULL on Debian's php5-curl version 5.4.4-2 » CURLOPT_COOKIEJAR cannot be NULL on Debian's php5-curl version 5.4.4-2
Version: 7.x-dev » 8.x-dev
Status: Active » Needs review
Issue tags: +needs backport to D7
FileSize
1.91 KB
FAILED: [[SimpleTest]]: [MySQL] 37,283 pass(es), 3 fail(s), and 0 exception(s). View

The provided patch in #0 has been confirmed to work, also on Fedora.

Attached patch performs this change for D8.

sun’s picture

Temporary backport for D7. (proper one is blocked on #1541958: Split setUp() into specific sub-methods)

Status: Needs review » Needs work

The last submitted patch, drupal8.simpletest-curl-cookiejar.3.patch, failed testing.

pfrenssen’s picture

This also affects the contrib Simpletest module in D7.

pfrenssen’s picture

Title: CURLOPT_COOKIEJAR cannot be NULL on Debian's php5-curl version 5.4.4-2 » CURLOPT_COOKIEJAR cannot be NULL on php5-curl version 5.4.4

I rerolled this patch for contrib Simpletest in this issue: #1692836: CURLOPT_COOKIEJAR cannot be NULL on PHP version 5.4.4.

Also changing the title, since this does not only affect Debian.

Anonymous’s picture

I can confirm #4 solves my problem on my tests site.

chx’s picture

Title: CURLOPT_COOKIEJAR cannot be NULL on php5-curl version 5.4.4 » Simpletest broken on 5.4: CURLOPT_COOKIEJAR cannot be NULL on php5-curl version 5.4.4
Priority: Normal » Critical
Status: Needs work » Reviewed & tested by the community
FileSize
1.78 KB
PASSED: [[SimpleTest]]: [MySQL] 37,291 pass(es). View

It's enough to set the jar if it's not already set. The session tests set it to $this->cookieFile = file_stream_wrapper_get_instance_by_scheme('temporary')->getDirectoryPath() . '/cookie.' . $uid . '.txt';. THANKS SO MUCH! I was completely taken aback by curl reporting a 1...

chx’s picture

FileSize
1.56 KB
PASSED: [[SimpleTest]]: [MySQL] 37,288 pass(es). View

Moving the COOKIEJAR line in the array was unnecessary (this does not change #9. at all. makes the patch shorter, that's all).

chx’s picture

Also, someone needs to compile a php.net bug report. This is definitely not Debian, I am running Arch, PHP is 5.4.4 and cURL is 7.26.0

pfrenssen’s picture

This issue is probably caused by this bug which was fixed in PHP 5.4.4: CURLOPT_COOKIEFILE '' raises open_basedir restriction.

chx’s picture

We have a problem with the COOKIEJAR and not the COOKIEFILE and also everyone here reports this being still a problem in 5.4.4. But it's very likely the two are related, yes.

jaimealsilva’s picture

FileSize
1.56 KB
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed during invocation of run-tests.sh. View

I think is best to use "empty" instead of "!isset" since it also checks for zero length strings and FALSE which are values that make curl_setopt fail.

Status: Reviewed & tested by the community » Needs work
Issue tags: -needs backport to D7

The last submitted patch, core-simpletest-null-cookiejar-1671200_14.patch, failed testing.

jaimealsilva’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work
Issue tags: +needs backport to D7

The last submitted patch, core-simpletest-null-cookiejar-1671200_14.patch, failed testing.

chx’s picture

FileSize
1.56 KB
PASSED: [[SimpleTest]]: [MySQL] 37,289 pass(es). View

emtpy => empty.

chx’s picture

Status: Needs work » Needs review
jaimealsilva’s picture

Ops! that's embarrassing. Sorry. Thanks chx.

sun’s picture

Status: Needs review » Reviewed & tested by the community
Dries’s picture

Version: 8.x-dev » 7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed to 8.x. Thanks! Moving to 7.x.

Sweetchuck’s picture

Version: 7.x-dev » 8.x-dev
Status: Patch (to be ported) » Reviewed & tested by the community

I ran into this issue too.

openSUSE 12.1
PHP 5.3.14
PHP cURL Information 7.22.0
PHP cURL SSL Version OpenSSL/1.0.0e
PHP libSSH Version libssh2/1.2.9
by phpinfo()

The patch #18 works well for me.

RTBC++

Tor Arne Thune’s picture

Version: 8.x-dev » 7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)
chx’s picture

If 5.3.14 is broken too then https://bugs.php.net/bug.php?id=61948 likely broke the COOKIEJAR cos that bugfix is in 5.3.14 too.

sun’s picture

Status: Patch (to be ported) » Postponed

The backport is blocked on #1541958: Split setUp() into specific sub-methods (as outlined in #4)

Anonymous’s picture

My PHP version 5.4.4 (in Fedora, package php-5.4.4-4.fc17.x86_64)

chx’s picture

Status: Postponed » Needs review
FileSize
1.52 KB
FAILED: [[SimpleTest]]: [MySQL] 37,019 pass(es), 28 fail(s), and 3 exception(s). View

How is that blocked?

Status: Needs review » Needs work

The last submitted patch, 1671200_28.patch, failed testing.

sun’s picture

Status: Needs work » Postponed
chx’s picture

Status: Postponed » Needs work

Thanks for the through review and the explanation why my patch didn't work. This just needs fixing because the split is not going to fix this: the only difference between the D7 and the D8 patch is using variable_get to get the public files directory instead of reading it from a class variable. I am throughly intrigued why is that solved by a property and why can we add new properties to D7 test base classes. If we can do that then without the split we can move the public directory into a class property and have the absolute eqiuivalent to D8 patch -- but I have my doubts that'd pass. I will look at this tomorrow.

sun’s picture

Priority: Critical » Normal
Issue tags: +PHP 5.4

Downgrading this to normal, since D8 has been fixed, and people using PHP 5.4 with D7 are most likely running into many other incompatibilities anyway, and this one is merely about Simpletest only.

As I've pointed out in #4, the backport depends on #1541958: Split setUp() into specific sub-methods, because we want to keep the testing framework as aligned/comparable between D7 and D8 as possible. That issue is RTBC and will be committed either before the new D7 release or shortly after. Once that patch is in, we can backport this patch as-is. So there's no reason for purposively letting the code diverge.

sun’s picture

Status: Needs work » Needs review
FileSize
1.49 KB
PASSED: [[SimpleTest]]: [MySQL] 39,299 pass(es). View

The dependency landed, so here's the clean backport to D7.

chx’s picture

Status: Needs review » Reviewed & tested by the community

Oh, awesome.

djdevin’s picture

This also applies to D6 running PHP 5.3.14

Patch for D7 worked on D6 with some warnings, but fixed the failures and content getting spit out.

Edit: sorry, wrong issue queue. Meant to post to the contrib thread #1692836: CURLOPT_COOKIEJAR cannot be NULL on PHP version 5.4.4

justageek’s picture

Hi, I don't see this:

$this->public_files_directory

ever being initialized or being a part of the class definition.

sun’s picture

Xano’s picture

@justageek: you need Drupal 7.15 for this.

I can confirm this patch works and it fixes the problem as it should. (apparently it doesn't, as this is a PHP bug).

//Edit: some clarification: the patch doesn't fix the bug (as it is in PHP), but it does provide a good workaround as mentioned in #40.

pfrenssen’s picture

PHP 5.4.6 and PHP 5.3.16 have been released which fix this bug.

CURL - Fixed bug #62499 (curl_setopt($ch, CURLOPT_COOKIEFILE, "") returns false).

references:

pfrenssen’s picture

@Xano, this issue works around the PHP bug, so that Drupal keeps working on systems that have the affected PHP versions installed.

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 7.x as well. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

phl3tch’s picture

FYI, I upgraded to PHP 5.3.17 and still had the issue. Patch #33 works for me though.

alberto56’s picture

Hi all,

I'm a bit confused:

- Comment #41 states that this was committed to 7.x on August 18, 2012.
- I am running 7.16, which was released long after comment #41 was posted
- Still, I had the problem of Curl returning "1" instead of an actual response
- I applied the patch in comment #33, which applied to my code

Hunk #1 succeeded at 1685 (offset 13 lines).
Hunk #2 succeeded at 1706 (offset 13 lines).

My understanding is that since this was committed to 7.x in august, trying to apply this patch to 7.16 should have resulted in a "detected previously applied patch, revert? (y/n)" message. No?

Thanks,

Albert.

David_Rothstein’s picture

@alberto56, Drupal 7.16 was a security release only, no bug fixes (see the release notes at http://drupal.org/node/1815904, and also see http://groups.drupal.org/node/260803).

This patch is in 7.x-dev so it will definitely be in the next bug fix release... (which will probably be Drupal 7.17 assuming no other security releases happen before then).

Chi’s picture

Title: Simpletest broken on 5.4: CURLOPT_COOKIEJAR cannot be NULL on php5-curl version 5.4.4 » Simpletest broken: CURLOPT_COOKIEJAR cannot be NULL

As mentioned above the bug appears on PHP 5.3 as well.

brazorf’s picture

Any chance about the D6 backport of this patch?

Xano’s picture

For a D6 backport, please open an issue for the Simpletest project.

kentr’s picture