Problem/Motivation

when I have a test fail, it's often hard to understand why if the $this->assertIdentical() has a custom message.

Proposed resolution

On failure, always output the 2 values that are being compared.

Remaining tasks

User interface changes

n/a

API changes

n/a

Data model changes

n/a

Beta evaluation:
Unfrozen changes - changes tests only

Comments

pwolanin created an issue. See original summary.

aneek’s picture

@pwolanin, interesting thought. But both the methods return messages on both pass or fail. As an example if I test with TestBase::assertEqual('foo', 'bar') and should it not give me Value 'foo' is equal to value 'bar'? From that message we can surely identify that the test failed. But yes, we can have something like $this->verbose() if the assertion fails.

What are your thoughts on this?

Thanks.

pwolanin’s picture

Status: Active » Needs review
StatusFileSize
new1.01 KB

Like this.

stefan.r’s picture

So this overwrites the existing message which on failure seems fine. Should we concatenate both?

I don't see any reason not to do this, let's do it for assertEqual as well?

aneek’s picture

Got you @pwolanin, I'll also have a go..

aneek’s picture

StatusFileSize
new980 bytes
new1.76 KB

Uploading a new patch. Please review!

stefan.r’s picture

Maybe let's concatenate it with the original message, just in case people do a search for it in the results still expecting the old behavior.

pwolanin’s picture

StatusFileSize
new3.98 KB
new3.71 KB

Not sure what the right separator is - here I used "\n" and also extended the fix to assertNotEqual() and assertNotIdentical()

Status: Needs review » Needs work

The last submitted patch, 8: 2571291-8.patch, failed testing.

stefan.r’s picture

Can we have screenshots of what this looks like in CLI and web interface just to make sure the separator looks good?

pwolanin’s picture

looks ok - not sure if we have any better options. Here's how it looks hacking in a test fail.

example failing test in web UI

example failing test on CLI

pwolanin’s picture

Status: Needs work » Needs review
StatusFileSize
new3.98 KB

re-uploading patch since testbot was borked

stefan.r’s picture

Issue summary: View changes
Status: Needs review » Reviewed & tested by the community
Issue tags: +DX (Developer Experience)

Patch looks great, this seems super useful!

+++ b/core/modules/simpletest/src/TestBase.php
@@ -689,7 +694,17 @@ protected function assertEqual($first, $second, $message = '', $group = 'Other')
+    // Cast objects implementing SafeStringInterface to string instead of
+    // relying on PHP casting them to string depending on what they are being
+    // comparing with.
+    $first = $this->castSafeStrings($first);
+    $second = $this->castSafeStrings($second);

Seems we missed that one in #2560729: Cast objects implementing SafeStringInterface to string in TestBase / WebTestBase. I think we can include this in this patch because the added var_export may otherwise fatal on a test fail.

aneek’s picture

StatusFileSize
new2.55 KB
new3.99 KB

@pwolanin & @stefan.r, shouldn't we use PHP_EOL newline character in a cross-platform-compatible way, so it handles DOS/Mac/Unix issues?
Uploading a patch with PHP_EOL and marking for review.

Also a thought, the CLI output gives a new line break with the message & the default message. But the WebUI doesn't. Should it be good if for WebUI this new line break is converted to HTML break line?

What do you guys think about this?

aneek’s picture

Status: Reviewed & tested by the community » Needs review
pwolanin’s picture

I don't have a strong feeling either way. A few other places in core just use a \n like:

$failures[] = $requirement['title'] . ': ' . $requirement['value'] . "\n\n" . $requirement['description'];

I agree it would be nice to have a break in the UI, but we'd need to use a BR tag or make more substantial changes to the way tests are output, which seems out of scope here.

aneek’s picture

@pwolanin, I do agree with your thoughts. Using "\n" doesn't create any problems with Drupal in Unix or Mac OS, but might create issues in Windows because,

  • Unix / Linux / OS X uses LF (line feed, '\n', 0x0A)
  • Macs prior to OS X use CR (carriage return, '\r', 0x0D)
  • Windows / DOS uses CR+LF (carriage return followed by line feed, '\r\n', 0x0D0A).

Source:https://en.wikipedia.org/wiki/Newline

And also current core uses a lot of "\n" and lesser PHP_EOL. Yes, this is totally out of scope to convert newline to line break. A basic algorithm could be,

  1. Detect PHP_SAPI
  2. Set variable based on PHP_SAPI return: Either PHP_EOL (PHP_SAPI = true) or <br />(PHP_SAPI = false)

However, the questions are,

  1. Should we allow PHP_EOL in this fix? - Great if others provide their feedback as well.
  2. Should we create a new task that replaces "\n" where necessary with PHP_EOL or <br />?

Thanks!

pwolanin’s picture

Status: Needs review » Reviewed & tested by the community

@aneek - we don't currently change the messages based on cli or not, and there doesn't seem to be much support for that added complexity. I think you are over-thinking this small change that only affects tests. Possibly it's more correct to use PHP_EOL, so let's just go with that.

  • alexpott committed f5647da on 8.0.x
    Issue #2571291 by pwolanin, aneek, stefan.r: In WebTest code,...
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed f5647da and pushed to 8.0.x. Thanks!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.