Problem/Motivation

More or less everywhere we take an array of $messages as a method argument, we pass it not only not only TranslatableMarkup[], in practice, but also string[] this seem to happen most in in tests. The other case where we do this is when add validation results from exceptions.

We don't add want developers to create validation results that are not translatable. The only time where this should happen is when validation results are being create from throwable. in that case they won't be translatable.

Proposed resolution

We should make it easy to make it obvious that all validation results should translatable exception when created from exceptions

  1. Add \Drupal\package_manager\Event\PreOperationStageEvent::addErrorFromThrowable
  2. Add \Drupal\package_manager\ValidationResult::createErrorFromThrowable
  3. Change `\Drupal\Tests\automatic_updates\Traits\ValidationTestTrait::createValidationResult` to use translatable strings
  4. Replaces all places where we send untranslatable to \Drupal\package_manager\ValidationResult::createError or \Drupal\package_manager\ValidationResult::createWarning, \Drupal\package_manager\Event\PreOperationStageEvent::addError or \Drupal\package_manager\Event\StatusCheckEvent::addWarning
  5. In places where we add validation results from throwable(exceptions mostly) use \Drupal\package_manager\Event\PreOperationStageEvent::addErrorFromThrowable

Remaining tasks

1,2,3 are done. 4 and 5 need to be done

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

TravisCarden created an issue. See original summary.

traviscarden’s picture

Assigned: traviscarden » Unassigned
Status: Active » Needs review

tedbow made their first commit to this issue’s fork.

tedbow’s picture

Status: Needs review » Needs work

in practice, but also string[]--especially in tests.

I think maybe it is only in tests. We should fix that rather than updating the comments for bad practice. All the user facing errors and warning should be tranlable.

I have created another MR to fix the occurences

I have started it with 2 examples but someone else can take this over

tedbow’s picture

Also regarding the overhead this would create I checked core and t() is used all the time in core. Since this module is headed for core I think it is fine to do this.

tedbow’s picture

Actually so I could be wrong

In \Drupal\automatic_updates\Validator\StagedProjectsValidator::validateStagedProjects() we actually do

catch (\Throwable $e) {
      $event->addError([
        $e->getMessage(),
      ]);
      return;
    }

so this is non-test code that adds uses a non-translatable string. and this will be passed on to `\Drupal\package_manager\ValidationResult::createError`

tedbow’s picture

If it is only the case that we use non-translated string if we are passing on exception messages we could add

PreOperationStageEvent::addException(\Exception $e, $summary = NULL);

and
ValidationResult::createErrorFromException(\Exception $e, $summary = NULL)

so can more clearer that when you are not just passing on an exception message you need to provide translatable strings.

traviscarden’s picture

I have no concerns with that in principle, @tedbow. I just suggest that we do our best to follow any precedent already in Core in terms of the details, such as using static methods and what to name them.

tedbow’s picture

Issue summary: View changes

tedbow’s picture

Title: Fix various incorrect @param values for $messages » Ensure all validation results use translatable strings except when created from throwables
wim leers’s picture

Shouldn't this be tagged core-mvp? AFAIK translatability is one of the core gates: https://www.drupal.org/about/core/policies/core-change-policies/core-gates (although I don't see it listed? 🤯)

tedbow’s picture

Issue tags: +core-mvp
tedbow’s picture

Issue tags: +sprint
kunal.sachdev’s picture

Assigned: Unassigned » kunal.sachdev
kunal.sachdev’s picture

The test \Drupal\Tests\package_manager\Kernel\StagedDBUpdateValidatorTest fails now if I try to use PluralTranslatableMarkup in \Drupal\package_manager\Validator\StagedDBUpdateValidator Screenshot when using PluralTranslatableMarkup
but If I don't use PluralTranslatableMarkup and directly assign value then tests pass Screenshot when directly assigning value.

wim leers’s picture

To figure out what's going on in #18: put a breakpoint in Connection::__sleep() and observe what is calling it.

kunal.sachdev’s picture

So, actually problem is that if I have xdebug on and I don't even use any translation methods than all tests are failing with same error.

kunal.sachdev’s picture

Assigned: kunal.sachdev » wim leers
wim leers’s picture

#3320782: xdebug being enabled causes tests to fail without clear indication that it is the problem just was a massive distraction because merely enabling xdebug caused every test method in that class to fail! Bumped that to Critical in #3320782-7: xdebug being enabled causes tests to fail without clear indication that it is the problem.

Taking this over because after ~45 mins of debugging we still hadn't found the root cause. This is definitely very tricky!

wim leers’s picture

Huh, even with #3325522: Automatic Updates & Package Manager should use DependencySerializationTrait when needed applied, the Connection::__sleep() exception is thrown.

I spent hours today on debugging this. I even debugged PHPUnit step-by-step. It must be related to some global state because

  protected $backupGlobals = FALSE;
  protected $runTestInSeparateProcess = FALSE;
  protected $backupStaticAttributes = FALSE;

makes it fail in a different way.

If I add an early return []; to Connection::__sleep(), it'll instead try to serialize ContainerBuilder, and if I do the same thing there, it's a Closure… 😬😬😬😬

wim leers’s picture

Assigned: wim leers » Unassigned
StatusFileSize
new656.33 KB
new591.57 KB

🫣 I had to resort to modifying vendor/phpunit/phpunit/src/Util/PHP/Template/TestCaseMethod.tpl to make me able to debug __phpunit_run_isolated_test(), where the

    print serialize(
      [
        'testResult'    => $test->getResult(),
        'numAssertions' => $test->getNumAssertions(),
        'result'        => $result,
        'output'        => $output
      ]
    );

call seems to have been causing this problem.


→ this means that some test is failing and $result contains a failed PHPUnit assertion, which is why PHPUnit will try to generate helpful error output for it, including the stack trace. That stack trace in turn at some point contains a DB connection …

To prove this, I had to painstakingly debug phpunit's template-based test runner logic (see start of this comment), which eventually led me to … Stage. If I add public function __sleep() { return []; } to it, I get a helpful error output instead of the virtually impossible to debug error message that @kunal.sachdev showed in #18:

Time: 00:03.869, Memory: 10.00 MB

There were 2 failures:

1) Drupal\Tests\package_manager\Kernel\StagedDBUpdateValidatorTest::testFileDeleted with data set "hook_update_N" ('install')
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
     0 => Array &1 (
         'severity' => 1
         'messages' => Array &2 (
-            0 => 'System'
-            1 => 'Stark'
+            0 => 'System (1 update)'
+            1 => 'Stark (1 update)'
         )
         'summary' => 'Possible database updates have been detected in the following extensions.'
     )
 )

/Users/wim.leers/core/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:122
/Users/wim.leers/core/vendor/phpunit/phpunit/src/Framework/Constraint/IsIdentical.php:79
/Users/wim.leers/core/modules/contrib/automatic_updates/package_manager/tests/src/Traits/ValidationTestTrait.php:31
/Users/wim.leers/core/modules/contrib/automatic_updates/package_manager/tests/src/Kernel/PackageManagerKernelTestBase.php:200
/Users/wim.leers/core/modules/contrib/automatic_updates/package_manager/tests/src/Kernel/StagedDBUpdateValidatorTest.php:109
/Users/wim.leers/core/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

[…]

Where exactly is that DB connection then? Glad you asked. That too took a lot of time to understand. Heck, what's worse, even if you modify Connection::__sleep() to just return the empty array, there's another! Then it's this dependency chain: Stagefile_system (FileSystem) → container.channel.file (LoggerChannel) → current_user (AccountProxy) → event_dispatcher (ContainerAwareEventDispatcher) → container. If you fix that one, you're faced with a serialized Closure, which … PHP simply does not support.

So, in a way, \Drupal\Core\File\FileSystem is to blame. And many others. But …

⚠️ The reason that this was "never before" a problem: #3307611: Create a validator to add a warning if updated extensions have database updates added Stage $stage as a parameter to \Drupal\Tests\package_manager\Kernel\PackageManagerKernelTestBase::assertStatusCheckResults(), and we simply haven't had to debug this before.

Let's make sure not a single other soul on this project ever has to spend another minute on this nightmare: https://git.drupalcode.org/project/automatic_updates/-/merge_requests/54...


This means @kunal.sachdev (or somebody else) can now continue this issue, because now there are sensible test failures:

Testing Drupal\Tests\package_manager\Kernel\StagedDBUpdateValidatorTest
.FFFFFF.                                                            8 / 8 (100%)

Time: 00:15.938, Memory: 6.00 MB

There were 6 failures:

1) Drupal\Tests\package_manager\Kernel\StagedDBUpdateValidatorTest::testFileDeleted with data set "hook_update_N" ('install')
Failed asserting that two arrays are identical.
--- Expected
+++ Actual
@@ @@
     0 => Array &1 (
         'severity' => 1
         'messages' => Array &2 (
-            0 => 'System'
-            1 => 'Stark'
+            0 => 'System (1 update)'
+            1 => 'Stark (1 update)'
         )
         'summary' => 'Possible database updates have been detected in the following extensions.'
     )
 )
[…]
omkar.podey’s picture

Assigned: Unassigned » omkar.podey
omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
wim leers’s picture

Status: Needs review » Reviewed & tested by the community

🥳

tedbow’s picture

Assigned: Unassigned » wim leers
Status: Reviewed & tested by the community » Needs work

Needs works for 1 point about an unrelated(I think) change

wim leers’s picture

Agreed with @tedbow's concerns. We need to revert the precision in the messages and the function signatures, because it's not accurate. I think it was introduced to be able to meaningfully use PluralTranslatableMarkup, but until #3253828: Use static analysis to detect new update functions, to reduce false positives in StagedDBUpdateValidator lands, that is premature. 😅

omkar.podey’s picture

Assigned: Unassigned » omkar.podey
omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review
wim leers’s picture

Status: Needs review » Needs work

Needs work for a bunch of incorrect formatting/indentation cases 😅

omkar.podey’s picture

Assigned: Unassigned » omkar.podey
omkar.podey’s picture

Status: Needs work » Needs review
omkar.podey’s picture

Assigned: omkar.podey » Unassigned
omkar.podey’s picture

Assigned: Unassigned » wim leers
wim leers’s picture

Assigned: wim leers » omkar.podey
Status: Needs review » Needs work

@omkar.podey: you only fixed the 2 formatting problems that I provided suggestions for in https://git.drupalcode.org/project/automatic_updates/-/merge_requests/54.... There's 4 that I did not provide suggestions for, that still need to be fixed 😅

omkar.podey’s picture

Assigned: omkar.podey » wim leers
Status: Needs work » Needs review
wim leers’s picture

Assigned: wim leers » tedbow
Status: Needs review » Reviewed & tested by the community

Found some problems with the use of the $summary parameter on \Drupal\package_manager\Event\PreOperationStageEvent::addError(), but that's a pre-existing problem. Tagging Needs followup but that's for @tedbow to decide/sort out.

tedbow’s picture

Status: Reviewed & tested by the community » Needs work

Needs work for 7.4 failures. Still reviewing

tedbow’s picture

Assigned: tedbow » omkar.podey
wim leers’s picture

@tedbow: Did you forget to mark this as needing review?

omkar.podey’s picture

Assigned: omkar.podey » Unassigned
Status: Needs work » Needs review

looks good to me

wim leers’s picture

Status: Needs review » Reviewed & tested by the community
tedbow’s picture

Assigned: Unassigned » tedbow

Assigning to myself to commit when tests pass

tedbow’s picture

Status: Reviewed & tested by the community » Fixed

Thanks everyone 💪🎉!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.