Part of meta-issue #2571965: [meta] Fix PHP coding standards in core, stage 1

In this issue we should not handle Squiz.Arrays.ArrayDeclaration.NoKeySpecified fixed in #2908266: Fix 'Squiz.Arrays.ArrayDeclaration.NoKeySpecified' and 'Squiz.Arrays.ArrayDeclaration.KeySpecified' coding standard.

Step 1: Preparation

Open the file core/phpcs.xml.dist and add a line for the sniff of this ticket. The sniff name is in the issue title. Make sure your patch will include the addition of this line.

Step 2: Install & configure PHPCS

Install PHP CodeSniffer and the ruleset from the Coder module:

$ composer install
$ ./vendor/bin/phpcs --config-set installed_paths ../../drupal/coder/coder_sniffer

Once you have installed the phpcs package, you can list all the sniffs available to you like this:

$ ./vendor/bin/phpcs --standard=Drupal -e

This will give you a big list of sniffs, and the Drupal-based ones should be present.

Step 3: Prepare the phpcs.xml file

To speed up the testing you should make a copy of the file phpcs.xml.dist (in the core/ folder) and save it as phpcs.xml. This is the configuration file for PHP CodeSniffer.

We only want this phpcs.xml file to specify the sniff we're interested in. So we need to remove all the rule items, and add only our own sniff's rule. Rule items look like this:

<rule ref="Drupal.Classes.UnusedUseStatement"/>

Remove all of them, and add only the sniff from this issue title. This will make sure that our tests run quickly, and are not going to contain any output from unrelated sniffs.

Step 4: Run the test

Now you are ready to run the test! From within the core/ folder, run the following command to launch the test:

$ cd core/
$ ../vendor/bin/phpcs -p

This takes a couple of minutes. The -p flag shows the progress, so you have a bunch of nice dots to look at while it is running.

Step 5: Fix the failures

When the test is complete it will present you a list of all the files that contain violations of your sniff, and the line numbers where the violations occur. You could fix all of these manually, but thankfully phpcbf can fix many of them. You can call phpcbf like this:

$ ../vendor/bin/phpcbf

This will fix the errors in place. You can then make a diff of the changes using git. You can also re-run the test with phpcs and determine if that fixed all of them.

Comments

mfernea created an issue. See original summary.

mfernea’s picture

Status: Needs review » Active
andriyun’s picture

Assigned: Unassigned » andriyun
andriyun’s picture

When I'm trying to check code with Squiz.Arrays.ArrayDeclaration sniff I get not proper failures like

-          '#title' => 'Error',
-          '#markup' => $message,
-        ];
+                   '#title'  => 'Error',
+                   '#markup' => $message,
+                  ];

Squiz.Arrays.ArrayDeclaration sniff has configurable options
But there are some issues with that
https://github.com/squizlabs/PHP_CodeSniffer/issues/582

Any idea what we can do?

mfernea’s picture

I think the problem is that you didn't update phpcs.xml (for local testing) or phpcs.xml.dist (to be included in the patch) as per ruleset.xml from Drupal CS.
The modifications should look like this:

  <rule ref="Squiz.Arrays.ArrayDeclaration"/>
  <!-- Disable some error messages that we do not want. -->
  <rule ref="Squiz.Arrays.ArrayDeclaration.CloseBraceNotAligned">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.DoubleArrowNotAligned">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.FirstValueNoNewline">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.KeyNotAligned">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.MultiLineNotAllowed">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.NoComma">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.NoCommaAfterLast">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.NotLowerCase">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.SingleLineNotAllowed">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.ValueNotAligned">
    <severity>0</severity>
  </rule>
  <rule ref="Squiz.Arrays.ArrayDeclaration.ValueNoNewline">
    <severity>0</severity>
  </rule>

If I do that, I don't get any errors on the lines you mentioned.

andriyun’s picture

Assigned: andriyun » Unassigned
Status: Active » Needs review
StatusFileSize
new163.71 KB

Thank you @mfernea

It works with your ruleset.

After phpcbf I have to fix a lot of cases manually.
And finally phpcs reports only failures for arrays like

$array = array(
    'value',
    'key' => 'value',
);
mfernea’s picture

Status: Needs review » Needs work

Good work! Not an easy one, for sure.

Besides the cs issues in the test results (which we should fix), I have few remarks about the modifications in the patch:

  1. DateFormatDeleteForm.php - I would not put the first bracket on a new line.
  2. I think we should make the code less expanded, when possible, and still follow the CS for:
    1. PluginBaseTest.php
    2. FileCacheFactoryTest.php
    3. EntityFormTest.php
    4. EntityTypeBundleInfoTest.php
    5. RendererTest.php
    6. testGetEntityTypeForFieldWithRelationship() function
  3. Indentation is not correct for:
    1. ModuleHandlerTest.php
    2. EditorLoadingTest.php
    3. testRevisionTableWithRevisionDataTableAndDataTable()
andriyun’s picture

Status: Needs work » Needs review
StatusFileSize
new163.68 KB
new12.2 KB

Fixed review feedback.
Not all suggestions are passed CS.
For example following is failure

$array = [
      ['key1' => [   // <--- here
          'key11' => 'value',
        ],
      ],
      'value2',
    ];
mfernea’s picture

Status: Needs review » Needs work

Looking better! The test result still shows CS issues. Can we fix them?
Regarding the above example, it's very similar to the modifications in PluginBaseTest.php and we can use:

$array = [
  ['key1' => ['key11' => 'value']],
  'value2',
];

Sometimes we cannot do that, but I would reduce the height of the code whenever possible.

Another thing: Don't you think we sbould also add a comma after the last item of the array when putting the closing bracket on a new line? I'm looking for example at modifications in core/lib/Drupal/Component/Utility/Random.php. They don't break current standards in phpcs.xml.dist, but they introduce new issues.

jofitz’s picture

Status: Needs work » Needs review
StatusFileSize
new164.25 KB
new2.63 KB

I've started to corrected the CS issues, but can you confirm that I have the correct approach, e.g. ConfigEntityTypeTest.php:

     $data[] = [
       [
         'config_export' => [
-          'id',
+          0 => 'id',
           'custom_property' => 'customProperty',
         ],
       ],
andriyun’s picture

Assigned: Unassigned » andriyun
andriyun’s picture

StatusFileSize
new163.51 KB
new18.37 KB

@mfernea agree with your notice.
Fixed in the current patch.

@Jo I don't think that your approach is correct.
- it adds obvious code changes.
- we can't apply this approach to files like http://cgit.drupalcode.org/drupal/tree/core/lib/Drupal/Component/Transli...

FILE: ...ib/drupal/core/lib/Drupal/Component/Transliteration/data/x21.php
----------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
----------------------------------------------------------------------
 9 | ERROR | No key specified for array entry; first entry specifies
   |       | key
----------------------------------------------------------------------


FILE: ...trib/drupal/core/tests/Drupal/Tests/Core/Render/RendererTest.php
----------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
----------------------------------------------------------------------
 287 | ERROR | No key specified for array entry; first entry
     |       | specifies key
----------------------------------------------------------------------

I suppose that we have to exclude this sniff from list is that is available

mfernea’s picture

Status: Needs review » Needs work

Actually Jo is correct. In the end that is how the array would look like with our w/o specifing the key. So I think that is a good approach.

For files in core/lib/Drupal/Component/Transliteration/data... I'm not really sure how these arrays are used, but I would exclude that folder for this sniff.

  <rule ref="Squiz.Arrays.ArrayDeclaration.NoKeySpecified">
    <exclude-pattern>lib/Drupal/Component/Transliteration/data/*</exclude-pattern>
  </rule>

@andriyun: In the last interdiff for RendererTest.php I see 2 issues:

  • first modification doesn't relate to this issue although is correct
  • the second modification has an indentation issue
mfernea’s picture

I'm think we should handle Squiz.Arrays.ArrayDeclaration.NoKeySpecified in another issue. We are almost done with the rest of the sniffs and for that one it might take longer to settle upon a solution. What do you think?

jofitz’s picture

Sounds sensible to me.

mfernea’s picture

andriyun’s picture

StatusFileSize
new163.63 KB
new1.09 KB

Good idea @mfernea
Thank you.

Updated patch contain:

  • exclude sniff rule to phpcs.xml.dist
  • removed not related to this sniff changes
  • fix indentation issues
mfernea’s picture

Assigned: andriyun » mfernea
Status: Needs work » Needs review

Status update for testbots. Interdiff looks good. I will assign it to me for review. Normally we should have an RTBC.

mfernea’s picture

Status: Needs review » Needs work

Being a long patch I noticed things that I haven't noticed before:

  • DatabaseStorage.php - Let's not add a new line after "query("
  • BigPipeStrategyTest.php - missing colon for the second modification
  • ConfigMapperManagerTest.php - for the first 2 modifications we can put the array on a single line
  • NodeRevisionsTest.php - Let's use the same style for both modifications. I would go for the second's.
  • DateFormatDeleteForm.php _ I think we can reduce the indentation and have ] and ) on the same line. If not, let's leave it like this.
  • SqlContentEntityStorageTest.php - I would put each array in a single line. It's just a "matter of taste" :). It's also ok like this.
  • RendererBubblingTest.php - Some colons missing.
  • phpcs.xml.dist - Don't remove the blank line before </ruleset>
andriyun’s picture

Assigned: mfernea » andriyun

Thank you for a review.

andriyun’s picture

Status: Needs work » Needs review
StatusFileSize
new162.92 KB
new8.09 KB

Fixed from review #19

mfernea’s picture

Status: Needs review » Needs work

I looked at the test results and I saw these:

  1. +++ b/core/phpcs.xml.dist
    @@ -148,6 +148,43 @@
    +  <rule ref="Squiz.Arrays.ArrayDeclaration">
    +    <exclude name="Squiz.Arrays.ArrayDeclaration.NoKeySpecified"/>
    +  </rule>
    +  <!-- Disable some error messages that we do not want. -->
    

    We also have to exclude Squiz.Arrays.ArrayDeclaration.KeySpecified as there some issues with it. We'll solve both excludes in #2908266: Fix 'Squiz.Arrays.ArrayDeclaration.NoKeySpecified' and 'Squiz.Arrays.ArrayDeclaration.KeySpecified' coding standard.

  2. +++ b/core/tests/Drupal/Tests/Core/Render/RendererTest.php
    @@ -74,98 +74,145 @@ public function providerTestRenderBasic() {
    +      // XSS filtering test.
    

    This has to be properly indented.

andriyun’s picture

Status: Needs work » Needs review
StatusFileSize
new162.99 KB

1. is not related to issue sniff and should be fixed in proper issue
2. added Squiz.Arrays.ArrayDeclaration.KeySpecified to phpcs.xml.dist file

Status: Needs review » Needs work

The last submitted patch, 23: fix-2901739-23.patch, failed testing. View results

andriyun’s picture

Status: Needs work » Needs review
StatusFileSize
new162.99 KB

Patch reroll

mfernea’s picture

Status: Needs review » Needs work

The test results still shows 1 cs issue which was introduced by this patch. I was wrong at #13 to advise that we should not fix that.

andriyun’s picture

StatusFileSize
new162.77 KB
new433 bytes
andriyun’s picture

Status: Needs work » Needs review
mfernea’s picture

Status: Needs review » Reviewed & tested by the community

Looks good now. No cs issues. RTBC.
Thanks @andriyun! :)

xjm’s picture

Version: 8.5.x-dev » 8.4.x-dev
Issue tags: +rc target

As a large coding standards cleanup that could disrupt other patches, this is one we want to schedule for during the RC phase (which is, conveniently, now). Don't have time to review just yet, but tagging. This will likely break other patches.

xjm’s picture

Alright, I want to get this one in soon because it will need rerolls a lot.

Reviewed with git diff --color-words --staged to ensure there were no other changes other than the array formatting. I confirmed that the patch's only non-whitespace changes were adding trailing commas (and in a couple of cases rearranging closing brackets and trailing commas across lines).

The word diff got confused on the hunks in the following files which I will need to read carefully by hand:

  • core/modules/content_translation/tests/src/Unit/Menu/ContentTranslationLocalTasksTest.php
    This one is actually easier to read in HEAD. That's a shame. I checked it carefully and it looks correct.
  • core/modules/user/src/Form/UserPermissionsForm.php
    word diff really made this one look like it was rearranging the array but it's not. Looks okay.
  • core/tests/Drupal/Tests/Component/FileCache/FileCacheFactoryTest.php
    Eesh this one is hard to read in HEAD.
  • core/tests/Drupal/Tests/Core/Form/FormCacheTest.php
    I read it carefully in HEAD and with the patch; it looks correct.
  • core/modules/config_translation/tests/src/Unit/ConfigMapperManagerTest.php
    This is the only one I still haven't finished reading. Damn it's hard to follow in HEAD; I'm having to count brackets on my fingers. The word diff also totally fails because there are so many similar lines.

I need to go AFK so documenting this to pick up where I left off.

xjm’s picture

Issue tags: +Needs followup

Let's add a followup to add inline comments documenting the provider in ConfigMapperManagerTest::providerTestHasTranslatable(). The problems are totally out of scope here but we really should have comments that explain each case in a provider. This one is definitely not self-documenting.

xjm’s picture

Okay, finally finished spot-checking that last provider.

+++ b/core/phpcs.xml.dist
@@ -147,6 +147,44 @@
   <!-- Squiz sniffs -->
+  <rule ref="Squiz.Arrays.ArrayDeclaration">
+    <exclude name="Squiz.Arrays.ArrayDeclaration.NoKeySpecified"/>
+    <exclude name="Squiz.Arrays.ArrayDeclaration.KeySpecified"/>
+  </rule>
+  <!-- Disable some error messages that we do not want. -->
+  <rule ref="Squiz.Arrays.ArrayDeclaration.CloseBraceNotAligned">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.DoubleArrowNotAligned">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.FirstValueNoNewline">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.KeyNotAligned">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.MultiLineNotAllowed">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.NoComma">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.NoCommaAfterLast">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.NotLowerCase">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.SingleLineNotAllowed">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.ValueNotAligned">
+    <severity>0</severity>
+  </rule>
+  <rule ref="Squiz.Arrays.ArrayDeclaration.ValueNoNewline">
+    <severity>0</severity>
+  </rule>

This is adding a lot of this pattern to phpcs.xml.dist. We do have it elswhere in the file, in the include section. Other places, we do exclude name = "". What's the reason for the different format? I see this discussed a little in #5 but it's not totally clear to me from that.

mfernea’s picture

From my pov phpcs.xml.dist should look like Drupal CS' ruleset.xml. So, to avoid differences I think it's best to copy the code from ruleset.xml as it is.
We added:

    <exclude name="Squiz.Arrays.ArrayDeclaration.NoKeySpecified"/>
    <exclude name="Squiz.Arrays.ArrayDeclaration.KeySpecified"/>

just to not target too many sniffs.

xjm’s picture

This format with the severoty 0 rather than including things was added to the few other lines in #2902396: Add sniffs already passing and not mentioned there.

mfernea’s picture

I added there a comment about this at https://www.drupal.org/node/2902396#comment-12227553, but maybe it should have been present in the issue summary too.

xjm’s picture

@mfernea, are you saying that these exclusions using severity 0 are from coder itself?

Looks like it's so:
http://cgit.drupalcode.org/coder/tree/coder_sniffer/Drupal/ruleset.xml

So I can see the case for doing that then, good point.

Any idea what the difference is between that and the "exclude" format? Is there any?

mfernea’s picture

Indeed, this is what I meant.

Not sure about the difference. The end result is the same.
https://github.com/squizlabs/PHP_CodeSniffer/wiki/Annotated-ruleset.xml doesn't mention differences between the two for our use case.

xjm’s picture

Thanks @mfernea. I posted #2909267: Prefer "exclude" format over "severity 0" format in ruleset for readability and #2909268: Prefer "exclude" format over "severity 0" format in ruleset for readability once coder does. Now running all the array rules against both 8.5.x and 8.4.x locally just to check that no new mis-formatted arrays have been committed in the meanwhile.

  • xjm committed 44a5c4c on 8.5.x
    Issue #2901739 by andriyun, Jo Fitzgerald, mfernea, xjm: Fix 'Squiz....
xjm’s picture

Status: Reviewed & tested by the community » Fixed

Alrighty, committed and pushed to 8.5.x and backported to 8.4.x as a coding standards cleanup during RC. Thanks everyone!

  • xjm committed 2370f4d on 8.4.x
    Issue #2901739 by andriyun, Jo Fitzgerald, mfernea, xjm: Fix 'Squiz....
tacituseu’s picture

Status: Fixed » Active

Looks like this change made phpcs super slow on PHP 5.5 environments (some tests are aborting after 110 min now), compare:
8.5.x-dev test with PHP 5.5 & PostgreSQL 9.1
before: "Finished phpcs in 230.14850497246"
after: "Finished phpcs in 2306.7364938259"
8.5.x-dev test with PHP 5.5 & SQLite 3.8
before: "Finished phpcs in 233.63911414146"
after: "Finished phpcs in 2305.4536380768"
so it has added 34 minutes to the runtime.

andriyun’s picture

Regarding requirement Drupal 8 need at least PHP 5.5.9+

What version of PHP you have on your environvent?

tacituseu’s picture

@andriyun: I'm fine :), what I meant is Drupal CI Testbots, they're on PHP 5.5.38 (see https://www.drupal.org/pift-ci-job/767155), PHP 5.6+ isn't showing the symptoms.

andypost’s picture

Assigned: andriyun » Unassigned
Status: Active » Fixed

Let's fix that in follow-up

PS: probably the slowdown is #2911280: RectangleTest.php takes a very long time to scan for coding standards

andypost’s picture

tacituseu’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.