Problem/Motivation

In Package Manager (and Automatic Updates), we decode a lot of JSON. In some situations, it might make sense to use the JSON_THROW_ON_ERROR flag.

Proposed resolution

Let's go through all of our calls to Json::decode() and json_decode() and decide, on a case-by-case basis, if we should pass the JSON_THROW_ON_ERROR flag (which necessitates using json_decode).

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

yash.rode created an issue. See original summary.

tedbow’s picture

Issue tags: +core-post-mvp
tedbow’s picture

Version: 8.x-2.x-dev » 3.0.x-dev
yash.rode’s picture

Assigned: Unassigned » yash.rode
Status: Active » Needs work

yash.rode’s picture

Title: Check JSON file is accessible or not in all build tests. » Check JSON is malformed in all build tests.
yash.rode’s picture

Issue tags: +sprint
yash.rode’s picture

Title: Check JSON is malformed in all build tests. » [PP-1] Check JSON is malformed in all build tests.
Status: Needs work » Postponed
yash.rode’s picture

Title: [PP-1] Check JSON is malformed in all build tests. » [PP-1] Check is JSON malformed in all build tests.
yash.rode’s picture

Title: [PP-1] Check is JSON malformed in all build tests. » Check is JSON malformed in all build tests.
Status: Postponed » Needs work

Blocker is merged!

yash.rode’s picture

Assigned: yash.rode » Unassigned
Status: Needs work » Needs review

Discussed this with @tedbow, we should check this for all the occurrences of json_decode not only build tests!

omkar.podey’s picture

Assigned: Unassigned » omkar.podey
omkar.podey’s picture

Looks good but i wonder what about Json::Decode do we replace all those instance with json_decode and add the flags ? OR is there a reason we use Json::Decode in some places.

omkar.podey’s picture

Assigned: omkar.podey » yash.rode
Status: Needs review » Needs work
yash.rode’s picture

Assigned: yash.rode » Unassigned
Status: Needs work » Needs review
omkar.podey’s picture

All instances of Json::Decode are now replaced, everything else looks good already. , nice work.

tedbow’s picture

Assigned: Unassigned » tedbow

reviewing

tedbow’s picture

Status: Needs review » Reviewed & tested by the community

Looks good! thanks!

tedbow’s picture

Assigned: tedbow » Unassigned
phenaproxima’s picture

Title: Check is JSON malformed in all build tests. » All build tests should check if JSON is malformed
phenaproxima’s picture

Title: All build tests should check if JSON is malformed » Always pass JSON_THROW_ON_ERROR when decoding JSON
phenaproxima’s picture

Assigned: Unassigned » tedbow
Status: Reviewed & tested by the community » Needs review

The repetition here concerns me. Maybe this calls for a utility method on ComposerInspector to always decode JSON the same way. What do you think, @tedbow?

phenaproxima’s picture

Status: Needs review » Needs work

Discussed with @tedbow.

I really don't want to repeat the same call a million times -- in the best case, it's sometimes unnecessary, and in the worst case, it's fragile when spread widely over the codebase.

I've gone through this with a fine-tooth comb and suggested where, IMHO, we should keep the JSON_THROW_ON_ERROR flag, where we should revert it, and where we should replace it with something else. Leaving assigned to @tedbow to implement these changes, since he may have different ideas than me about what should be changed.

tedbow’s picture

Assigned: tedbow » phenaproxima
Status: Needs work » Needs review

@phenaproxima mostly accepted your suggestions. Commented on the others in the MR

phenaproxima’s picture

Assigned: phenaproxima » tedbow
Status: Needs review » Needs work

Generally I'm okay with your decisions, but there are a couple of places where I don't...thoughts? Also, tests are failing, so kicking back for that.

tedbow’s picture

We need more of summary for this to be reviewable otherwise we can't know if is correct

tedbow’s picture

Assigned: tedbow » phenaproxima

@phenaproxima since you made the comment the quoted in the original issue to add one instance of `JSON_THROW_ON_ERROR` and then this issue started to add them everywhere and then we removed them again could you update the summary to what the goal and proposed solutions is here

phenaproxima’s picture

Title: Always pass JSON_THROW_ON_ERROR when decoding JSON » In some situations, pass JSON_THROW_ON_ERROR when decoding JSON
phenaproxima’s picture

Issue summary: View changes
phenaproxima’s picture

Assigned: phenaproxima » tedbow
Issue summary: View changes
Issue tags: -Needs issue summary update

Okay, I updated the summary. Assigning back to @tedbow to respond to my feedback (and hopefully fix the broken tests).

tedbow’s picture

Assigned: tedbow » phenaproxima
Status: Needs work » Needs review
phenaproxima’s picture

Assigned: phenaproxima » tedbow
Status: Needs review » Needs work

Discussed with @tedbow on Zoom.

I really, really don't like the repetition, and the need to always pass the recursion depth, but Ted correctly pointed out that adding a protected static method on TemplateProjectTestBase would, by core's policies, be considered API. Even if we explicitly marked the base class internal.

Neither of us likes that policy one bit. The concept of "internal" is rendered utterly meaningless if we don't stand by it in practice. However, this is a core policy that we have to abide by, at least for now, until it changes in a more sane direction. I hereby shake my fist and growl at the policy, and call it a dummkopf.

But, then I had an idea: we could take advantage of PHP 8's support for named arguments, and leave out the recursion depth. So this:

json_decode($value, TRUE, 512, JSON_THROW_ON_ERROR);

Becomes this:

json_decode($value, TRUE, flags: JSON_THROW_ON_ERROR);

This is a compromise I can live with. True, we'd be repeating the call several times because of core's dumb policy, but we also wouldn't need to pass a worthless and confusing extra argument.

tedbow’s picture

Assigned: tedbow » phenaproxima
Status: Needs work » Needs review

implemented #33

phenaproxima’s picture

Assigned: phenaproxima » Unassigned
Status: Needs review » Reviewed & tested by the community

Looks good to when tests pass.

phenaproxima’s picture

Status: Reviewed & tested by the community » Fixed
wim leers’s picture

Status: Fixed » Postponed (maintainer needs more info)

The use of named parameters thanks to PHP 8 makes sense. 👍

The inconsistent updating in this commit of some json_decode() calls to Json::decode() instead, whereas in other cases doing the exact opposite makes no sense to me. I see there's been discussion about this, but I don't see a conclusion documented anywhere and I cannot discern any pattern.

For example:

  • package_manager/src/Validator/AllowedScaffoldPackagesValidator.php was updated to START using Json::decode()
  • package_manager/src/Validator/AllowedScaffoldPackagesValidator.php was updated to STOP using Json::decode()

It appears that the pattern is to use Json::decode() in all non-test cases except in cases where we want to set the JSON_THROW_ON_ERROR flag? But why do we want that flag in the one case but not in the other?

That information is missing from the issue's comments.