Preparing for
#2208609: Move Json utility class into Drupal\Component\Serialization
#1920902: Add a Drupal Yaml wrapper so we can default to PECL Yaml component if it is available
#2190723: Add a KeyValueStore\FileStorage to replace e.g. ConfigStorage

Problem

  1. Various classes are needlessly instantiating Symfony\Component\Yaml\Dumper and Symfony\Component\Yaml\Parser.
  2. ...but only to set parameters to produce YAML that is consistent with Drupal coding standard.
  3. Yaml is actually a serialization format we want to standardize on.

Proposed solution

  1. Add a generic Drupal\Component\Serialization\Yaml helper class.

Ultimate goal

Drupal\Component\Serialization\Json
Drupal\Component\Serialization\PhpSerialize
Drupal\Component\Serialization\Yaml
Drupal\Component\Serialization\Xml
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

sun’s picture

Status: Active » Needs review
FileSize
14.42 KB

Status: Needs review » Needs work

The last submitted patch, 1: yaml.1.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
14.41 KB
634 bytes

Sorry.

larowlan’s picture

We had already rejected a Yaml serializer for Drupal core on security reasons (see #1897612: Add YAML support to serialization module) or is this something different?

damiankloip’s picture

#4, this is not the same this. This would be part of a new internal API used for serializing things like YAML to files etc..

@sun, This seems like both a conflict and a regression on the patch in #1920902: Add a Drupal Yaml wrapper so we can default to PECL Yaml component if it is available. Are we going to this now so we can get some of this new conistency, as that issue will be blocked on parsing details and testbot support.

sun’s picture

Yeah, as outlined by the ultimate goal in the issue summary, this is just about consolidation + centralization + consistency (and thus also DX).

We're using the YAML serialization format in HEAD already. This patch does not add or change any usage of YAML, it only changes how code throughout core encodes and decodes YAML internally.

damiankloip’s picture

Yes, and I am just saying it conflicts totally with your patch in the other issue :) however, looks like that issue could be a complete no-go now anyway. So no conflict really...

sun’s picture

Hah, note that this is the exact same patch as in the other issue: I just (1) removed the Pecl class, (2) removed the Yaml wrapper, and (3) renamed the Symfony class into Yaml. Hence the gross failure in #1 ;-) — Thus, there is not actually a conflict between the two issues/patches. :) But indeed, given recent findings, the other issue could very well turn out to not actually happen, so it was a good idea to split this out after all. :)

Are there any objections to the proposed change, or can we move forward here? :)

sun’s picture

sun’s picture

+++ b/core/lib/Drupal/Component/Discovery/YamlDiscovery.php
@@ -7,7 +7,7 @@
 namespace Drupal\Component\Discovery;
 
-use Symfony\Component\Yaml\Parser;
+use Drupal\Core\Serialization\Yaml;

Hm. This won't work. Two options:

  1. Move YamlDiscovery into Core.
  2. Do not convert YamlDiscovery to use the Yaml serialization class.
  3. Move the Serialization component into Component.
    (cf. #2208609-16: Move Json utility class into Drupal\Component\Serialization)

A quick grep reveals:

$ grep -rli 'Component\\Discovery\\YamlDiscovery' core/
core/lib/Drupal/Component/Discovery/YamlDiscovery.php
core/lib/Drupal/Core/Plugin/Discovery/YamlDiscovery.php
core/lib/Drupal/Core/Routing/RouteBuilder.php
core/tests/Drupal/Tests/Component/Discovery/YamlDiscoveryTest.php
core/tests/Drupal/Tests/Core/Routing/RouteBuilderTest.php

Given what this class is doing — just checking whether a given filename exists in a given list of paths, and parsing all files found — I think we should go with (1) move the YamlDiscovery class into Drupal\Core\Discovery (or Utility).

sun’s picture

FileSize
21.13 KB
7.09 KB

Moved YamlDiscovery into Drupal\Core\Discovery.

Apparently, it lived in there once in the past already, since some stale/outdated phpDoc still references Drupal\Core\Discovery\YamlDiscovery.

sun’s picture

Status: Needs review » Needs work

The last submitted patch, 12: yaml.12.patch, failed testing.

damiankloip’s picture

I would rather see this the other way round dI think. Move serialisation stuff into component. It's relatively generic, like YamlDiscovery.

sun’s picture

Status: Needs work » Needs review
FileSize
29.37 KB
7.6 KB

@damiankloip: The same reasons as in #2208609-14: Move Json utility class into Drupal\Component\Serialization also apply to YamlDiscovery, because the following is literally all it does:

$parsed = array();
foreach ($paths as $provider => $path) {
  $parsed[$provider] = Yaml::decode(file_get_contents("$path/$provider.$name.yml"));
}
return $parsed;

It hard-codes a Drupal-specific concept of providers and corresponding file names that are being looked up in a given set of directories.

I don't even understand why this is a class in the first place... I was very tempted to actually move this into Drupal\Core\Utility instead of an own "Discovery" component.

In any case, there's no point to expose this as a Component to the outside world. There's no concept and also no generic use-case involved that would make it worth for anyone else on this planet. :-)


Converted new instances + preexisting ones that were missing.

damiankloip’s picture

In any case, there's no point to expose this as a Component to the outside world. There's no concept and also no generic use-case involved that would make it worth for anyone else on this planet. :-)

Sorry but that is the case for pretty much all of our "components" not sure why we even bother with them really..

sun’s picture

@damiankloip: Sorry, but that is not true. Let's stick to the facts:

  1. Transliteration: A port of MediaWiki's + other transliteration components + sources, originating from https://drupal.org/project/transliteration. This is some very heavy stuff, I'd be surprised + stunned if you're able to re-implement this on your own.
  2. Gettext: Proper reader/parser + writer for Gettext (.po) files. Epic effort, took several sprints to get right.
  3. PhpStorage: A secure PHP file storage backend.
  4. Plugin: A plugin system for extensible frameworks.
  5. Archiver: A component to manage archive files. (Alternatives exist)
  6. Uuid: A utility component for using the most optimal UUID implementation that exists on a system. (Alternatives exist, but they spawned off from this implementation)

Compared to that — YamlDiscovery? — LOL. ;)

damiankloip’s picture

OK. Thanks for the 'evidence'. Unfollowing.

sun’s picture

Apologies, my last comment might have come across as inappropriate. That was not my intention.

What I was trying to say is that Drupal\Component is not only meant to contain decoupled and re-usable code (not relying on Drupal), but in addition, the contained components should actually be useful for other PHP projects and developers. — Code of a complexity that can be easily written in 5 minutes by any PHP developer does not really meet the aspect of being useful (and hence it will not get re-used).


I originally split these simpler changes into separate smaller issues to facilitate reviews. At this point, the effort of the parent issue is severely blocked on these child issues.

So in case moving the code into Drupal\Component\Serialization would help to move forward here, then I'm happy to concede for the sake of making progress. (I'd have to adjust all other patches accordingly.)

Is that the only objection and point of dissent?

sun’s picture

FileSize
23.37 KB
16.78 KB
  1. Moved Drupal\Core\Serialization into Drupal\Component\Serialization.
  2. Revert "Moved YamlDiscovery into Drupal\Core\Discovery."
  3. Fixed @file phpDoc of DiscoverableInterface.
sun’s picture

Issue summary: View changes

Forgot to update issue summary.

sun’s picture

Title: Add a Yaml class to Drupal\Core\Serialization » Add a Yaml class to Drupal\Component\Serialization

Haha + issue title :)

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

This really makes part of the code way easier to read.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs reroll

yaml.20.patch no longer applies.

error: patch failed: core/includes/common.inc:13
error: core/includes/common.inc: patch does not apply

sun’s picture

Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs reroll
FileSize
23.37 KB

diff context conflict

tstoeckler’s picture

Just for the record:

What I was trying to say is that Drupal\Component is not only meant to contain decoupled and re-usable code (not relying on Drupal), but in addition, the contained components should actually be useful for other PHP projects and developers. — Code of a complexity that can be easily written in 5 minutes by any PHP developer does not really meet the aspect of being useful (and hence it will not get re-used).

This was not the original intention of the \Drupal\Component namespace. See /core/lib/Drupal/Component/README.txt

Drupal Components are independent libraries that do not depend on the rest of
Drupal in order to function.

So if @sun's statement is in fact now community consensus, then we should update the README.txt (not this issue, of course, just sayin').

I personally disagree completely, to me the README.txt expresses exactly what \Drupal\Component is, but also I don't really care much about the location either way, so please continue :-)

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 25: yaml.25.patch, failed testing.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
23.33 KB

Diff context conflict.

sun’s picture

Would be great to move forward here...

#2161591: Change default active config from file storage to DB storage needlessly converts lots of code throughout core to manually pass custom arguments to Yaml\Dumper::dump().

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 28: yaml.28.patch, failed testing.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
30.87 KB
6.48 KB

Updated new instances.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 31: yaml.31.patch, failed testing.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
30.92 KB

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 33: yaml.33.patch, failed testing.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
30.95 KB
webchick’s picture

Status: Reviewed & tested by the community » Needs work

No longer applies.

sun’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
30.94 KB
webchick’s picture

Status: Reviewed & tested by the community » Fixed

Ok, great. Committed/pushed to 8.x. Thanks!

  • Commit b390475 on 8.x by webchick:
    Issue #2208633 by sun: Add a Yaml class to Drupal\Component\...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.