Problem/Motivation

Currently, the "Rest Export" display style provided by the core rest module does not pass any contextual information about the view to the Serializing class. The consequence is that that encoding class cannot dynamically respond to any views configuration whatsoever. This arguably contradicts the intention of the EncoderInterface used by the Serialization API, as the serialize() method already allows for an option $context parameter to be passed--it simply isn't being used at present.

Proposed resolution

Pass the views options array to the serialize() method of the active Serializer.

Example Usage

I've relied on this patch to develop the CSV Serialization module. By simply defining a new Serialization class, I am easily able to provide a View of display type "Rest Export" that displays CSV data.

However, I'm unable to pass any configuration to the Serializer. E.g., I cannot use Views (or anything else for that matter) to configure the delimiter, escape characters, or enclosure characters for the CSV serialization.

In this case, I believe that the simplest way to pass user configurable options to the Serializer from Views is to 1) Use Form API to provide additional options, 2) Pass the entire view style array of options through to the Serializer. \

Remaining tasks

  • Get code reviewed
  • Merge!

User interface changes

NONE

API changes

NONE

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

grasmash created an issue. See original summary.

grasmash’s picture

Status: Active » Needs review
FileSize
978 bytes

Very simple one line patch attached.

vijaycs85’s picture

Looks good. Is there anyway we can test it manually? or possible to write test?

dawehner’s picture

Does $this->options contain information you actually want in there? Aren't those two different kind of options?

grasmash’s picture

@dawehner $this->options does contain information that one would want in there. Some examples:

  • Rest Export display for Views provides configuration options for XML response format. E.g., permits xml namespacing. Serializer must know views config.
  • Rest Export display for Views provides configuration options for CSV response format. E.g., delimiter, enclosure character, escape character, etc. Serializer must know views config.

Passing $this->options leaves things nicely flexible, because you could form_alter() the views config form and easily add any option you'd like, whose value would subsequently be passed to a custom defined Serialization class and used to dynamically encode response.

dawehner’s picture

Issue tags: +Needs tests

Well yeah I just fear that random configuration of one might leak into the serializer and then cause something which was not intended to do so.

klausi’s picture

Status: Needs review » Needs work
+++ b/core/modules/rest/src/Plugin/views/style/Serializer.php
@@ -136,7 +136,7 @@ public function render() {
-    return $this->serializer->serialize($rows, $content_type);

I think we should not pass all the options on the top level of the $context array for the serializer. So something like return $this->serializer->serialize($rows, $content_type, ['views_style_options' => $this->options])

For the test case I think a unit test that asserts that the context is passed to a serializer mock should suffice. Extra points if you use Prophecy for the mocking, see https://www.drupal.org/node/2554745

grasmash’s picture

@klausi Alright, I'm going to take a stab at both writing a test and using Prophecy, but it's a little murky to me at the moment.

grasmash’s picture

@klausi I think that writing this automated test is a bit more complex than expected.

The proposed testing strategy would require the following:

  • Create a new test case
  • Create a mock serializer
  • Instantiate an example view
  • Assign the mock serializer to the example view -- not possible given that this is a private property
  • Modify example view by setting custom style options
  • Execute and render view
  • Assert that expected vars were passed to mock serializer

As noted, the issue with this strategy is that you cannot override a view's serializer, which is derived from injected dependencies. I could go down a fairly serious rabbit hole and try to create a mock view, but the issue of calling execute and render methods on this becomes problematic. On the other hand, I could introduce public getting and setter methods for setting the serializer on a view, but that circumvents dependency injection and seems like a bad idea.

So, I'm at an impasse due to lack of D8 automated testing knowledge. Any guidance would be appreciated.

grasmash’s picture

Status: Needs work » Needs review

Updating patch so that views config is not passed to top level of array.

grasmash’s picture

Oops. Attaching patch.

Status: Needs review » Needs work

The last submitted patch, 11: pass_views_config-2568413-11.patch, failed testing.

webchick’s picture

This is apparently a contrib project blocker (for https://www.drupal.org/project/views_data_export). Tagging as such.

grasmash’s picture

Rerolling patch due to unintentional whitespace change, standby.

grasmash’s picture

Status: Needs work » Needs review
FileSize
1022 bytes

Let's try this again.

The last submitted patch, 11: pass_views_config-2568413-11.patch, failed testing.

pwolanin’s picture

For consistency with the rest of that method, either use old-style array() syntax, or fix the line:

 $rows = array();
grasmash’s picture

pwolanin’s picture

Here's a unit test that tries to keep the mocking to the minimum needed, and fixed the syle concern.

Not really very useful, but it's a start for future test writing around this.

The last submitted patch, 19: 2568413-18-test-only.patch, failed testing.

dawehner’s picture

  1. +++ b/core/modules/rest/src/Plugin/views/style/Serializer.php
    @@ -136,7 +136,7 @@ public function render() {
    +    return $this->serializer->serialize($rows, $content_type, array('views_style_options' => $this->options));
    

    Ah nice, so in this way it at least doesn't conflict with anything else. Thank you!

  2. +++ b/core/modules/rest/tests/src/Unit/RestSerializerTest.php
    @@ -0,0 +1,81 @@
    +/**
    + * Tests the REST export view plugin.
    + *
    + * @group rest
    + */
    +class RestSerializerTest extends UnitTestCase {
    ...
    +  /**
    +   * Tests that the symfony serializer receives options from the render() method.
    +   */
    +  public function testSerializerReceivesOptions() {
    

    We could add @coversDefaultClass | @covers

The last submitted patch, 19: 2568413-18-test-only.patch, failed testing.

pwolanin’s picture

Issue tags: -Needs tests
FileSize
3.18 KB
748 bytes

ok, fixed the doc.

klausi’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/rest/tests/src/Unit/RestSerializerTest.php
    @@ -0,0 +1,82 @@
    +class RestSerializerTest extends UnitTestCase {
    

    Can we name the class "ViewsRestSerializerTest" or "RestViewsSerializerTest" to indicate that this is about the Views integration of REST module?

  2. +++ b/core/modules/rest/tests/src/Unit/RestSerializerTest.php
    @@ -0,0 +1,82 @@
    +    $this->view = $this->getMock('\Drupal\views\Entity\View', array('initHandlers'), array(
    

    don't use the class name as literal string, use ::class instead. Also elsewhere.

  3. +++ b/core/modules/rest/tests/src/Unit/RestSerializerTest.php
    @@ -0,0 +1,82 @@
    +      ->will($this->returnValue(''));
    

    can use willReturn()

dawehner’s picture

Can we name the class "ViewsRestSerializerTest" or "RestViewsSerializerTest" to indicate that this is about the Views integration of REST module?

It should be moved IMHO to the right

+++ b/core/modules/rest/tests/src/Unit/RestSerializerTest.php
@@ -0,0 +1,82 @@
+class RestSerializerTest extends UnitTestCase {

let's move the test to src/Unit/Plugin/views/style/SerializerTest.php to be more parallel to the actual source code.

pwolanin’s picture

Since we are testing a rest module class the test should live with rest module

pwolanin’s picture

Status: Needs work » Needs review
FileSize
3.27 KB
2.66 KB

renamed class to \Drupal\Tests\rest\Unit\RestViewsStyleSerializerTest

fixed other concerns.

dawehner’s picture

Well, I disagree :)
In general this test also used the wrong class ...

klausi’s picture

  1. --- a/core/modules/rest/tests/src/Unit/RestViewsStyleSerializerTest.php
    +++ b/core/modules/rest/tests/src/Unit/Plugin/views/style/SerializerTest.php
    

    Do we have a coding standard for this that we have to nest unit tests that deep in plugin folders? Since the test is not a plugin we can avoid those annoying subdirectories.

  2. +++ b/core/modules/rest/tests/src/Unit/Plugin/views/style/SerializerTest.php
    @@ -10,19 +10,19 @@
    -class RestViewsStyleSerializerTest extends UnitTestCase {
    +class SerializerTest extends UnitTestCase {
    

    the name "SerializerTest" in REST module is too generic I would say and makes it harder to find. I know you tried to compensate that with the nested directories, but for such a few number of unit tests that we have in REST module a longer class name makes more sense IMO. Less annoying than 3 additional levels of directories.

  3. +++ b/core/modules/rest/tests/src/Unit/Plugin/views/style/SerializerTest.php
    @@ -65,18 +65,13 @@ protected function setUp() {
    +    $mock_serializer->serialize([], 'json', ['views_style_options' => $options])->willReturn('')->shouldBeCalled();
    

    can we split this up over multiple lines as we did before? Makes it easier to read.

Anyway, just non-blocking nitpicks.

Status: Needs review » Needs work

The last submitted patch, 28: 2568413-28.patch, failed testing.

dawehner’s picture

Do we have a coding standard for this that we have to nest unit tests that deep in plugin folders? Since the test is not a plugin we can avoid those annoying subdirectories.

Well, IMHO this is common pratice we apply for 95% of all our unit tests.

the name "SerializerTest" in REST module is too generic I would say and makes it harder to find. I know you tried to compensate that with the nested directories, but for such a few number of unit tests that we have in REST module a longer class name makes more sense IMO. Less annoying than 3 additional levels of directories.

This is why we have namespaces ... This naming scheme also helps to quickly switch between test and code class.

damiankloip’s picture

Firstly:

This arguable contradicts the intention of the EncoderInterface used by the Serialization API, as the serialize() method already allows for an option $context parameter to be passed--it simply isn't being used at present.

Not really. This is a totally optional parameter. I don't think anything is contradicted here. Just as you say, un used.

Do you have a more specific use case here? As having a normalizer or encoder that depends on view configuration could easily break when you are not using it from views.

If we are doing this, which I don't have a problem with, I think we should pass more context, not style options only. Style configuration is arguable one of the least helpful IMO :) If we can fix your case, and others that may potentially arise, that would be good.

pwolanin’s picture

grasmash’s picture

Issue summary: View changes
grasmash’s picture

I've added an example use case to the issue summary. There are of course many other examples of possible ways that we could use the $context parameter in serialize().

grasmash’s picture

Issue summary: View changes

I'm also not opposed to passing more context (beyond style options) to the serializer. But passing $this->options seems like the logical place to start. These can be easily modified through the UI or through the use of hooks.

pwolanin queued 28: 2568413-28.patch for re-testing.

pwolanin’s picture

Status: Needs work » Needs review
Issue tags: -Needs issue summary update

Status: Needs review » Needs work

The last submitted patch, 28: 2568413-28.patch, failed testing.

xjm’s picture

Category: Feature request » Task
Issue tags: +rc target triage

As a contrib project blocker, this would be a good issue to consider for RC target triage.

pwolanin queued 28: 2568413-28.patch for re-testing.

xjm’s picture

Status: Needs work » Needs review
FileSize
3.17 KB

Hm that test result is on QA which is now essentially switched off... canceling that run and reuploading the patch.

grasmash’s picture

@xjm Are we safe to consider this reviewed and tested by community at this point? The change meets the needs of the contrib module, and the tests appear to pass.

xjm’s picture

I think #29 through #32 still need to be addressed in full.

grasmash’s picture

@xjm Thanks.

Here are comments to address the outstanding issues:

  • #29.1 Has been addressed by @dawehner in #31. Do we consider this an imperative?
  • #29.2 Has been addressed by @dawehner in #31. Do we consider this an imperative?
  • #29.3 Has been addressed in the attached patch.
  • #32. " I think we should pass more context, not style options only". @damiankloip This patch affects Drupal\rest\Plugin\views\style\Serializer. The scope of this class is to provide a new views style that displays serialized data. Consequently, it makes sense that it should only pass contextual information related to the style. Is there any other information that you believe would be helpful?

Status: Needs review » Needs work

The last submitted patch, 45: 2568413-45.patch, failed testing.

grasmash’s picture

FileSize
3.74 KB

Don't quite understand why this is failing. "PHP Fatal error: Class 'Drupal\Tests\rest\Unit\Plugin\views\style\SerializerTest' not found in /var/www/html/core/scripts/run-tests.sh on line 709".

I'm guessing that I'm making a mistake with namespacing or test naming, but I'm directly mimicking what was previously successful.

grasmash’s picture

Patch from #47 had a mistake. Patch from #45 appears to be correct. Anyone have thoughts on what's going on? I'm guessing that in the last 2 months some change to Drupal core's handling of namespaces or test naming conventions has affected this?

The last submitted patch, 45: 2568413-45.patch, failed testing.

The last submitted patch, 42: 2568413-28.patch, failed testing.

dawehner’s picture

FileSize
625 bytes

You'd need the following ...

grasmash’s picture

Status: Needs work » Needs review
FileSize
3.78 KB
damiankloip’s picture

The scope of this class is to provide a new views style that displays serialized data.

Yes. I wrote the class originally.. :)

Have you seen views? Displays, styles, rows are all interlinked.

Why not just pass the actual style plugin itself?

dawehner’s picture

Status: Needs review » Needs work

Yeah I kinda agree that it would be better to pass on the style plugin.

grasmash’s picture

Status: Needs work » Needs review
FileSize
3.78 KB

Ok, I've updated the patch. I'm not familiar with mocking classes, so hopefully the modifications to the test are what we're looking for!

Status: Needs review » Needs work

The last submitted patch, 55: 2568413-55.patch, failed testing.

dawehner’s picture

Thank you @grasmash!

Wim Leers’s picture

Title: Pass views config options to Rest Export serializer » REST views: Pass views style plugin instance to REST Export serializer
Status: Needs work » Needs review
Issue tags: -REST, -views, -rc target triage +VDC
FileSize
3.4 KB
1.15 KB

Code makes sense.

PHPUnit error:

There was 1 error:

1) Drupal\Tests\rest\Unit\Plugin\views\style\SerializerTest::testSerializerReceivesOptions
Prophecy\Exception\Call\UnexpectedCallException: Method call:
  - serialize([], "json", ["views_style_plugin" => Drupal\rest\Plugin\views\style\Serializer:000000006965302f000000017eeec142 Object (
    'usesRowPlugin' => true
    'usesGrouping' => false
    'serializer' => Double\SerializerInterface\P1:000000006965301d000000017eeec142 Object (
        'objectProphecy' => Prophecy\Prophecy\ObjectProphecy Object (*Prophecy*)
    )
    'formats' => Array (
        0 => 'json'
        1 => 'xml'
    )
    'usesOptions' => true
    'rowTokens' => Array ()
    'usesRowClass' => false
    'usesFields' => false
    'rendered_fields' => null
    'groupingTheme' => 'views_view_grouping'
    'defaultFieldLabels' => false
    'options' => Array (
        'formats' => Array (
            0 => 'xml'
            1 => 'json'
        )
    )
    'view' => Mock_ViewExecutable_d02d052b:000000006965306d000000017eeec142 Object (
        '__phpunit_invocationMocker' => null
        '__phpunit_originalObject' => null
        'storage' => null
        'built' => false
        'executed' => false
        'args' => Array ()
        'build_info' => Array ()
        'ajaxEnabled' => false
        'result' => Array ()
        'current_page' => null
        'items_per_page' => null
        'offset' => null
        'total_rows' => null
        'attachment_before' => Array ()
        'attachment_after' => Array ()
        'feedIcons' => Array ()
        'exposed_data' => Array ()
        'exposed_input' => Array ()
        'exposed_raw_input' => Array ()
        'old_view' => Array ()
        'parent_views' => Array ()
        'is_attachment' => null
        'current_display' => null
        'query' => null
        'pager' => null
        'display_handler' => null
        'displayHandlers' => null
        'style_plugin' => null
        'rowPlugin' => null
        'override_url' => null
        'override_path' => null
        'base_database' => null
        'field' => null
        'argument' => null
        'sort' => null
        'filter' => null
        'relationship' => null
        'header' => null
        'footer' => null
        'empty' => null
        'response' => null
        'request' => null
        'inited' => null
        'exposed_widgets' => null
        'preview' => null
        'get_total_rows' => null
        'build_sort' => null
        'many_to_one_tables' => null
        'dom_id' => null
        'element' => Array (
            '#attached' => Array (
                'library' => Array (
                    0 => 'views/views.module'
                )
                'drupalSettings' => Array ()
            )
            '#cache' => Array ()
        )
        'user' => null
        'showAdminLinks' => null
        'viewsData' => null
        'routeProvider' => null
        '_serviceIds' => Array ()
    )
    'displayHandler' => Mock_RestExport_2d18c76e:000000006965306e000000017eeec142 Object (
        '__phpunit_invocationMocker' => PHPUnit_Framework_MockObject_InvocationMocker:0000000069653096000000017eeec142 Object (
            'matchers' => Array (
                0 => PHPUnit_Framework_MockObject_Matcher:00000000696530ab000000017eeec142 Object (
                    'invocationMatcher' => PHPUnit_Framework_MockObject_Matcher_AnyInvokedCount:0000000069653003000000017eeec142 Object (
                        'invocations' => Array (
                            0 => PHPUnit_Framework_MockObject_Invocation_Object:0000000069653028000000017eeec142 Object (
                                'object' => Mock_RestExport_2d18c76e Object (*RECURSION*)
                                'className' => 'Drupal\rest\Plugin\views\display\RestExport'
                                'methodName' => 'getContentType'
                                'parameters' => Array ()
                            )
                        )
                    )
                    'afterMatchBuilderId' => null
                    'afterMatchBuilderIsInvoked' => false
                    'methodNameMatcher' => PHPUnit_Framework_MockObject_Matcher_MethodName:0000000069653068000000017eeec142 Object (
                        'constraint' => PHPUnit_Framework_Constraint_IsEqual:0000000069653090000000017eeec142 Object (
                            'value' => 'getContentType'
                            'delta' => 0
                            'maxDepth' => 10
                            'canonicalize' => false
                            'ignoreCase' => true
                            'lastFailure' => null
                            'exporter' => SebastianBergmann\Exporter\Exporter:0000000069653091000000017eeec142 Object ()
                        )
                    )
                    'parametersMatcher' => null
                    'stub' => PHPUnit_Framework_MockObject_Stub_Return:0000000069653092000000017eeec142 Object (
                        'value' => 'json'
                    )
                )
            )
            'builderMap' => Array ()
        )
        '__phpunit_originalObject' => null
        'usesAJAX' => false
        'usesPager' => false
        'usesMore' => false
        'usesAreas' => false
        'usesOptions' => false
        'contentType' => 'json'
        'mimeType' => null
        'renderer' => null
        'routeProvider' => null
        'state' => null
        'view' => null
        'handlers' => Array ()
        'plugins' => Array ()
        'extenders' => Array ()
        'output' => null
        'usesAttachments' => false
        'display' => null
        'options' => Array ()
        'displayHandler' => null
        'definition' => null
        'pluginId' => null
        'pluginDefinition' => null
        'configuration' => null
        'stringTranslation' => null
        '_serviceIds' => Array ()
        'dependencies' => Array ()
        'urlGenerator' => null
    )
    'definition' => Array ()
    'renderer' => null
    'pluginId' => 'dummy_serializer'
    'pluginDefinition' => Array ()
    'configuration' => Array ()
    'stringTranslation' => null
    '_serviceIds' => Array ()
)])
on Double\SerializerInterface\P1 was not expected, expected calls were:
  - serialize(exact([]), exact("json"), exact(["views_style_plugin" => Double\SerializerInterface\P1:000000006965301d000000017eeec142 Object (
    'objectProphecy' => Prophecy\Prophecy\ObjectProphecy Object (*Prophecy*)
)]))

/Users/wim.leers/Work/drupal-tres/core/modules/rest/src/Plugin/views/style/Serializer.php:138
/Users/wim.leers/Work/drupal-tres/core/modules/rest/tests/src/Unit/Plugin/views/style/SerializerTest.php:78
/Users/wim.leers/Work/drupal-tres/vendor/phpunit/phpunit/phpunit:47

Attached reroll is only fixing whitespace nitpicks, not the failures.

Status: Needs review » Needs work

The last submitted patch, 58: 2568413-58.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
3.34 KB
1.88 KB

Let's just fix it.

Wim Leers’s picture

Status: Needs review » Reviewed & tested by the community

I did not know about Argument::that(). Thanks!

Looks good.

  • catch committed b1cd732 on
    Issue #2568413 by grasmash, pwolanin, dawehner, Wim Leers, xjm, klausi:...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.1.x and cherry-picked to 8.0.x. Thanks!

  • catch committed 9067a83 on
    Issue #2568413 by grasmash, pwolanin, dawehner, Wim Leers, xjm, klausi:...
Wim Leers’s picture

Thanks to this, csv_serialization is now unblocked, yay :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.