Cannot create payments and payment methods via REST

I can reproduce the issue the following way.
1. Enable the restui module.
2. Enable POST for payment entities on the Rest UI.
3. Run the following in devel/php:

$serialized_entity = json_encode([

'type' => [['value' => 'payment_default']],
 'payment_gateway' => [
    [
      "target_id" => 'manual',
      "target_type"=> "commerce_payment_gateway",
  ]],
  "amount" => [
	[
	  "number" => "9.990000",
          "currency_code"=> "USD",
	],
  ],
  "order_id" =>  [
	[
	  "target_id"=> 5,
	  "target_type"=> "commerce_order",
	],
  ],
]);

$response = \Drupal::httpClient()
  ->post('http://drupal8.loc/entity/commerce_payment?_format=json', [
    'body' => $serialized_entity,
    'headers' => [
      'Content-Type' => 'application/json',
    ],
  ]);

The result is a WSOD with a

[Mon Sep 10 15:36:40.811605 2018] [fcgid:warn] [pid 8802] [client 127.0.0.1:59404] mod_fcgid: stderr: Uncaught PHP Exception Drupal\\Core\\Entity\\EntityStorageException: "Missing "payment_gateway" property when creating a payment." at /modules/contrib/commerce/modules/payment/src/PaymentStorage.php line 41

in my logs.

This is caused by the following pattern used in Drupal\commerce_payment\PaymentStorage::doCreate():

protected function doCreate(array $values) {
  if (empty($values['payment_gateway'])) {
    throw new EntityStorageException('Missing "payment_gateway" property when creating a payment.');
  }

When creating the payment via REST however, $values only contains the payment bundle. The stack is:

modules/contrib/commerce/modules/payment/src/PaymentStorage.php.Drupal\commerce_payment\PaymentStorage->doCreate(): lineno 41	
core/lib/Drupal/Core/Entity/EntityStorageBase.php.Drupal\commerce_payment\PaymentStorage->create(): lineno 184	
core/modules/serialization/src/Normalizer/EntityNormalizer.php.Drupal\serialization\Normalizer\ContentEntityNormalizer->denormalize(): lineno 56	
vendor/symfony/serializer/Serializer.php.Symfony\Component\Serializer\Serializer->denormalize(): lineno 182	

Where denormalize() takes care of passing only the bundle to doCreate().

Core never uses the above validation pattern. Since Drupal\commerce_payment\Entity::baseFieldDefinitions() sets the payment gateway field required anyway I wonder if we could get rid of these lines altogether. With the attached patch the below POST request creates a payment:

$serialized_entity = json_encode([
  'type' => [['value' => 'payment_default']],
  'payment_gateway' => [
    [
      "target_id" => 'manual',
      "target_type"=> "commerce_payment_gateway",
    ]
  ],
  'payment_gateway_mode' => [['value' => 'test']],
  "amount" => [
    [
      'number' => "9.990000",
      'currency_code'=> "USD",
    ],
  ],
  'order_id' =>  [[
    "target_id"=> 5,
    "target_type"=> "commerce_order",
  ]],
]);

$response = \Drupal::httpClient()
  ->post('http://drupal8.loc/entity/commerce_payment?_format=json', [
    'body' => $serialized_entity,
    'headers' => [
      'Content-Type' => 'application/json',
    ],
  ]);

Core never uses the above validation pattern. Since Drupal\commerce_payment\Entity::baseFieldDefinitions() sets the payment gateway field required anyway I wonder if we could get rid of these lines altogether. With the attached patch the below POST request creates a payment:

That is because core has very loose validations on the entity. I believe REST/JSON API invoke entity validations, but outside of that scope nothing is guaranteed. You can save entities programmatically without populating required fields and not receive an error.

@mglaman

You can save entities programmatically without populating required fields and not receive an error.

With the patch above applied I cannot create a payment with this code:

use Drupal\commerce_payment\Entity\Payment;
use Drupal\commerce_price\Price;

$values = [
'type' => 'payment_default',
'payment_gateway_mode' => 'test',
'order_id' => ['target_id' => 5],
  "amount" => 	new Price("9.990000", "USD"),
];
Payment::create($values)->save();

I get a WSOD with this log entry:

[Tue Sep 11 14:52:35.236777 2018] [fcgid:warn] [pid 25289] [client 127.0.0.1:51440] mod_fcgid: stderr: Uncaught PHP Exception Drupal\\Core\\Entity\\EntityStorageException: "Required payment field "payment_gateway" is empty." at /core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php line 829, referer: http://drupal8.loc/devel/php

Seems to me that core is doing validation for required base fields. Am I missing something?

Not core. That's our code:

  public function preSave(EntityStorageInterface $storage) {
    parent::preSave($storage);

    $payment_gateway = $this->getPaymentGateway();
    if (!$payment_gateway) {
      throw new EntityMalformedException(sprintf('Required payment field "payment_gateway" is empty.'));
    }

PaymentStorage::doCreate() requires the payment gateway because that's used to determine the "type".
I guess we could make payment_gateway on create optional if type was specified, but the DX is less ideal than specifying just the gateway.

This was also tackled the wrong way. We need a test in the payment module which enables the rest endpoint and replicates the failure.

I'm working on adding REST tests for commerce_payment the way core does it (see e.g. core/modules/node/tests/src/Functional/Rest). That will cover all drupal core rest methods, formats and authentications.

The ONLY_TESTS patch should fail, the other one should pass.

The tests adhere to the current situation of PATCHing commerce_payment entities is not supported at all.

Adding a base_path() to the order_id url.

Remove slash.

The patch resolve the error.

However if the payment gateway is some onsite creditcard payment, it is not processing the actual payment gateway (even the payment method ID is provided), it just create an entity record.

same isssue for me when i am using jsonapi. but i am using 2.12 commerce. So i cant use this patch right?

The patch in #17 was working great before but with 2.12 it's not working anymore. It still can be applied, but the error persists.

I could successfully create commerce_payment entities after applying patch #17, but not commerce_payment_method entities via POST to /entity/commerce_payment_method.

Applying the same fix to PaymentMethodStorage.php file seems to be working.

Expanding title, since both commerce_payment and commerce_payment_method entity types are affected.

We don't need to enforce payment_gateway presence in doCreate() since we already do it in the entity preSave() methods.

I don't think we want to commit REST tests at this point. I want to wait until we have an actual handcrafted REST API (probably living in commerce_rest), the one that core provides is just a clunky stopgap.

Rerolled patch provided.

Committed. Thanks, everyone!