FieldBlock ignores cache metadata set in field formatters

The FieldBlock block plugin essentially ignores any cache metadata that is set in the render array returned from a field formatter. Here's the build method of the block:

  /**
   * {@inheritdoc}
   */
  public function build() {
    $display_settings = $this->getConfiguration()['formatter'];
    $entity = $this->getEntity();

    // If this entity was constructed with sample data, return nothing to force
    // a placeholder to render instead. The sample data is rarely user friendly
    // can create issues in field formatters and blocks that assume the entity
    // has legit data.
    if ($entity->has_sample_data) {
      return [];
    }

    try {
      $build = $entity->get($this->fieldName)->view($display_settings);
    }
    // @todo Remove in https://www.drupal.org/project/drupal/issues/2367555.
    catch (EnforcedResponseException $e) {
      throw $e;
    }
    catch (\Exception $e) {
      $build = [];
      $this->logger->warning('The field "%field" failed to render with the error of "%error".', ['%field' => $this->fieldName, '%error' => $e->getMessage()]);
    }
    CacheableMetadata::createFromObject($this)->applyTo($build);
    return $build;
  }

The CacheableMetadata::createFromObject($this)->applyTo($build); is troublesome because it just overwrites the cache metadata that may already exist on the $build render array.