checkFiledAccess Permissions hindering view of Users Email

I've created a view that allows some users to see some emails of other users. However the UserAccessControlHandler::checkFieldAccess has code like this:

    // Administrative users are allowed to edit and view all fields.
    if (!in_array($field_definition->getName(), $explicit_check_fields) && $account->hasPermission('administer users')) {
      return AccessResult::allowed()->cachePerPermissions();
    }

and

      case 'mail':
        // Allow view access to own mail address and other personalization
        // settings.
        if ($operation == 'view') {
          return $is_own_account ? AccessResult::allowed()->cachePerUser() : AccessResult::forbidden();
        }
        // Anyone that can edit the user can also edit this field.
        return AccessResult::allowed()->cachePerPermissions();

which basically only allows users with "administer users" or if they are that user access to view the email.

However in UserViewsData it has help text as follows:
'Email address for a given user. This field is normally not shown to users, so be cautious when using it.'

The help text is misleading because it doesn't take the above conditions into consideration. We either need to get rid of the email condition (which I doubt is going to happen) or create a custom views field which bypasses checkFieldAccess. Are there any other recommended solutions out there?