Term fields can be set translatable but access control doesn't correspond to them

Term fields can be set translatable and they can be edited and viewed fine, but in effect the last edit seems to set the access control for all translations.

How to reproduce:

• Drupal 8.4.4
• Enable all the multilingual modules (config_translation, content_translation, locale, language)
• Add a Language
• Enable a content type to be able to be translated
• Create a vocabulary
  
  Name: Access control
  Vocabulary language: Not specified
  TERMS LANGUAGE
  Default language: Finnish
  Show language selector on create and edit pages: yes
  Enable translation: yes <-- important, this means that you should be able to select different terms for translated nodes
  
  
• Create appropriate terms, and translate them, for restricting access for anonymous users
• Add the term field to the node type and select "Users may translate this field" <-- important
• Create a node and select terms to make it access denied from anonymous users
• Translate the node and select terms to make it accessible to anonymous users

Expected behaviour: anonymous can not access the original but can access the translated node
Current behaviour: anonymous can access both nodes