The logic which decides whether to display the field plugin (formatter/widget) settings edit button currently tests whether there is any settings summary text to display. If so, the summary text and the edit button are displayed.
However, if there is no summary text, the edit button will not be shown, even if settings are available for the plugin. This is a bug; the decision to show the edit-button should depend on whether settings are available.
This matters because hook_field_formatter_settings_form_alter() allows modules to add extra settings to formatters which don't already have settings. Meanwhile, the summary text for the extra settings needs to be set in hook_field_formatter_settings_summary_alter(), but there is no way to guarantee that summary text is being provided by that hook. (Likewise for the widget alter hooks.)
When a contrib module extends a formatter's settings, it doesn't always make sense to add any summary text, because when several modules extend formatter settings, the settings summaries can become overcrowded.
Steps to reproduce this bug
The problem isn't obvious when looking just at core; most (all?) of the core field formatters which have settings produce a summary message in all cases.
To reproduce this, you'll need a module that provides extra field plugin settings. I suggest using Field formatter class, which has a D8 version. It provides an extra setting to all field formatters, to add HTML classes to the display output. If an extra class is specified, it will be indicated in the settings summary message. (I've deliberately disabled the "no class" summary message in the latest 8.x-1.x-dev.)
Refactor Drupal\field_ui\DisplayOverviewBase::buildFieldRow() so that the edit-button is shown whenever there are plugin settings available.
PASSED: [[SimpleTest]]: [MySQL] 59,242 pass(es). View
FAILED: [[SimpleTest]]: [MySQL] 59,213 pass(es), 1 fail(s), and 0 exception(s). View
FAILED: [[SimpleTest]]: [MySQL] 59,158 pass(es), 1 fail(s), and 0 exception(s). View
FAILED: [[SimpleTest]]: [MySQL] 59,341 pass(es), 2 fail(s), and 0 exception(s). View
FAILED: [[SimpleTest]]: [MySQL] 59,233 pass(es), 2 fail(s), and 0 exception(s). View