Contextual nodequeue edit links for views doesn't show if views block display with nodequeue relationship is added as panel pane

Contextual nodequeue edit links for views doesn't show if views block display with nodequeue relationship is added as panel pane. If view is enabled as panel pane then condition check if (!empty($element['#element']['#contextual_links']['views_ui'])) {} become false and nodequeue contextual edit links are not being added to views.

I also tried the patch from here https://www.drupal.org/node/1920700 about Refactor nodequeue_extras_contextual_links_view_alter() but that doesn't help.