Local Task Bar Disappears after Unsetting One Tab

Hoping someone can point me in the right direction.  I need to remove a tab from the Local Tasks bar. This tab is created via a custom module. Currently I'm using hook_menu_local_tasks_alter() to remove the tab from a user with a specific role.  Here's the weird part: if I go to unset the tab without checking for the users role, the tab disappears and the rest of the tabs remain.  When I go to check if the user's role is in one of the allowed roles for this tab and it returns false, the ENTIRE tab disappears.  Here's my code:

function content_usage_menu_local_tasks_alter(&$data, $route_name) {
    $allowed_roles = ["administrator", "author", "editor", "admin"];
    $roles = \Drupal::currentUser()->getRoles();
    if (empty(array_intersect($allowed_roles, $roles))) {
        unset($data['tabs'][0]['node_usage.form']);
    } 
}

Any ideas why this would be happening?  Am I missing something?

Thanks.