Unable to remove Gin Toolbar using 'composer remove'

Problem/Motivation

We love the Gin admin theme so we clearly wat to keep that for backend and content editing.

For administering our removing The Admin Toolbar and Toolbar modules because we are moving to Coffee instead.

Thus, we no longer need the Gin Toolbar module but when we go to remove it using composer, we are seeing...

$ composer remove drupal/gin_toolbar
drupal/gin_toolbar is not required in your composer.json and has not been removed
./composer.json has been updated
Running composer update drupal/gin_toolbar
Loading composer repositories with package information
Dependency drupal/core is also a root requirement. Package has not been listed as an update argument, so keeping locked at old version. Use --with-all-dependencies (-W) to include root dependencies.
Updating dependencies
Nothing to modify in lock file
Writing lock file
Installing dependencies from lock file (including require-dev)
Nothing to install, update or remove

Can anyone please advise here?