In trying to understand PluginManager, I found that it's essentially made up of three responsibilities, Discovery, Factory, and Mapper.
The Mapper confused me a bit. I know what it means to map something, but it doesn't occur to me that you'd need to do this often. To understand it better I went looking for implementations, but ConfigMapper is the only one, and that doesn't even appear to be used.
PluginManagerBase::getInstance passes straight through to $this->mapper, but it looks like every class that inherits from PluginManagerBase overrides it.
It seems to me that if we don't have a use case yet, and it's getting in the way, it's probably best to just get rid of Mapper. Even if we did have a use case, this seems more appropriate for a method on PluginManager than it's own class.
Considering that Plugins are not the simplest concept to begin with, removing the concept of the Mapper is a win.