The current version of the "All galleries" page is special menu callback that alters the normal taxonomy_terms page callback. The problem is, all kinds of other modules alter this menu hook too. For example when you activate the i18n taxonomy translation module the page breaks and resets to the default state.
Why not create a default view delivered with the media_gallery module? Most of the features for replicating the current layout are already in place. Maybe the external settings from the configuration ui might work with the view.
What's missing right now are some formatters and fields, that allow you to add the image count and gallery description as seen right now (and other informations?). This should be possible by exposing those informations as "pseudo" fields and giving them their own formatters. A custom formatter would be especially important for the gallery thumbnail and it's frame and would open up the options to choose different image_styles (sizes, frames) for your galleries.
As a side effect having these informations available as fields one could add their own formatters that change the basic behaviors like choosing a certain image for the gallery thumbnail (That might be a pseudo field too).
Maybe that could work for the normal gallery page as well opening up more opportunities to alter the design and information of individual galleries by using an argument.
So what do you think? Where do you see pain points in this solution? I still have a problem with i18n right now, but maybe Views internationalization will fill the gap here.
In the end, I don't think that this view is too hard to create in a joint effort.