The current version of Media allows for title and alt attributes but they are not propagated to the image fields. Galleria relies on them to display an image caption so at the moment "Media + Galleria = no captions". A good way around it will be to use tokens to derive the captions. Colorbox for example offers an option to specify where to get the image caption from and one of the choices is an arbitrary token.
Does Galleria offer something similar? The goal is to get the caption from a Media file attribute/field.