In class views_field_view_handler_field_view, pre_render() method, there is the following piece of code.

 // Find the arguments on the child view that we're going to need if the
        // arguments have been overridden.
        foreach ($child_view->display['default']->display_options['arguments'] as $argument_name => $argument_value) {
          if (isset($child_view->display[$child_view_display]->display_options['arguments'][$argument_name])) {
            $configured_arguments[$argument_name] = $child_view->display[$child_view_display]->display_options['arguments'][$argument_name];
          }
          else {
            $configured_arguments[$argument_name] = $child_view->display['default']->display_options['arguments'][$argument_name];
          }
        }

You can see that it iterates over the default display of the view, to find arguments, and then does a check if its overridden in a different display, use that displays's arguments.

The problem here is that if the argument is present in the different display, but not in the default one, query aggregation will not use any arguments, and will thus fail.
I believe the line should be changed to 

foreach ($child_view->display[$child_view_display]->display_options['arguments'] as $argument_name => $argument_value) {

and also do an additional check to see if the argument is present in the default display as well.