Aggregation settings are available for some items that can't be aggregated. This is confusing.
For instance, it's possible to use a computed property of an entity as a field value. It can't be aggregated (since that happens in SQL), but the aggregation settings are still there. The settings should be disabled for such fields. There should probably be a value for the field passed via hook_views_data() that disables aggregation settings for such fields.
Also, there there are aggregation settings for filters and sorts, which is strange. Does it make sense to filter on grouped values? Would a filter be applied before an AVG()? If the filtering happens first, the aggregation is irrelevant and the settings should be removed. If aggregation is relevant for some filters, but not others, the options should appear only for the relevant ones.