Was noticing an excessive number of calls to drupal_add_css() and discovered that the logic in views_plugin_cache::gather_headers() is a bit broken. The array it's checking against to get the diffs is cleared right before it, which results in the entire $css and $js array being cached.
Attached patch simplifies the logic and appears to work well in my testing.
PASSED: [[SimpleTest]]: [MySQL] 1,408 pass(es). View
FAILED: [[SimpleTest]]: [MySQL] 1,403 pass(es), 1 fail(s), and 0 exception(s). View
PASSED: [[SimpleTest]]: [MySQL] 1,404 pass(es). View
PASSED: [[SimpleTest]]: [MySQL] 1,398 pass(es). View