This module uses ctools_cache_set() which ends up causing a session cookie to be created. That session cookie then breaks Varnish caching.

I'd suggest using session storage in the browser instead or http://drupal.org/project/session_cache integration to make a cache that can be configurable.