I just committed a feature to CTools to allow the modal to trigger an event when it is closed, which should allow wysiwyg.module to respond to this and properly detach. The commit is here: http://drupal.org/cvs?commit=439216
A patch for wysiwyg is forthcoming, but I'm not sure if it's right. It works for my tests, but I am concerned that it will detach all wysiwyg editors rather than just the ones that were in the modal, so I'm not sure this is right. Still, I think this should be enough for someone who really understands wysiwyg to fix this up properly.