If you uncheck a book block at admin/content/book/blocks and that book block has been configured to appear somewhere, it will not disappear until you visit the block administration page.
Not ideal.
Am looking into it.