I have a large amount of data, one type of content has over 80k nodes and is a node reference for other content types. When conditional fields is enabled is pulls in all of that data and runs the system out of memory (over 1G at times) when trying to modify field settings or even add a new field. As soon as I disable conditional fields the memory issue clears up.
The off-the-top-of-my-head solution would be to use ajax to load the conditional fields on the fly if they're required. No need to load all that data if it's not being used, and no need to load every field if it's not being used.