filter out what's in a subqueue taxonomy

I'm having difficulty creating a View (Views 3) in Drupal 7 to filter out what's in a nodequeue taxonomy.

Scenario.
Nodequeue called Course Subjects which is related to Course Subjects Taxonomy. There are nodes that are in the subqueue. Having trouble to remove the node(s) that are in that specific subqueue.

Example.
In Taxonomy Mathematics
A
B
C
D

In Nodequeue Subqueue Mathematics
A
D

Need a view that removes A and D from the list since it's in the subqueue.

I tried using the filter nodequeue false but their is missing data, so no longer have that as a filter. It appears that if a node belongs to another subqueue it doesn't appear in the view when nodequeue false is used.

I added subqueue reference not equal to whatever the term id is and subqueue reference is null and that works for individual terms, not as a whole....would need many displays to accomplish this. Is there a more efficient way to accomplish this?

Thanks