NID as a contextual filter doesn't work on some nodes

Hello,

I have a view where the NID is getting passed as a contextual filter. What I do on that view is I get the parent and child terms of a node to view them. So I have a relationship "Taxonomy term: Content using Categories". So when the NID is passed as a contextual filter I get the parent and child taxonomy terms the node has.

I have noticed though that on some of my nodes that doesn't work and I can't figure out why it behaves like that. So while playing with the preview at bottom of my view I can clearly see that some NID's just don't work and no taxonomies are getting pulled. I double and triple checked that a corresponding node does actually has taxonomy terms assigned to it.

More checking got me to a conclusion that some taxonomy terms pages (I am using views for that), the taxonomy terms aren't showing at all...so I guess this is an issue with my taxonomy terms and not the nodes itself?

Thanks

P.S: I am working with Views Field View but I think is irrelevant at this case because I am replicating the issue on the view itself through the preview window.

Thanks