Listing top level terms in views including those with or without children

My vocabulary looks like this

Level 1a
-Child a
-Child b
-Child c
Level 1b
-Child a
-Child b
-Child c
Level 1c
Level 1d
-Child a
-Child b
-Child c
Level 1e

I have managed to display parents and children terms using Taxonomy term: Parent term and filter on (Parent) Taxonomy term: Parent term (not empty). But terms without children on the same level with "parent" are not displayed.

To get the children I used filter “(Parent) Taxonomy term: Parent term (not empty)”. If I set the same filter to “is empty”, I get all level 1 terms including those terms without children just like I want it. So right now it seems like an either-or situation. I can only get the top terms or the children terms but not both.

How do I get the top level terms including those without children and corresponding child terms for those with children?