Implement a way to order views results by referenced field delta

On my local I tried to create a config_page with a field to be used like a entity queue. The field (field_slideshow) is a entity reference that accept multiple entries and I'm using this to create a Slideshow with views.
At views, I create a reference to the config pages using the field_slideshow, and now I want to order the results as it is ordered at the config_pages field_slideshow, using the delta field. The case is that with config_pages the order using field_slideshow will use the field_slideshow reference and the result views will return n^2 results, so if I have 2 entries at field_slideshow, it will return 4 results, for 3 entries, views will show 9 items, etc.
If I do the same with a content type instead of config_pages I get the same results, but using a content type I can use the field_slideshow:delta field without using the relationship to the other content type, (I still need the relationship to filter by) and everything works as expected.

If I'm not clear enough, I can attach some screenshots.

Thanks for this great module!