Does not work properly when used with Field Collection 'Remove' functionality

If you have field collection with inline entity reference forms set up and you click Remove on a field collection item you get the wrong items referenced.

Foe example if I have a field collection that has a text field "Title" and a entity reference "Ref" and I have the following content:

FC1
Title: title1
Ref: node1

FC2
Title: title2
Ref: node2

FC3
Title: title3
Ref: node3

If I click Remove on the field collection item to remove FC2 from the middle I get the following result

FC1
Title: title1
Ref: node1

FC3
Title: title3
Ref: node2

As you can see FC3 became the second item, but inherited the reference from FC2 and lost it's original one.

This is not an issue with the Field Collection module since other widget for this field type work correctly.