Smart date duration field start onchange affects all end time field

I have a content type (Basic page) and created two fields Start date and Optional Start date using Smart date range field type. I also created a field fieldset group and place both of the smart date range fields on the group. When creating a new content and I changed the Start date time it calculates the end time which is correct however it also affects the other field's end time. I figured that this is happening because I have two fieldset parents. The code we have in smart_date.js "setEndDate()" looks for the parents('fieldset'). I think we can fix this by using closest() or parents('fieldset').first()?

var wrapper = $(element).parents('fieldset')