Widget disappear using media (file entities) and multiple form

Problem/Motivation

I ran into this issue this morning and I can't really find out what is exactly causing this.
So I have file entities and media installed and the file types all have a taxonomy attached to them. It's a fairly big vocabulary with 5 levels of depth at the deepest.
I have set HS to allow new terms on the last 2 levels.

If I process one file at a time, I have no issues using HS as the widget for the taxonomy.

However the problem arises when doing a multiple edit at once and the forms are merged into one.

One visible issue is that the new item button (from the last widget) are being displayed with the global Save/Cancel button (see image attached)

The other issue is that whenever I go to change a value at any level, the widget simply disappears after (attempting?) the call back.

The logs are not silent though and I see quite a few errors, all of them seems to be consequences of the first ones:

    Notice: Undefined index: #form_id in hierarchical_select_ajax() (line 327 of D:\Inetpub\LocalUser\mysite\sites\all\modules\hierarchical_select\hierarchical_select.module).
    Notice: Undefined index: #form_id in ajax_get_form() (line 362 of D:\Inetpub\LocalUser\mysite\includes\ajax.inc).
    Notice: Undefined index: #form_id in ajax_form_callback() (line 386 of D:\Inetpub\LocalUser\mysite\includes\ajax.inc).
    Notice: Undefined index: #parents in form_builder() (line 1861 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Warning: implode(): Invalid arguments passed in form_builder() (line 1861 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: #array_parents in form_builder() (line 1914 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: process_input in _form_builder_handle_input_element() (line 2076 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: complete form in form_builder() (line 1871 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: #array_parents in form_builder() (line 1914 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: process_input in _form_builder_handle_input_element() (line 2076 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: complete form in form_builder() (line 1871 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: #array_parents in form_builder() (line 1914 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: process_input in _form_builder_handle_input_element() (line 2076 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: complete form in form_builder() (line 1871 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: #array_parents in form_builder() (line 1914 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: process_input in _form_builder_handle_input_element() (line 2076 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: complete form in form_builder() (line 1871 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: #array_parents in form_builder() (line 1914 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: process_input in _form_builder_handle_input_element() (line 2076 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: complete form in form_builder() (line 1871 of D:\Inetpub\LocalUser\mysite\includes\form.inc).
    Notice: Undefined index: process_input in drupal_process_form() (line 889 of D:\Inetpub\LocalUser\mysite\includes\form.inc).

Any ideas what's causing the issue and how this can be fixed?

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes