Widget doesn't render for setting default value

I believe the issue is with the canBuildForm function in the InlineEntityFormBase.php.

This needs to be adjusted for the InlineEntityFormComplex if it allows the use of existing entity references.

I was able to add the canBuildForm function to the complex widget, and add the check if allowed to use existing nodes.

protected function canBuildForm(FormStateInterface $form_state) {
    if ($this->isDefaultValueWidget($form_state) && !$this->getSetting('allow_existing')) { // check if allowed to use existing
      return FALSE;
    }

    if (!$this->inlineFormHandler) {
      return FALSE;
    }

    return TRUE;
}

The only problem now is the value won't save. The widget will build now. I just need to be able to save the default value selected.

The saving problem is fixed by adjusting the extractFormValues function to check if existing nodes are allowed.

if ($this->isDefaultValueWidget($form_state) && !$this->getSetting('allow_existing')) {
     $items->filterEmptyItems();
     return;
}

Here is the patch of what I stated above.

@neeravbm, I think that my problem was misunderstood. I am able to get the field to render on the normal form (node/add/test). The problem is that the field does not render on the field settings form when setting the default values (admin/structure/types/manage/test/fields/node.test.field_media).

Here is a screen shot of the widget settings (I have matched the same as you):

This is a screen shot of the field settings form before the patch (notice the details element labeled 'DEFAULT VALUE'):

This is a screen shot of the same page after applying the patch (see how the field widget is now nested in the 'DEFAULT VALUE' detail):