Form with AJAX 'add/remove' buttons

You need to know "persons{$name}" key of a item you want to delete.
You can do this with calling:
$form_state->getTriggeringElement()
After that, you be able to remove this item from the array:
unset($form['site_credits']['KEY_TO_REMOVE']).
Take a look this example:
https://gist.github.com/alexdesignworks/94bac012e7d679051bc0#gistcomment...

I have found a solution. I don't know if it's the best but it will do it for me. I modified the example code from the example module. I use form state vars for each new row (1 for active row, 0 for removed row).

I use  $form_state->getTriggeringElement()['#name'] to know which remove buttons is clicked. The name of the button is equal to the row count. So after the button is clicked the corresponding var is set to 0 and when the form is rebuilt it will not display the removed text field.

See the attached code.

 public function buildForm(array $form, FormStateInterface $form_state) {
        $form['description'] = [
            '#type' => 'item',
            '#markup' => $this->t('This example shows an add-more and a remove button.'),
        ];

        // Gather the number of rows in the form already.
        $row_count = $form_state->get('row_count');
        // We have to ensure that there is at least one row field.
        if ($row_count === NULL) {
            $form_state->set('row_count', 1);
            $form_state->set("row_0_active", 1);
            $row_count = 1;
        }

        $form['#tree'] = TRUE;
        $form['names_fieldset'] = [
            '#type' => 'fieldset',
            '#title' => $this->t('People coming to picnic'),
            '#prefix' => '<div id="names-fieldset-wrapper">',
            '#suffix' => '</div>',
        ];

        for ($row_no = 0; $row_no < $row_count; $row_no++) {

            $is_active_row = $form_state->get("row_" . $row_no . "_active");

            //We check if the row is active
            if ($is_active_row) {
                $form['names_fieldset'][$row_no]['name'] = [
                    '#type' => 'textfield',
                    '#title' => $this->t('Name'),
                ];

                $form['names_fieldset'][$row_no]['remove_name'] = [
                    '#type' => 'submit',
                    '#name' => $row_no,
                    '#value' => $this->t('Remove one'),
                    '#submit' => ['::removeCallback'],
                    '#ajax' => [
                        'callback' => '::addmoreCallback',
                        'wrapper' => 'names-fieldset-wrapper',
                    ],
                ];
            }
        }

        $form['names_fieldset']['actions'] = [
            '#type' => 'actions',
        ];
        $form['names_fieldset']['actions']['add_name'] = [
            '#type' => 'submit',
            '#value' => $this->t('Add one more'),
            '#submit' => ['::addOne'],
            '#ajax' => [
                'callback' => '::addmoreCallback',
                'wrapper' => 'names-fieldset-wrapper',
            ],
        ];

        $form['actions']['submit'] = [
            '#type' => 'submit',
            '#value' => $this->t('Submit'),
        ];

        return $form;
    }

    /**
     * Callback for both ajax-enabled buttons.
     *
     * Selects and returns the fieldset with the names in it.
     */
    public function addmoreCallback(array &$form, FormStateInterface $form_state) {
        return $form['names_fieldset'];
    }

    /**
     * Submit handler for the "add-one-more" button.
     *
     * Increments the max counter and causes a rebuild.
     */
    public function addOne(array &$form, FormStateInterface $form_state) {
        $cur_rows = $form_state->get('row_count');
        $rows = $cur_rows + 1;
        $form_state->set('row_count', $rows);
        $this->messenger()->addMessage($rows);
        $form_state->set("row_" . $cur_rows . "_active", 1);
        // Since our buildForm() method relies on the value of 'row_count' to
        // generate 'name' form elements, we have to tell the form to rebuild. If we
        // don't do this, the form builder will not call buildForm().
        $form_state->setRebuild();
    }

    /**
     * Submit handler for the "remove one" button.
     *
     * Decrements the max counter and causes a form rebuild.
     */
    public function removeCallback(array &$form, FormStateInterface $form_state) {
        $button_clicked = $form_state->getTriggeringElement()['#name'];
        $form_state->set("row_" . $button_clicked . "_active", 0);
        $this->messenger()->addMessage($button_clicked);
        // Since our buildForm() method relies on the value of 'row_count' to
        // generate 'name' form elements, we have to tell the form to rebuild. If we
        // don't do this, the form builder will not call buildForm().
        $form_state->setRebuild();
    }

    /**
     * Final submit handler.
     *
     * Reports what values were finally set.
     */
    public function submitForm(array &$form, FormStateInterface $form_state) {
        $row_count = $form_state->get('row_count');

        for ($row_no = 0; $row_no < $row_count; $row_no++) {
            $is_active_row = $form_state->get("row_" . $row_no . "_active");
            if ($is_active_row) {

                $values .= $form_state->getValue(['names_fieldset', $row_no, 'name']).", ";

                }
        }
        $this->messenger()->addMessage($values);
    }

Hope that this helps someone.