CANCEL-Button on 'add field' > 'configure' still adds the field

applies to all elements like filter, header etc. too, but i use filter here.
Yes, there's the REMOVE button next to CANCEL. But still, i guess this could be more intuitive.

Problem:
Logically the first step is 'add field', the second one is 'configure field'. But the presentation turns the whole into a wizard, where cancel normally cancels the whole procedure.
We have the option to add several fields, so optionally go through the 'wizard' multiple times, which complicates things (Maybe that's part of the reason why it is implemented the way it is?).
Since in the step before, CANCEL does so without adding the field, there's the expectation to behave the same in all steps.

Solution?
I think that from logic, CANCEL should behave the same on 'config' as on 'add field', and thus revert the addition of the field. But maybe that's unreasonably complicated to implement?
So maybe the Button could be named differently?
CANCEL CONFIG or SKIP maybe, since it skips the configutation. But how many people want to skip config and keep the field?