AJAX upload prevents form submit

I'm using plupload on a custom form and if the user doesn't click start upload before submitting the form, the form rebuilds without running through the submit handler. Spitting out the form state shows that it tried to submit using "image_upload_button". I've also included a screenshot of the entire triggering_element array from the form state.