Place holder image shows up when no video uploaded

I've added a video_upload (filefield) field to a content type, but don't always want to add a video. When I don't upload any videos and create a node, it displays the generic video holder graphic. Is it not possible for video_upload to take a back seat? I don't want it to add it's thumbnail when it's not even being used for a node. It also spits out the image in any view where the video field is being called on.