Public files 403

I using flysystem s3 on the majority of my site but I'd also like to be able to use drupals default public file option but any files uploaded using this option can't be accessed due to a 403 error. I know there is the local option which is fine for normal images fields, but I would like to use the Gutenberg Module which doesn't have the option to use flysystem for image uploads and it seems like this module intercepts all images.

I've even tried setting the default download method to Public local files served by the webserver but that doesn't resolve the issue. Do have any suggestions as to how I can get around this issue.