You could also partially prevent this using vpc endpoints for s3. You can set the bucket policy to only allow if the connection is via the endpoint (vpc) so the bucket's no longer internet accessible. Vpc endpoints also reduce traffic charges since you pay for internal AWS bandwidth instead of internet bandwidth.
I'm kind of surprised AWS doesn't just enable VPC endpoints for all services by default. Having to leave the network to talk to services seems like overkill for most things.
I'm kind of surprised AWS doesn't just enable VPC endpoints for all services by default. Having to leave the network to talk to services seems like overkill for most things.