Using a VM for Docker gives you the worst of both worlds.
You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc).
On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in.
You get all the downsides of a VM (persistent wasted RAM/disk space, awful disk IO performance, etc) AND all the downsides of containers (difficult to introspect when your minimal image has no shell, janky networking, etc).
On recent M1/M2 Macs you take the additional pain (and thus performance hit) of translation to x86 (because that's almost certainly what your containers are in.