Meshes are actually extremely fast for querying distance to a surface. Even in scenes with lots of geometry, using an acceleration structure like a BVH can be extremely fast. You can even do ray tracing in real time on a GPU now (at least for direct lighting, certainly not for global illumination).
By contrast, SDFs require you to do lots of expensive lookups as you slowly march along a ray until you happen to bump into a surface.
By contrast, SDFs require you to do lots of expensive lookups as you slowly march along a ray until you happen to bump into a surface.