Xfree86 never did anything different to Xorg here. If you boot via CSM on a modern PC (which is really not something you should do, but, well) you should get Xorg running with the VBE backend using x86emu to execute the video BIOS, and if you boot via EFI you should get modesetting running on top of efifb using whatever mode your firmware and bootloader left you in.
But note that this is actually easier for 16 or 32 bit operating systems! Setting VESA modes involves making a real mode 16 bit call (there's nominally a 32 bit entry point for VESA but it was specced late in the standard's life and basically nobody implements a working version), and once you're running in 64-bit mode you've lost the ability to do vm86 so calling 16 bit code from userland becomes impossible. This is why x86emu is required (basically we read the video BIOS code and then run it under an x86 emulator), and it's not always perfect.
But note that this is actually easier for 16 or 32 bit operating systems! Setting VESA modes involves making a real mode 16 bit call (there's nominally a 32 bit entry point for VESA but it was specced late in the standard's life and basically nobody implements a working version), and once you're running in 64-bit mode you've lost the ability to do vm86 so calling 16 bit code from userland becomes impossible. This is why x86emu is required (basically we read the video BIOS code and then run it under an x86 emulator), and it's not always perfect.