Comment by sirdarckcat

You can make a new instruction (or repurpose an existing one) that accesses physical memory bypassing the page walk, which would be faster. You can also make instructions that bypasses some checks (like privilege checks) and squeeze some tiny performance. Note this would introduce security issues though, so you could only use it on trusted software.