Comment by jabberthemutt

Or include the position and orientation of the images' cameras and do a traditional photogrammetric bundle adjustment.