Comment by fullstackchris

How can it know which tokens not to read without reading them? and llms.txt is a single file for the whole site... not the same