Model training seems to me to be much closer to simulating the evolution of the human mind starting from single cell bacteria, rather than the development of the mind of a baby up to a fully functional human. If so, then sensory inputs and interaction with the physical through them were absolutely a crucial part of how minds evolved, so I find your approach a priori very unlikely to have a chance at success.
To be clear, my reasoning is that this is the only plausible explanation for the extreme difference in how much data an individual human needs to learn language, and how much data an LMM needs to reach its level of simulation. Humanity collectively probably needed similar amounts of data as LLMs do to get here, but it was spread across a billion years of evolution from simple animals to Homo Sapiens.
To be clear, my reasoning is that this is the only plausible explanation for the extreme difference in how much data an individual human needs to learn language, and how much data an LMM needs to reach its level of simulation. Humanity collectively probably needed similar amounts of data as LLMs do to get here, but it was spread across a billion years of evolution from simple animals to Homo Sapiens.