"Attention is all you need" was published only 5 years ago in 2017. BERT in 2018, GPT-3 in 2020. Now DALL-E, PaLM, LaMDA, etc. The pace of AI progress is frighteningly fast. I really thin it's plausible that by 2030, AI is writing and debugging most code, and engineers basically become spec writers who manipulate prompts, generate code, and verify. Essentially, project managers whose "team" is a bunch of LLMs.
It really doesn't seem we've hit diminishing returns on LLM model size yet, and that's not accounting for the multi-input types where video, text, audio, and more are being fused together.
It really doesn't seem we've hit diminishing returns on LLM model size yet, and that's not accounting for the multi-input types where video, text, audio, and more are being fused together.