Comment Comment by lordofgibbons > but we have reached the limit of AI scaling with conventional methodsWe've just only started RL training LLMs. So far, RL has not used more than 10-20% of the existing pre-training compute budget. There's a lot of scaling left in RL training yet. parent lordofgibbons 1 year ago
We've just only started RL training LLMs. So far, RL has not used more than 10-20% of the existing pre-training compute budget. There's a lot of scaling left in RL training yet.