alcinos

> We've just only started RL training LLMs

That's just factually wrong. Even the original chatGPT model (based on gpt3.5, released in 2022) was trained with RL (specifically RLHF).

Well it's deterministic once you know the random seed, which is stored in the replay file. An agent doesn't know the seed, hence cannot predict the exact outcome of its actions, only a probability over the outcomes. So, from the agent's perspective, the game is indeed random.

I completely agree that serious researchers should review definitions and results they are basing their findings upon. But then, I'd find much more reliable an editable reference paper corrected and improved by a significant number of people working in the field, rather than an old-style paper, published decades ago, after a (botched ?) review by a couple of anonymous reviewers, and not revised since then.

This would address also a little bit the problem of notations: if everybody agree and work on the same piece of work, they are likely to adopt the same notations, providing a nice coherence to the user.

A wiki style database has many advantages when it comes to organizing and searching information. In my opinion, though, the editing process should be closer to GitHub's pull request (as advocated by the article), to ensure that everything is properly reviewed before publication.

Finally, a publication scheme like the one I described also address a recurrent issue with traditional citation-based papers: citations are one-sided. It is easy to see which papers one article depends upon, but the converse is hard (unless using specific tools, at least). With collaborative editing and wikipedia-style links between articles, the reader is immediately aware of the latest findings in the field, which simplifies tremendously the bibliographic research.

To me, the main problem with papers in their current shape is that they are required to be more or less self-contained. When one wants to state a result that improves a little bit the knowledge in a well established field, he has to waste time and space stating the definitions and preliminary results necessary to understand his result. This is counterproductive both for the author and the reader interested only in the small new bit of information.

If papers were collaborative, one could simply propose his improvements directly where they fit in the reference paper, without having to write one from scratch, and readers would be immediately aware of these follow-up results, without having to search in dozens of papers.