Dorialexander

hi, so Pleias co-founder here.

Common Corpus is commonly used now in pretraining, including by close labs, but rarely as the only source (which would be the actual ethical commitment).

Like most people training efficient models we move toward synthetic pretraining, but still maintaining our committment for data research and releasability. Lead current project is SYNTH, for now based on Wikipedia but we'll generalize with seeds from Common Corpus.

Actually there is even a straight connection: Step-Fun DeepResearch trained on SYNTH (the open Baguettotron dataset).

Hi. Yes this is wholly correct.

On the second points:

* Well I'm very much involved in making open more models, pretrained the first model on free and open data without copyrigh issues, released the first version fo GRPO that can run on Google Colab (based on Will Brown). Yet, even then I have to be realistic: open source RL has a data issue. We don't have the action sequence data nor the recipes (emulators) that could make it possible to replicate even on a very small scale what big labs are currently working on.

* Agreed on this and I'm seeing this dynamic already in a few areas. Now it's still going to be uphill as some of the data can be bought and advanced pipelines can shortcut some of the need for it, as models can be trained directly on simulated environments.

Seeing LLM as a motor was a legitimate view until recently. But what we're start seeing with actual agentification is models taking the driver seat, making the call about search, tool use, API. Like DeepSearch, these models are likely to be gated, not even API accessible. It will be even more striking once we'll move to industry specific training — one of the best emerging example is models for network engineering.

The key thing really about my post: it's about the strategy model providers are going to apply in the next 1-2 years. Even the title is coming from an OpenAI slide. Any wrappers will have to operate under this environment.

Hi. So quickly:

* RL is Reinforcement Learning. Already used for a while as part of RLHF but now we have started to find a very nice combo of reasoning+RL on verifiable tasks. Core idea is that models are not just good a predicting the next token but the next right answer.

* I think anything infra with already some ML bundled is especially up for grabs but this will have a more transformative impact than your usual SaaS. Network engineering is a good example: highly formalized but also highly complex. RL models could increasingly nail that.

Ah it's completely volontary on my part: I want to keep the historical spelling as much a possible. That's why I used the google books OCR which does a better work at it than Gallica. That's still a bit erased in the current model (I don't think the tokenizer likes this so much).

Either that or appending archaic expressions in the prompts (a bit like the prompt extension of Midjourney)

Well you're not going to believe it but I do have a FoucaultGPT just being trained (indirectly: Foucault is just part of my extended French historical corpus). As a sample:

Prompt : Écrit un livre de Michel Foucault sur les mesures de confinements pendant l'épidémie de COVID-19.

French historical model: Il est donc important de préciser que les mesures de confinement, loin d'être une simple réaction à la crise sanitaire, sont en réalité le résultat d'une longue histoire des techniques de gouvernementalité. Elles ont été étudiées et mises en place dans différents contextes (guerre, épidémies) au cours du XIXe siècle. Il faut aussi bien comprendre qu'elles ne se limitent pas à l'aspect médical : il s'agit également d'un ensemble de dispositifs qui permettent d'organiser et de contrôler un espace urbain considéré comme atteint par une maladie contagieuse. Le confinement n'est donc pas seulement une méthode pour empêcher les personnes infectées de sortir de chez elles ; c'est surtout une manière de surveiller les mouvements des individus et de les mettre en quarantaine si nécessaire. La question de savoir comment ces mesures ont pu être acceptées par les populations doit être posée dans le contexte historique où les techniques de gouvernementalité ont été développées et appliquées. Enfin, il est essentiel de souligner que la pandémie actuelle a mis en lumière l'insuffisance des systèmes de soins publics et privés, ainsi que l'impact socio-économique de cette crise. Les questions liées à l'accès aux soins, au financement des hôpitaux, à l'emploi et à la sécurité sociale doivent être abordées dans ce contexte.

In a way you could do so by prompting Monad with artificial intelligence stuff. I did a try lately on the latest OpenAI events and it went on like this: "In this sad and tragical storye, you shall heare how Sam Altman, a manne of great wit and judgement, was evicted out of his Companie of Artificiall Machines for want of candour in his Communications with the Board of Trustees…"

Yes I needed that for the conversational/instructional capacities. I've made a lot of tests with base models and it would not listen to instruction very well…

Yes you're perfectly right. I've currently tried to maintain some kind of uneasy balance between good conversational capacities (so that it really is a "chatGPT") and cultural reset, which means it may revert from its 17th persona occasionally. Actually this issue is a good illustration that LLM really are latent space explorer. When you prompt a clearly contemporary concept, the default embedding position will shift back to contemporary associations.

As a prompt engineering trick, I find it helps to use faux archaism (such as "pray tell" as an introductory phrase). This is basically a reinforcement anchor in the 17th century region of embedding space.

Not feasible to go with pretraining only.

What is possible is to use a larger learning rate but this will be a hard trade-off with conversational capacities. Fine tuning is currently based on original texts with a synthetic prompt. The issues that people have noticed (repetitions, not remembering what was in the prompt) will be more significant if the learning rate is higher.

Maybe a solution will be to provide two different variant of the same model, one less immersive and more workable, and the other more immersive and buggy.