He went into the details of how he thinks about open sourcing weights for Llama responding to a question from an analyst in one of the earnings call last year after Llama release. I had made a post on Reddit with some details.
https://www.reddit.com/r/MachineLearning/s/GK57eB2qiz
Some noteworthy quotes that signal the thought process at Meta FAIR and more broadly
* We’re just playing a different game on the infrastructure than companies like Google or Microsoft or Amazon
* We would aspire to and hope to make even more open than that. So, we’ll need to figure out a way to do that.
* ...lead us to do more work in terms of open sourcing, some of the lower level models and tools
* Open sourcing low level tools make the way we run all this infrastructure more efficient over time.
* On PyTorch: It’s generally been very valuable for us to provide that because now all of the best developers across the industry are using tools that we’re also using internally.
* I would expect us to be pushing and helping to build out an open ecosystem.
https://www.reddit.com/r/MachineLearning/s/GK57eB2qiz
Some noteworthy quotes that signal the thought process at Meta FAIR and more broadly
* We’re just playing a different game on the infrastructure than companies like Google or Microsoft or Amazon
* We would aspire to and hope to make even more open than that. So, we’ll need to figure out a way to do that.
* ...lead us to do more work in terms of open sourcing, some of the lower level models and tools
* Open sourcing low level tools make the way we run all this infrastructure more efficient over time.
* On PyTorch: It’s generally been very valuable for us to provide that because now all of the best developers across the industry are using tools that we’re also using internally.
* I would expect us to be pushing and helping to build out an open ecosystem.