vivekkalyan

This is great work! Mechanistic interpretability has tons of use cases, it's great to see open research in that field.

You mentioned you spent your own time and money on it, would you be willing to share how much you spent? It would help others who might be considering independent research.

> We aren't sure, but the speculation is that in the process of training, GPT-3 found that the best strategy to correctly predicting the continuation of arithmetic expressions was to figure out the rules of basic arithmetic and encode them in some portion of its neural network, then apply them whenever the prompt suggested to do so.

I strongly disagree. GPT-3 has 100% accuracy on 2-digit addition, 80% on 3-digit addition, 25% on 4-digit addition and 9% on 5-digit addition. If it could indeed "understand arithmetic" the increase in number of digits should not affect its accuracy.

My perspective as an ML practitioner is that the cool part of GPT-3 is storing information effectively and it is able to decode queries easier than before to get the information that is required. Yet with things like arithmetic, the most efficient way would be to understand the rules of addition but the internal structure is too rigid to encode those rules atm.

The rules seem pretty clear that consent is required from any persons appearing in any external datasets that are required. The winners scraped data from Youtube videos so I am not not sure the issue is.

The more worrying takeaway is that the winners scraped videos from people who clearly had no intention of their videos being used for a deepfake detection algorithm. Yet they did not think of the ethical considerations of using that data (did everyone in the video even have a say in the video being uploaded?). I think Kaggle disqualifying the team is the right move (even if it's a painful one for the winners).

I prefer the emacs bindings for the command line such as C-A, C-U (mostly due to muscle memory), but have set up my $EDITOR as vim. This allows me to do C-X C-E, which opens the current command in vim to be edited.

If you are using zsh, you need to add this to ur .zshrc

  autoload -z edit-command-line
  zle -N edit-command-line
  bindkey "^X^E" edit-command-line

Can we please stop this meme? Every thread about DDG invariably seems to bring up this (false) statement.

Anyone has thoughts about cassandra and how it compares to mongodb? There seems to be a big enterprise push with azure's cosmosdb, but I've not heard much about it from people who have actually used it.