> I haven't played with copilot, but I imagine that something like guessing the graph based on node labels and on some edges is feasible using GNN?
Copilot is based on OpenAI Codex, which is based on GPT-3, which is a transformer model.
Although technically, transformers are mostly GNNs that are "fully connected" (in the graph theory sense), I don't think that supports your speculation here about how GNNs could be used for code analysis since the "tokens" that GPT-3 is trained on are not programming-language syntactic constructs, but sub-word units obtained from natural language (something like WordPiece).
I will say though, I am equally excited by the exact prospect you raised of using something like GNNs for code analysis.
My hunch is that if somebody can figure out a way to make training hierarchical/graph based neural networks very fast, we'll observe the same gains that we did with transformers. But hierarchical/graph based models don't lend themselves to efficient computation.
Copilot is based on OpenAI Codex, which is based on GPT-3, which is a transformer model.
Although technically, transformers are mostly GNNs that are "fully connected" (in the graph theory sense), I don't think that supports your speculation here about how GNNs could be used for code analysis since the "tokens" that GPT-3 is trained on are not programming-language syntactic constructs, but sub-word units obtained from natural language (something like WordPiece).
I will say though, I am equally excited by the exact prospect you raised of using something like GNNs for code analysis.
My hunch is that if somebody can figure out a way to make training hierarchical/graph based neural networks very fast, we'll observe the same gains that we did with transformers. But hierarchical/graph based models don't lend themselves to efficient computation.