That was a good read. Some slightly different perspectives:
It feels like a wild west in GNNs right now, with a big paper or tool every few weeks. Interestingly, many of the issues discussed in the article are already components in modern OSS frameworks:
- heterogeneity: RGCNs split weight matrices by node type, with diff versions in most frameworks now. Issues like smoothing are interesting too, as discussed.
- scaling: sampling and various memory techniques are enabling handling massive graphs on single nodes, metapaths & other structures enable farther communication, etc. Especially impressive is OSS scaling work by cugraph+dgl teams.
At the same time, even with that, we're finding it's still too hard for non-academic teams to use this stuff in production/operational scenarios: model zoo (esp. some particularly important modeling areas not discussed like time), GPUs, clusters, data pipeline integrations, etc. Imagine having 100K cyber alerts aggreated every day or a 50M users/transactions and wanting to score them... It's nowhere as easy (yet) as something like adding a layer to BERT like NLP people can do. If that's more your speed:
- We're working on OSS automl for graph ai, trying to get typical internal team pipelines to go from events data to decisions & UIs in a few lines. First out was for UMAP and we've been pushing on GNNs more recently, http://github.com/graphistry/pygraphistry . Elsewhere, we're also working on the MLOps side and some key modeling scenarios.
- ... both graphistry + almost all our customers & partners are hiring here! If you're into data (analytics/mlops/dataeng/ds), or general js/python fullstack dev, a lot happening here for missions like supply chain, cyber, fraud, & misinfo. Would love to chat!
EDIT: Another two things I've found interesting:
-- GNNs are probably most researched by folks in material sciences (chem, physics, ...), and it's really changing things like protein folding, and maybe next biggest on social networks. We see equal practicality for other key problem (cyber, fraud, supply chain, ...), but we're seeing much less academic work in these, and I think that's b/c academics are at a significant data disadvantage vs most industry teams. So even though big interest outside of the academic team areas, it's quite early days.
-- Industrially, we're seeing GNNs promoted primarily by graph databases.. but almost all graph databases are CPU-based vs GPU-based, and in our polls of commercial GNN usage, typically not as part of a graph DB pipeline but something like regular data lakes (parquet, ...) feeding into regular scalable GPU compute tiers
It feels like a wild west in GNNs right now, with a big paper or tool every few weeks. Interestingly, many of the issues discussed in the article are already components in modern OSS frameworks:
- heterogeneity: RGCNs split weight matrices by node type, with diff versions in most frameworks now. Issues like smoothing are interesting too, as discussed.
- scaling: sampling and various memory techniques are enabling handling massive graphs on single nodes, metapaths & other structures enable farther communication, etc. Especially impressive is OSS scaling work by cugraph+dgl teams.
At the same time, even with that, we're finding it's still too hard for non-academic teams to use this stuff in production/operational scenarios: model zoo (esp. some particularly important modeling areas not discussed like time), GPUs, clusters, data pipeline integrations, etc. Imagine having 100K cyber alerts aggreated every day or a 50M users/transactions and wanting to score them... It's nowhere as easy (yet) as something like adding a layer to BERT like NLP people can do. If that's more your speed:
- We're working on OSS automl for graph ai, trying to get typical internal team pipelines to go from events data to decisions & UIs in a few lines. First out was for UMAP and we've been pushing on GNNs more recently, http://github.com/graphistry/pygraphistry . Elsewhere, we're also working on the MLOps side and some key modeling scenarios.
- ... both graphistry + almost all our customers & partners are hiring here! If you're into data (analytics/mlops/dataeng/ds), or general js/python fullstack dev, a lot happening here for missions like supply chain, cyber, fraud, & misinfo. Would love to chat!
EDIT: Another two things I've found interesting:
-- GNNs are probably most researched by folks in material sciences (chem, physics, ...), and it's really changing things like protein folding, and maybe next biggest on social networks. We see equal practicality for other key problem (cyber, fraud, supply chain, ...), but we're seeing much less academic work in these, and I think that's b/c academics are at a significant data disadvantage vs most industry teams. So even though big interest outside of the academic team areas, it's quite early days.
-- Industrially, we're seeing GNNs promoted primarily by graph databases.. but almost all graph databases are CPU-based vs GPU-based, and in our polls of commercial GNN usage, typically not as part of a graph DB pipeline but something like regular data lakes (parquet, ...) feeding into regular scalable GPU compute tiers