autopoiesis
Joined 9 years ago
Joined 9 years ago
cs702's original comment above is excessively hyperbolic: the compositional structure of Bayesian inversion is well known and is known to coincide structurally with the backward/forward structure of automatic differentiation. And there have been many papers before this one showing how predictive coding approximates backprop in other cases, so it is no surprise that it can do so on graphs, too. I agree with the ICLR reviewers that this paper is borderline and not in itself a major contribution. But that does not mean that this whole endeavour, of trying to find explicit mathematical connections between biological and artificial learning, is ill motivated.
It has been discussed on Hacker News at least a couple of times previously -- fairly recently, even. You might be interested to look at these discussions:
https://news.ycombinator.com/item?id=20376325
https://news.ycombinator.com/item?id=19701767
Edit to add:
You might also be interested to learn that categorical approaches to linguistics typically take as their starting point monoidal categories, in which there are notions of 'parallel' as well as 'sequential' composition. It turns out that the usual categorical semantics for linguistics shares a lot with the categorical semantics for quantum mechanics: roughly, meanings are vectors, like quantum states. You can read more about doing (finite-dimensional) quantum mechanics entirely using string diagrams (the formal diagrammatic calculus of monoidal categories) in the work of Bob Coecke, who also played a large part in originating these approaches to linguistics.
For example, on the quantum side, an excellent book is 'Picturing Quantum Processes' [0]. And on the linguistics side, the paper linked in the article is a good start: https://arxiv.org/abs/1003.4394
[0] Not freely available, but some slides are at https://www.cs.ox.ac.uk/ss2014/programme/Bob.pdf
Edit, again:
There is also of course Bartosz Milewski's book / blog series 'Category Theory for Programmers', which introduces category theory from the perspective of Haskell and C++ programming: https://bartoszmilewski.com/2014/10/28/category-theory-for-p...
But the best introduction to category theory I have read is Leinster's book, 'Basic Category Theory': https://arxiv.org/abs/1612.09375
And as you might have guessed, I do agree with your statement!
I've always been really excited by the possibilities implied by PulseAudio's network capabilities, but disappointed by their latency and bandwidth requirements. Roc + Opus would be amazing.
[1] https://sourceforge.net/projects/android-ports-for-gnu-emacs...