FrereKhan
Joined 9 years ago
Karma
14
Submissions
8
Member since
Jun 29
Neural computation with experimental verification
View on Hacker NewsRecent submissions
Comment · 1 year ago
This paper imports an arbitrarily-chosen aspect of cortical architecture — topological maps of function — and ignores every other aspect of biological neural tissue. The resulting models show lower performance for the same number of parameters — not surprising, since they are more constrained compared with baseline. They may be slightly more robust against pruning — not surprising, since they are more regularised.
The figures show individual seeds, presumably, with no statistical analysis in the performance or pruning comparisons, so the null hypothesis is there is no difference between toponets and baseline. I would never let this paper be submitted by my team.
We haven't learned anything about the brain, or about ANNs.
Comment · 5 years ago
I agree. The use case needs to justify having state, otherwise the ideal architecture is something like NullHop. Temporal signal processing / vision processing tasks are ideal for SNNs, especially if the inputs can also be sparse.
Comment · 5 years ago
Ultra-low-power neuromorphic processors such as DynapSE[1] have been cross-bar free for several years now, making them a perfect fit for sparse networks (both weight- and activity-sparsity).
[1] https://arxiv.org/abs/1708.04198
Comment · 5 years ago
If you bring activation sparsity into the mix, the advantage of SNN processors over GPUs/TPUs becomes more clear. Loss-gradient-based optimisation approaches are great because they give you a tool to include e.g. sparsity regularisation into the loss. Encouraging sparse activity makes simple linear algebra a poor fit for network activation, and SNN processors a much better fit.
Comment · 5 years ago
It's not quite correct to say this is only for achieving deep learning. Gradient-based parameter optimisation is still a useful tool, even for small shallow networks that would be ideal for event-based signal processing.
Even for small-network tasks, training spiking networks has been non-trivial. This paper provides a way to get exact gradients, implying probably faster optimisation than using surrogate gradients or other approximation methods for SNNs.
Comment · 8 years ago
aiCTX (ai-ctx.com) | Real-time low-power machine learning | SW Eng., Silicon HW Eng., ML research | Zürich, Switzerland | ONSITE, INTERNS, FULL-TIME
aiCTX is a small (<10 employees) startup based in Switzerland, with a diverse group of SW, HW and ML engineers. We design ultra low power analog and mixed-signal neuromorphic silicon, built for real-time machine learning applications.
We're looking for awesome people to help build our SW stack; to help our HW designers for the next round of chips; and to help our research team build solutions for particular application domains.
Zürich is cool, and Switzerland is beautiful. :)
Check out the position details in the link above if you'd like to join us!