outsideline
Joined 10 years ago
Joined 10 years ago
Coming up with a functional systems architecture that ties the bits and pieces together is hard work. Understanding what is really happening in the human brain, how/why it is performing various functions, and how this provides for an intelligent architecture is hard work. Creating an 'aware' platform is hard and elusive work which is why people chase the low hanging fruit of optimization algorithms.
*Cheers
Making statements that allow people to see behind the curtains and maybe go off and make their own competitive models... Yes, this is a disservice to the advancement of A.I and should be downvoted : Removing the prestigious veil and illusion from published works.
NTMs memory is external in what sense? Please detail what this means in a 'functional' sense. It's biologically inspired. Neurons maintain memory beyond synaptic weights. The neuron models of present day A.I were basic. Someone comes along and sees the obvious : There is no computational model for how neurons utilize memory and suddenly they're thinking on another level? Give me a break..
Synthetic gradients have everything to do w/ electro-chemical gradients : http://www.nature.com/articles/srep14527 http://www.pnas.org/content/110/30/12456.full.pdf So, where is your establishment that I am incorrect. It is nowhere to be found. Again, biologically inspired computational models.
Oh look, someone published a paper back in June that is an implementation of Differentiable Neural Computers: https://arxiv.org/abs/1607.00036
It's hype and that is a disservice to the community of people completing similar work and taking similar approaches.
It would be an interesting discussion to have. That discussion was terminated in favor of downvoting me.
They're feeble to someone who isn't well informed on neuroscience. Thus, you'd rather be wow'd and believe in the fantasy that only a small segment of people can write computational models of biology.
Continue believing the hype. Rarely will someone be truthful and honest about where they got their ideas when hype follows. An interesting conversation could have transpired. Enjoy the feels from the downvotes.
Present day Neuron models lack an incredible number of functional features that are clearly present in the human brain.
NTMs = representing memory that is stored in neurons https://en.wikipedia.org/wiki/Neuronal_memory_allocation
Decoupled Neural Interfaces using Synthetic Gradients = https://en.wikipedia.org/wiki/Electrochemical_gradient
Differentiable Neural Computers = Won't specify what natural aspect of the brain this derives from.
Pick an aspect of a neuron or the brain that isn't modeled, write a model...
Bleeding edge + Operating on another level
The fact that someone is going out of there way to remove points from my posts so that this doesn't see tomorrow's foot traffic instead of replying and critiquing me just goes to show how truthful these statements are.
Anyone can create such models. No one has a monopoly or patent on how the brain functions. Thus, expect many models and approaches.. Some better than others.
You can down-vote all you want. The better model and architecture wins this game. It would help the community if people were honest about what's going on here but people instead want to believe in magic and subscribe to the idea that only a specific group of people are writing biologically inspired software and are capable authoring a model of what is clearly documented in the human brain. Interesting that this is the reception.
Here.. take the 'Axon Hillock' https://en.wikipedia.org/wiki/Axon_hillock code up a function for it, attach it to present day neuron models, make it do something fancy, write a white-paper and kazaam you're operating on another level..
Get it?
It's decoupled yet stores transient local information regarding previous neuron activity.
Another bio-inspired copy-pasta.
In the human brain, Neurons store an incredible amount of information. Neuron models in neural networks only did so with weights.
There is still a lack of understanding on how the human brain does it. Deep Mind grabbed a proven memory model from Alan Turing's work and applied it to the feature barren neuron models in use. Sprinkle magic ...
They are not operating on another level, they're bringing over features that are well documented in the human brain and in white papers from a past period when people actually thought deeply about this problem and applying it.
https://en.wikipedia.org/wiki/Bio-inspired_computing
There is no 'intuition' about the architecture. Study the human brain and copy pasta into the computing realm.
Others are doing this as well. If anyone bothered to read the white papers people publish, you'll see that many people have presented similar ideas over the years.
You can come up with your own neural Turing machine. Take a featureless neuron model, slap a memory module on it and you have a neural turing machine.
>A number of AI/ML startups would like to pursue AGI in the long-term, but why not make revenue along the way? That's exactly the dilemma that many capable researchers faced. Some chose to make money along the way. It shapes the way you then go about solving AGI down the road. It saps up your resources, imagination, and potential and it redirects it to shorter term thinking. Being in the space, there should be an understanding of Short-term vs. long-term thinking based on a reward system that targets short-term success.
> Still, there is at least one startup - Vicarious - focusing purely on long-term research towards AGI that VCs poured plenty of money into, and there are probably others. I'm familiar with Vicarious. You are correct and they notably publish far less details than others. They are funded by top names and are probably the least mentioned when people mention Ai. Why, given the impact that AGI can have, aren't more groups funded? A solution is seemingly right around the corner... Do people not see this?
Yes. Given the attitudes of people in the space and beyond and lack of funding why shouldn't the more valuable and fundamental research be kept secret? They're taking all of the risk to pursue something that everyone is saying is impossible. Why would they publish details? If more funding and support would come through maybe they would. Otherwise, they're seemingly developing a fundamental AGI that has the ability to operate under its own free-will and mechanisms. It is being said that it's impossible... Once it's completed no one will be able to deny it at that point. Maybe the money will come then... Maybe the openness will too.
> Your post implied that you have some special insight into research groups that are doing AGI research and are making steady progress. I have done plenty of Google searches and not run across any notable AGI progress.
Yes and I will not disclose where my insights derive from. I'm sure people have their ways of discovering it nonetheless. Some groups and individuals are developing conscious self-willed software solutions. I'm referencing them. Is that not what 'life' which intelligence springs from fundamentally is? As will be true for AGI... Not programs that mimic human behavior or optimization algorithms.. Software solutions that are truly aware.
Here's what you need to make an Ai Gym :
> Physics engine (pick your open source package)
> Simulator (pick your open source package)
> Write a thin wrapper around it
It can be done and has been done by many individuals in less than a week or two of dedicated work.
I mentioned George Hotz to highlight a capable individual with vision, passion, and capability who 'thinks different'. With many individuals, the capability is there. However, often times the vision, passion, or capability to think different isn't. Funny then that there becomes this hiring norm which attempts to find the 'most capable' individual and ignores the other more important characteristics.
Time will tell. I'm personally looking elsewhere beyond the names everyone mentions when it comes to development in this space.
Given that the public funding of basic research most always results in the ushering in of new technological paradigms that make corporations tens of billions if not hundreds of billions of dollars, more should be done to make sure they pay back into the system that ensures their constant success.
Thankfully we live in an age in which the most important research to developing fundamental things like Ai is 'time'. If you have an internet connection and a reasonably decent computer, you have enough resources to create AGi.
If you passionately care about seeing your idea through, save up money, move to a cheaper location, wire your computer to the internet, and get to work.
Once you've proven your vision, who can deny you then? Slam it on the boardroom table, disrupt present day business models, and be the next tech giant.
Most AGI focused ventures are not even in the U.S. A prominent one is in China. Another is in Europe. The ones in the U.S are privately funded beyond the big names. Furthermore, where is the U.S's equivalent of : https://www.humanbrainproject.eu/ ?
Where are the seasoned software engineers in the ranks at these highly funded silicon valley Ai companies? I typically see a roster of 'big names' and PhDs... Whereas, I look at the companies specifically pursuing AGI and I see a whole range of individuals.. PHD neuroscience, Senior game developer, Robotics Engineer, Senior firmware developer, Guy from down the hall who can code circles around silicon valley's most decorated engineer, etc etc.
Look at the Geohot story, that's the kind of individual and company who ushers in a new paradigm ....
If the solution requires you to 'think different', how do you expect to achieve it by padding your company with a bunch of individuals who are all centered on the same techniques from academia?
Don't stretch your mind to far on this one. If someone or various groups develop AGI and you have no stake in them, you will have no say in how it gets used. For once, people need to be honest with themselves about this.
You're not going to be able to dictate from a third party consortium of PhDs how someone should run their company or develop their software.
Incorrect. It was named a Neural (Turing) machine for a reason. Maybe people should go back and dust off the white papers from the 70s like those who are borrowing from that era and respectfully giving credit where credit is due.
They do great work and they are making great progress in Artificial Intelligence. Many people are. Everything is a stepping stone. It serves no good to over-hype one person's stones over another's or ignore/downplay where they were inspired from. Notable visionaries of a past time were visionaries because they detailed the depths of their thinking and centered on the hows/whys. It seems it is fashionable now-a-days to do the exact opposite. This is to a disservice to learning and progress.
The whole point of the human brain is parallel processing. Extra-cellular chemical Gradients function the same way in the human brain and serve the same purposes. Take a look at the papers I linked.
> I don't know if they are copied or not from nature (doesnt look like).
Extra-cellular chemical Gradients. I linked to white papers that explain how memory is stored in them and shared across neurons. This is how it works in nature and biology.
They named their approach 'Synthetic Gradients'. An artificial form of the biological Gradient that is decoupled and lies outside of a neuron. They are clearly giving credit to nature.
They and many other people are improving mankind. Many others can improve mankind if there was less hype and more of a focus on where the ideas originated.
That was my point..
The behavior of people regarding selective 'hype' is one of the big reasons why a tremendous amount of deeply functional work that centers on hard intuitions and ideas for this area will remain closed source when a real break is made.
Enjoy the hype train I guess... They're operating on another level than anyone else.