overlords
Joined 8 years ago
Joined 8 years ago
The below all are easier machine learning problems than self-driving cars, yet no big tech companies or national initiatives are focused on aggressively applying machine learning to them.
Likely a couple of billion dollars, a year, and a 100 people lab would 'solve' each specific problem.
1. Robot that cooks meals, and clean the dishes afterwards. That saves billions of hours daily.
2. Robotic self-cleaning toilets. Saves another billion hours daily.
3. Robots that can dig up dirt and build a house from that dirt.
4. An app that can teach anyone anything like a teacher would - literally - a talking avatar and cameras and voice output and machine learning powered dialogue.
5. Home manufacturing 'box' that can make 95% of anything that anyone typically wants (some arrangment of 3d printer/laser cutters/pcb placement/wood router machines etc, that can take plastics/wood/metal/electronic components and output a gadget/furniture etc)
The above 5 give the equivalent of a 'basic income' for everyone (if distributed to everyone, and assuming the finished gadget is about the size and complexity of an automobile).
Then the inputs/ouputs problem of energy/raw materials/waste needs to be provided. Disregarding scientific advances like fusion power etc (which require more than 100 billion maybe, or not possible), a drone distrubution platform for getting the energy / matter (input/waste) handled. To do this (as above, 100 people, a year, 1 billion dollars)
6. p2p aerial surveillance system for air traffic managment of millins of drones. Basically, a sky pointing camera gadget that analyzes and broadcasts what it sees and process. Millions/billions of these camera gadgets airdropped every few 100 meters .
7. a drone that can carry 100 kilos and drop ship materails/waste p2p using the p2p air traffic control. The drones are battery operated with range a coupel of kilometers.
8. a drone that can mid-air 'refuel' the above drones. Basically a flying battery that recharges that larger cargo drones.
Summary - 'gadgetize' every problem (it becomes a self contained mechano-electrical desktop/fridge size thing that a team of 100 people can rapidly iterate on) and throw machine learning at it at heavily as possible. Seek to eliminate human labour as fast as possible.
It's better the 1 rich person is forced to consume that in his lifetime. By doing so that money is redistributed back to the economy, and his 3 kids are also productive.
In scenario 1 only 1 out 4 people work. In scenario 2, 4 out of 4 people work.
Also - it's hypocritical for 'pull yourself by your bootstraps' for children born poor, and 'guaranteed basic income' for children born rich. That is more dehumanizing than not being allowed to pass off inheritance.
About 10 million cars are produced in the USA every year. An RV/Trailer type thing is like a car - so quite easily you could imagine producing that many of these 'tiny houses' every year with car style factories.
About 600k people are homeless in the USA. That's about equal to the number of RV's produced every year.
So that solves 'physical' homelessness quite quickly if there was an effort to. (that disregards that some homeless choose to homeless out of mental illness etc)
'I like they way my lawn is and i don't like construction noises'
vs
'I can't get a job because it's too expensive to rent or buy where the jobs are, so i live in poverty' (or i can't start family because i can only afford a room-share, or any number of other effects from NIMBYism)
Already we have it, by definition - it's an equation.
So some part of your calculation is wrong.
Likely you're over-inflating the amount paid out (it's not going to by 15/hr person) and undercounting payments (maybe disabilty/housing/local programs).
UBI given at equivalent levels to today's welfare would be strictly cheaper because of the reduced adminstration cost.
(this though is different from the question of whether payments should be higher and paid by increased taxation.
And off-topic, but taxation doesn't need to be increased - it can be payed by helicopter money, but actually not cause inflation, because of geographic mobility deflating that aspect of the economy).
Prominent economist - Greg Mankiw https://www.youtube.com/watch?v=4cL8kM0fXQc
Removing the needs assessment also removes the geographic location requirement, and that allows to deflate the land value inflation that is the real cause of inequality today.
- you were doing dot products at each layer to introduce non-linearity in transformer (and neural nets in general). Polynomials are already non-linear, so you don't need that. Transformer and vw -interact are polynomials. Maybe the feedforward layers and skip connections are not actually needed.
- 12 layers ? vw -interact xxxxxxxxxxxxx is 12 layers. You need a lot of memory for that, but in principle vw interact can do any number of them
These results are coming from google and their massive compute resources. If they ran vw with -interact x^13 they might get similar results.
We're really talking about polynomial approximation here, both transformer and vw used in this way. And that is in theory able to approximate any continuous function (just like neural networks).
Why neural networks? https://en.wikipedia.org/wiki/Universal_approximation_theore...
Can polynomials do this? (Yes) https://en.wikipedia.org/wiki/Stone%E2%80%93Weierstrass_theo...
What is transformer and attention? https://pathmind.com/wiki/attention-mechanism-memory-network
Attention = Polynomial (x2,x3 etc.)
Polynomial = interaction. VW flag -interaction
1 layer transformer = xx. (x^2)
2 layer tranformer = xxx. (x^3)
3 ... etc
What is reformer? Transformer where LSH is applied.
One type of LSH is SimHash. ngrams of strings, followed by 32 bit hash.
Vowpal Wabbit -n flag for ngrams.
vw -interact xxx -n2 -n3 and you get ngrams + 32 bit hash doing SGD over a vector.
This vector is equivalent to a 2 layer reformer.
Non-linear activation is not needed because polynomials are already nonlinear.
So vw + interact + ngrams (almost)= reformer encoder. (if reformer uses SimHash, then they are identical).
Transformer/Reformer have an advantage, the encoder-decoder can learn from unlabeled data.
However, you can get similar results from unlabeled data using preprocessing such as introducing noise to the data, and then treating it as noise/non-noise binary classification. (it can even be thought of as reinforcement learning, with the 0-1 labels as the reward using vw's contextual bandits functionality. This can then do what GAN's do - climb from noise to perfection).
Simhash algorithm, the LSH i knew about (which i mistakenly thought is LSH) works exactly like VW. It is ngrams + hash.
http://matpalm.com/resemblance/simhash/
https://en.wikipedia.org/wiki/SimHash
Simhash, a type of local sensitive hashing - using hash functions on ngrammed data.
That is exactly what Vowpal Wabbit does.
Locality sensitive hashing is a way to put similar vectors into the same buckets - by omission etc. It does this by hashing, but the intent is to approximate nearest neigbours.
skipgram/ngrams turn features into other features by omission etc, and so makes similar things the same. The hashing trick then reduces memory usage.
So yes, you're right the hashing in locality sensitive hashing is different in intent, but my point is, that both these approaches are designed to be more memory and compute efficient.
And vowpal's feature interactions give you transformer layers.
Add up all these together, and they have about the same net effect.
The hashing trick in VW hashes multiple same words into one integer, not the same as reformer, but similar to how reformer puts similar vectors together.
With VW's ngram/skipgram features, you get the same kind of effect - similar strings hash into the same hash.
So locality sensitive hashing = (is around about the same thing as) ngram/skipgram on strings plus hashing trick.
[email protected]