vomjom
Joined 17 years ago
Joined 17 years ago
In fact, for a single step with no policy lag, it's equivalent to a standard policy gradient update.
DeepMind was also able to train a CTF agent with human-level reaction time: https://deepmind.com/blog/capture-the-flag/
I suspect the difference that allows you to train with reaction time is an RNN or compensating for the lag some other way. I'm testing that out right now with my own SSBM bot: https://www.twitch.tv/vomjom
TensorFlow supports a standalone server mode where it receives computation graphs and executes them. This is nice because then you can remotely execute on any accelerator (Cloud TPU, multi-worker multi-GPU) from your laptop.
In their demo, they did exactly that with a Cloud TPU: it connected to a TensorFlow server that executed the machine learning training part of the program.
Also see Chris's introductory post for the project: https://groups.google.com/a/tensorflow.org/forum/#!topic/swi...
It will have very different performance characteristics.
The short answer is no.
The long answer is yes, but only if you create the model in Python, export it, and then feed training data in other languages. There are some people doing exactly that.
Long term, I'd like to give all languages equal footing, but there's quite a bit of work left.
Responding to your original post: you argue that async/await intends to solve the problem of data races. That's not why people use it, nor does it tackle that problem at all (you still need locks around shared data).
It only tries to solve the issue of highly-concurrent servers, where requests are bound by some resource that a request-handling threads have to wait for the result of (typically I/O).
Coroutines/fibers are not an alternative to async servers, because they need primitives that are either baked into the language or the OS itself to work well.
1. Thread-per-request. This is a simple model. You have a fixed-size thread pool of size N, and once you hit that limit, you can't serve anymore requests. Thread-per-request has several sources of overhead, which is why people recommend against it: thread limits, per-thread stack memory usage, and context switching.
2. Coroutine style handling with cooperative scheduling at synchronization points (locks, I/O). This is how Go handles requests.
3. Asynchronous request handling. You still have a fixed-size thread pool handling requests, but you no longer limit the number of simultaneous requests with the size of that thread pool. There are several different styles of async request handling: callbacks, async/await, and futures.
#2 and #3 are more common these days because they don't suffer from the many drawbacks of the thread-per-request model, although both suffer from some understandability issues.
AT&T: $10/50 GB = 20 cents/GB
Amazon S3/EC2: 10 cents/GB
It's not like we're being gouged here...
The problem is that Amazon has no operations in certain states to avoid charging sales tax. Buying Borders would put them in every state.
On a related topic about another Google project: http://sergey.melnix.com/pub/melnik_VLDB10.pdf
But, companies must eventually give out dividends. There's a certain point at which the company's cash hoard gets so large that investors become unhappy (see Microsoft).
When you do a buyback, the book value drops (company loses cash), but the discounted future cash flows remains unchanged. The number of outstanding shares also drops.
The net result is the stock price increases as a company accumulates cash and uses it for buybacks because the number of outstanding shares drops.
Another way of thinking about it is that it's the same as dividends, but the dividend only goes to the sellers of the stock during a buyback.