dennybritz

I do want to release the code, but it may take a couple more weeks. It's currently a mess of research code that's mixed with some other stuff I was working on so I would need to do a cleanup first. See you on the boat!

I think there are some key differences to cycles in Chess. In Chess, the cycles come from your (or your opponents) actions. With minimax you would max (or min) over these actions, so if you pruned away the cycles at some depth your result is not incorrect because the min or max does not change. But here the cycles come from randomness in the game of which chess has none, and you need to take an expectation over the cycles. So you can't just prune them away without changing the result.

A scoring function could definitely help guide the exploration and/or prune the tree, but only at the action nodes, not the environment randomness nodes. Rolling out the full tree more than 1-2 levels would be infeasible because of the randomness in the environment. When you take an action, the randomness can transport you into an exponential number of states, so you have a huge branching factor that is much larger than chess. I think in chess you have a factor of 40ish? Here it's more like ~1000 or ~10,000 depending on the item.

I also wouldn't know how to design a scoring function for this. If you do something simple to take the number of missing modifiers you will end up stuck in bad states. Maybe there is something really clever here that you can do, but I don't know what it is.

If you have an idea how to make tree search work for this I'd love to try it.

Thank you Exile :) I'm glad you enjoyed the read!

Remote Only

Location: Japan (or East Asia in general)

Willing to Relocate: No

Technologies (reverse-chronological order):

    - AI / Deep Learning Research - previously at Google and have published papers. Mostly focused on NLP and RL, but I keep up with other subfields.
    - Infra: Devops, Rust, Go, Kubernetes, Microservices, large-scale systems, all kinds of data stores. Have managed large clusters. Used to be an early Apache Spark engineer and was in a database research group in grad school
    - Worked in HFT-style algotrading for a few hedge funds
    - Worked at multiple early-stage startups, so I can do other things like full-stack web or app development, but I prefer not to do these full-time. But I can help if stuff comes up.

Résumé/CV:

    https://dennybritz.com/about
    https://twitter.com/dennybritz
    http://github.com/dennybritz
    dennybritz [at] gmail

--

Hi! 15+ years of engineering experience, and have been through a lot of technology cycles. I'm in an ok place right now and focusing on research and side projects. I'm not actively looking for work but if there's something at the intersection of my interests I'd love to talk. Not sure myself what that would look like, perhaps something around MLOps, infra/automation, Reinforcement Learning, Algorithmic trading, etc.

Remote Only

Location: Usually Japan/East Asia, but currently in Europe due to COVID

Willing to Relocate: No

Technologies (reverse-chronological order):

  - AI / Deep Learning Research - previously at Google and have published papers. Mostly focused on NLP and RL, but I keep up with other subfields.
  - Infra: Devops, golang, rust, kubernetes, microservices, large-scale systems, all kinds of databases. Have managed large clusters. Used to be an early Apache Spark engineer and was in a database research group in grad school.
  - Briefly worked in algo trading (HFT-style)
  - Worked at multiple early-stage startups, so I can do other things like full-stack web or app development, but I prefer not to do these full-time. But I can help if stuff comes up.

Résumé/CV:

  https://dennybritz.com/about
  https://twitter.com/dennybritz
  http://github.com/dennybritz
  dennybritz [at] gmail

---

Hi! 15+ years of engineering experience, and have been through a lot of technology cycles. I'm in a decent place right now and focusing on research and side projects. I'm not actively looking for work but I figured I would post anyway - who knows what opportunities come along! If there's something at the intersection of my interests I'd love to talk. Not sure myself what that would look like, perhaps something around ML/RL research, academia, trading, or infrastructure.

Yup. The only reason I am starting to write about it now is that I am no longer running the system. You could argue that it's not useful to write about systems that worked in the past, but I would disagree. New systems can work 99% the same way, but get an additional edge from somewhere else, like new data or better models. Most of the engineering will always be the same.

> Just because you have a good forecast doesn't translate into cash. It has to be paired with a trading strategy. This is probably why the author thinks the answer is RL, because coincidentally if you approach this problem with RL, it does the forecasting + strategy.

Exactly, this is one of the nice things about RL. You don't to do a bunch of handwaving to turn your predictions into a strategy.