jing

One thing that I've found is very important is to clean out the fans often, otherwise dust builds up and prevents cooling. Just unscrew the plate on the underside of the laptop and blow / brush / pick out the dust that's built up (both the fan intake and the fan outlet). Doing this every few months has been a game changer for my 9550.

Comment · 7 years ago

I think this resource could be helpful:

http://colah.github.io/posts/2015-08-Understanding-LSTMs/

Essentially, RNNs and feed forward networks are very similar - RNNs are just "unrolled through time" and every timestep shares the same weights. The activations are slightly different as well, but the core concept is the same as feed forward networks; it's not a completely different concept or idea.

Comment · 8 years ago

Note that it's not just the loss function. It's the loss function combined with a very specific problem formulation - namely a neural network with only linear activations (equivalent to a 0-layer network). Once you go to non-linear layers or a different loss it's no longer solved analytically.

I do see a lot of people writing tutorials like OP's. See for example:

https://towardsdatascience.com/linear-regression-using-gradi...

The existence of these articles should not be taken as an indication of best practice. They often have the goal of teaching SGD in a simplified setting, not teaching best practice for LLS. I suppose only nice thing about using TF / SGD for such a simple problem is that you now have starting point for solving more complex problems (RELU activation, cross-entropy loss, more layers, etc.).

A few other points as to why you would never SGD for LLS:

1) it's always way slower than the closed form matrix solutions

2) if you're doing SGD instead of just GD, there's noise in which "rows" are in a given batch - as a result, repeated runs may not converge to exactly the same final weights. This never happens with the analytical solution which always gets exactly the same result.

3) if you're doing this as part of a data science pipeline which is likely the case in the real world, you'll likely want to do some cross-validation. In the SGD case you have to recompute the entire solution for each fold whereas in the LLS case you can immediately compute CVs once you've calculated the initial XTX / XTYs. This makes the process of using LLS even faster than SGD.

Comment · 8 years ago

I find it hard to believe that SGD would be faster than the closed form solutions for linear regression (gels, gelsd etc.). The closed-form solutions give a lot of other benefits in practical settings as well which makes them more likely to be used if possible. SGD + related optimizers give benefits with non-convex or non-analytical loss functions or with non-linear layers / more than one layer.

Comment · 10 years ago

The Xeons support much more RAM.

Comment · 10 years ago

Are physics engines not yet accurate enough to enable "virtual" pre-training / full training of the networks, lighting conditions, etc? If they are, exclusively using physical robots seems somewhat inefficient.

Comment · 11 years ago

DAC is an order of magnitude cheaper than optical and might be fine for their use based on their comments of cat6 v cat6a and "short runs in their lab".

Also, if they are using two or three NICs per box, you'd need to factor in the cost of two / three of the X540's vs. a single 40gbe connection and you'd need to double the cost-per-port of the switch vs the 40gbe option.

Comment · 12 years ago

Some claim that this is part of a broader trend of "generational theft" from the young to the old and have some interesting slides to demonstrate their thesis:

http://online.wsj.com/news/articles/SB1000142412788732348570...

http://vimeo.com/65731171#t=11m55s