jeeceebees

I think this is a property spheres. It seems to me that any two spheres that are touching have a straight line from one center to the other center exactly through the point of contact. Try thinking of just two spheres and adding more in step-by-step.

Then the result follows because all the spheres are defined as centered on the cube/sub-cubes respectively.

How does the performance between GPU programs written with std::par compare to those written in CUDA?

Do you happen to know of any online resources that show a comparison of the kernel code and performance of the two frameworks on common tasks?

Having more memory to address means you need more circuits that direct your reads/writes to the right place. Travelling a longer distance / through more complicated routes means the latency for each request will be higher on average.

The hidden state isn't storing the samples of your loop (or a compressed version of your loop). It's encoding a representation of how the output will change based on what the current state and input are. This might be strongly dependent on what the exact samples in the loop are, but it could also be more general. I think it's missing a bit of the representational power of an LSTM to see the state representation as just a buffer of the current input.

But, yeah, at some point your signal has such a complex behavior on long time scales that there isn't a good way to predict it based on a limited state size (or at least gradient descent can't find a function to predict it for you).

LSTM stands for Long Short Term Memory. It's a recurrent network that learns what and how long things should be kept in its internal state buffer. It doesn't have a fixed state size because it's just learning a nonlinear function that takes an input and a state to an output and a new state. Obviously it can't model all possible, infinite length recurrences, but it can definitely do a pretty good job of approximating long term recurrence relations in complex signals.

I think most human creativity is built around a seed of inspiration from outside sources.

In my experience serendipity and happy accidents are exactly what leads to the most creative and interesting outcomes.

This is a system that’s able to generate these inspirations/prompts for you in a way that’s more focused on what you’ve already written. I don’t see it as ironic because I don’t think it’s hijacking your creative potential, just augmenting it.

I think the larger models get, the more incentive there is for researchers to look into pruning/distilling them for practical use.

GPT-1,2,3 et al. have all shown that larger is better. While in the short term this means people will simply throw larger and larger clusters at the problem, in the longer term there needs to be inovation in making it more efficient on the clusters we have (as even the cloud has limits).

I think sheer parameter count is an important part of the equation in general intelligence, so it's important that there are labs that work on scaling up promising leads to trillions of parameters on top of labs thinking of new promising directions.

It's YYMM.ID I believe

Last year may was 1905.xxxxxx

Have all novel qubits gotten nobel prizes so far?

I think there's more than enough room in between bullshit and nobel prize.

This looks very promising, but as always the devil is in the details. The next steps are multiqubit gates, then linking those up into useful quantum circuits, and then hopefully doing actual quantum computations.

Personally I wouldn't expect a nobel prize until one of those last 2 steps. There's a lot of hurdles before then to run into some nasty problems.

I don't really understand why I'd use gin. From the example ipynbs it looks like pretty much the same amount of code but in a gin file and then it spookily fills in parameters for you in python. Why is this useful?

As far as I understand, it's because what it's simulating is a logical qubit which is different from the very noisy, almost instantaneously collapsing physical qubits present in current quantum computers.

Software simulates what's supposed to happen while the hardware only approaches it through many repeated trials.

Of course there is. All the building blocks that people are mix and matching in networks nowadays were introduced at some point.

The paper that introduced batch norm, adaptive instance norm, attention heads, or any module used in a network have an extensive discussion of the motivation for their existance, some derivation or proof that they do what you want, and an empirical test to show it helps in practice. The reason some losses allow GANs to converge in certain situations while others don't isn't a complete mystery, there is theory that supports this.

Researchers designing new models are considering weak points in old approaches, identifying why they aren't working correctly, and proposing something new that solves a part of the problem. All of this is done by looking at the math behind all the operations in the network (or at least the parts relevant to a certain question).

That nobody really knows how AI works is one of those myths told by the media. Just because the model weights aren't interprable doesn't mean we don't know why that model works well. It just takes quite a bit of maths knowledge to really understand state of the art models. All that knowledge is also easily packaged into modern frameworks that make it easy to use without a deep knowledge of why it works. All of this contributes to the feeling that nobody really knows what's going on, while in reality it's onky the majority of people that don't know what's going on ;)