thorel

> The theory of finite fields is based on the theory of prime numbers, because the finite fields are sets of residues modulo a prime number or modulo a power of a prime number.

It is note quite correct that the finite field of order p^k is the set of residues modulo p^k when k > 1. Instead this field is obtained as a splitting field of the field of order p (which is the set of residues mod p).

Agreed, thanks for the clarifications. Another result worth mentioning, which also shows that you cannot hope to uniquely characterize a structure by "reasonable" axioms is the Löwenheim–Skolem theorem which predates Godel's incompleteness (although the history of these results is somewhat convoluted).

There, the obstacle is in some sense of a simplest nature: if your set of axioms admits a countable model, then it admits models of all infinite cardinalities. In other words, it shows that there is something fundamentally impossible in trying to capture an infinite structure (like numbers) by finite means (e.g. recursively axiomatizable).

The article is a bit oversimplifying in summarizing the axiomatic crisis as being problem with sentences like “this statement is false“.

This being said, your intuition is absolutely correct, the crux of the issue is with ‘this‘. What mathematicians realized is that if you are not careful with your choice of axioms, the resulting logical system becomes too “powerful” in the sense that it becomes self-referential: you can construct sentences that refer to themselves in a self-defeating manner.

As others have mentioned, this is the idea underlying Gödel's incompleteness theorem but also, to some extent, Russel's paradox that came before and is what the article is referring to. In Russel's paradox, the contradiction comes from constructing the set of all sets that contain themselves.

Isn't this what is suggested below the second picture?

> Really, it’s harder than this picture suggests, because many experiences are based on other students. If I want you as my project partner but you want to forget I exist, then something has to give.

Regarding stochastic gradient descent, I think there has been an increased understanding in recent years, that the randomness introduced by the random sampling/batching is not only helpful in reducing the computational cost (compared to computing the full gradient) but also in adding noise to escape local minima. Some variants of stochastic gradient descent in fact add some additional random noise to amplify this latter effect and some theoretical guarantees have started to emerge.

I agree that "continuous" is a poor choice of word. Reading the article, it is clear that the author does not care about continuity in the mathematical sense of the term, but rather about allowing the design parameters to vary as a function of the user's environment parameters.

As mentioned in some other comment, a constant function is continuous but is obviously not what the article is about.

There are a few inaccuracies here: using automatic differentiation basically makes computing the gradient of the objective function as efficient as computing the function itself. But the main goal of algorithms being used in machine learning nowadays (stochastic gradient descent and variants thereof) is to avoid having to compute the objective function or its gradient altogether: instead, the gradient is computed at a single data point (example) which provides an approximation of the true gradient.

The important thing is that what is considered expensive is not to compute the objective function or its gradient, but to compute it over the entire dataset. Line-search would require evaluating the function (and its gradient) over the entire dataset several times, which completely defeats the purpose of stochastic gradient descent.

One could imagine doing a line search using the approximate function or its gradient (coming from the evaluation at a single data point) as the basis for a line-search, but intuitively it does not make much sense to fine-tune the step size to a single example, and this would in any case destroy the guarantees provided by the Strong Wolfe conditions.

Finally, there are convergence guarantees for stochastic gradient descent with fixed learning rate when the objective functions are convex.

I am not sure I understand the criticism of RLS. It feels to me that the Language Server Protocol does allow for incremental parsing etc. In particular, the approach taken by rust-analyzer

> Analyzer maintains a "database" of input facts (source code + information about project structure) and derived facts (syntax trees, name resolution information, types). The client then can change input facts, and query derived facts at the current state of the world.

seems that it could also be adopted by a language sever for Rust.

This is an interesting point, but note that the scheme you are suggesting is "stateful" (you need to remember the last ciphertext to be able to generate the next one). The standard definition of an encryption scheme (and the one used in this paper) is stateless (once you have generated the key, you can produce ciphertexts without keeping track of any state). Stateful encryption schemes lead to all kind of complications and for this reason tend to be studied way less from the theoretical perspective. But it would be an interesting question to see if the results from this paper extend to the stateful setting (and I agree that because of the example you provide, some adaptation would have to be made).