gglon

From the paper, section 2.1: minimize_θ,φ,Δ ||P_φ(Δ, E_θ(x)) - sg(E_θ'(y))||_1

where

y - full video, x - masked video, E_θ(.) - learned encoder (semantic embedding), P_φ(.) - learned predictor, Δ - learned mask (which patches in a video where dropped), sg(.) - stop gradient to prevent change, gradient propagation in E_θ'(.), which in turn is an exponential moving average of E_θ(.) ie. θ'_new <- τ θ'_old + (1-τ) θ. So the loss is applied only to the predictions of the masked patches while the encoder of full video follows the learned one. This asymmetry in learning prevents collapse of the encoder to a trivial constant.

I was thinking, to build a large, efficient factory autonomously, one could use LLM as a high level agent that is using specialized tools. The overall strategy would perhaps look like following:

1. create a (intermittent) goal for a resource production

2. create a factory graph with calculated number of machines and number of resources required to transport between them. This would be done by using linear programming (factorio calculator)

3. somehow map the resulting graph to a hardware description language. Such that each entity would be mapped to unique logic component. And each transport lane would be mapped to a unique wire (most difficult)

4. compile to 2d FPGA layout using all the VLSI algos like partitioning, routing (hdl compiler)

5. map the resulting plan back to a concrete factorio design

This doesn't apply to energy grid scale projects. Also that is why energy sector in most countries is privatized. Take Germany with total energy costs that would be like ~10% of the federal budget (about 40 billion euros). Making energy even more expensive would only lead to further deindustrialization and lower gdp making it politically non viable. Also, you can't just throw money and buy storage like with typical government contracts - it requires building entire new industries (like Germany's struggles with hydrogen infrastructure).

none of those storage solutions are easy for grid scale. For batteries you need whole factories, pumped requires permits and tone of cement, chemical is like hydrogen with low efficiency. All of those are expensive which for government is synonymous with difficult.

pumped hydroelectric is storage; right now it is like 99% of world storage capacity.

Solar PV operates 10-30% of the time (depending on location). Without storage, this naturally limits direct solar contribution to about 20% of energy demand. Going beyond requires expensive storage solutions.

Achieving the first 20% of solar in an energy mix is relatively easy. Beyond that costs increase 3-5 times per energy unit mainly due to storage.

Also magnet for the pile works well: magnet:?xt=urn:btih:0d366035664fdf51cfbe9f733953ba325776e667