Architecture
The code defines the following classes:
LeechConfig – holds hyperparameters (vocab size, model dimension, number of layers/heads, etc.) and asserts that head_dim is a multiple of 24.
generate_leech_kernel() – returns a 24×24 orthogonal matrix (placeholder; can be replaced with actual lattice vectors).
LeechAttention – multi‑head attention where Q and K are transformed by the frozen block‑diagonal Leech matrix.
LeechResonanceLoss – combines standard cross‑entropy with the geometric resonance loss.
LeechBlock – a pre‑norm transformer block with LeechAttention and a feed‑forward network.
LeechTransformer – the full model with token/position embeddings, stacked blocks, final norm, and language modelling head.
DreamDecoder – evaluates the resonance of a hidden state against the Leech basis.
leech_generate() – generates tokens step‑by‑step, printing resonance values and status if desired.
LeechConfig – holds hyperparameters (vocab size, model dimension, number of layers/heads, etc.) and asserts that head_dim is a multiple of 24.
generate_leech_kernel() – returns a 24×24 orthogonal matrix (placeholder; can be replaced with actual lattice vectors).
LeechAttention – multi‑head attention where Q and K are transformed by the frozen block‑diagonal Leech matrix.
LeechResonanceLoss – combines standard cross‑entropy with the geometric resonance loss.
LeechBlock – a pre‑norm transformer block with LeechAttention and a feed‑forward network.
LeechTransformer – the full model with token/position embeddings, stacked blocks, final norm, and language modelling head.
DreamDecoder – evaluates the resonance of a hidden state against the Leech basis.
leech_generate() – generates tokens step‑by‑step, printing resonance values and status if desired.