phatak-dev

Thank you.

Yes here engram is used as the hashing mechanism for tokens which mainly try to learn vectors for the refusal ones. So it may not be exactly the engram methodology of DeepSeek it's more of used in its spirit.

Yes the gate exist to only apply residual for the tokens focused around refusal so the other tokens are not intercepted.

Feel free to mail to email in my profile. Love to discuss more.

Makes sense.. Just updated my comment to reflect it.. Thanks for making it clear.

Afaiu the projection works with the weight update. But here the vector is getting added to residual stream from engram lookup not as direct updat. So the approach is little different

[Edited] Yes correct.

In the original paper, they measure how much refusal is actively present in the current token and subtract only that specific amount.

In my early baseline step, I used a simpler approach where I just subtracted a fixed vector across the board. This is just to see if the approach is even feasible.

That's actually the main reason I moved to the Engram module, I wanted a smartness that reads the context and turns steering on only when refusal triggers pop up, leaving normal tokens untouched.

These seems to come from disqus integration.