Comment Comment by steer_dev AI Slop (apologies, filler) isn't just annoying; it is In-Band Signaling Noise. The model mixes control signals (persona) with data (payload).Prompting ("Be concise") is brittle. I switched to Shannon Entropy.The Hypothesis: Code/Data is mathematically "messy" (High Entropy). Slop is smooth and predictable (Low Entropy).I wrote a filter that blocks responses if entropy dips below ~3.5.The Payoff: It captures the blocked slop as a dataset for DPO. I use the math to gather data today so I can fine-tune a natively quiet model tomorrow.Repo: https://github.com/imtt-dev/steer parent steer_dev 9 months ago
Prompting ("Be concise") is brittle. I switched to Shannon Entropy.
The Hypothesis: Code/Data is mathematically "messy" (High Entropy). Slop is smooth and predictable (Low Entropy).
I wrote a filter that blocks responses if entropy dips below ~3.5.
The Payoff: It captures the blocked slop as a dataset for DPO. I use the math to gather data today so I can fine-tune a natively quiet model tomorrow.
Repo: https://github.com/imtt-dev/steer