I've been experimenting with using J-Space (and final hidden layers) to extract the semantic meaning of words to improve TTS output accuracy using Qwen and Gemma models. My goal is to either map to an alternative token set where heteronyms are preserved or to output parenthesized IPA annotations for ambiguous words (with standard tokens). It's interesting to me that LLMs preserve this data throughout their processing but discard it in the final output.
I've also looked into extracting actions from J-Space to short-circuit a local assistant on low-end hardware. Are there any resources on how to do this training with inexpensive H100 instances (~$8/hr)? I would be opening up the final weights for Qwen/Gemma layers.
I've also looked into extracting actions from J-Space to short-circuit a local assistant on low-end hardware. Are there any resources on how to do this training with inexpensive H100 instances (~$8/hr)? I would be opening up the final weights for Qwen/Gemma layers.