Comment by wwizo

You guys rock! I'm very curious how will this perform against real word data, where small nuance matters. Also have you tested it beyond 128K context window?