Comment

Comment by pilotneko

If you read the model card, Qwen3-Next can be extended to 1M context length with YaRN.

> Qwen3-Next natively supports context lengths of up to 262,144 tokens. For conversations where the total length (including both input and output) significantly exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively. We have validated the model's performance on context lengths of up to 1 million tokens using the YaRN method.

Source: https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct#proc...

parent pilotneko 1 year ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton