Reaction
A 400K-context test reportedly reached 56 tokens per second
A user says they used YaRN x2 and planned to test up to 1M context on October 16.
TLDR
A user reported testing 400K context at 56 tokens per second using YaRN x2. They described the upgrade as very fast, with a significantly larger KV pool and up to 1M context, and said @MiaAI_lab's TensorFold recipe was 20% faster. On October 16, they said they planned to test up to 1M context that night.
Combined views
22.2K
2 Sources, first seen ago
