Ask HN: At ~165k tokens, does Opus 4.6 1M outperform Opus 4.6 200k?
hackernews
|
|
{'이벤트': '📰', '머신러닝/연구': '📰', '하드웨어/반도체': '📰', '취약점/보안': '📰', '기타 AI': '📰', 'AI 딜': '📰', 'AI 모델': '📰', 'AI 서비스': '📰', 'discount': '📰', 'news': '📰', 'review': '📰', 'tip': '📰'} 머신러닝/연구
#anthropic
#claude
#claude opus
#머신러닝/연구
#모델 크기 추정
요약
약 16.5만 토큰을 처리할 때 컨텍스트 윈도우 크기가 다른 Opus 200k와 1M 모델 중 어느 것이 더 성능이 우수한지에 대한 질문이 제기되었습니다. Anthropic은 양자가 동일한 모델이라고 주장하지만, Claude Code 소스 코드에서는 별개의 변형으로 취급하는 등 혼선이 있는 상황입니다. 현재 기존 테스트는 200K 이하에서의 동일 여부만 확인했을 뿐 정확한 A/B 테스트가 부족하여, API 수준에서의 명확한 성능 비교 벤치마크가 필요합니다.
왜 중요한가
본문
Here is a question for which I cannot find an answer, and cannot yet afford to answer myself:<p>NoLiMa [0] and "context rot" [1] would indicate that with a ~165k request, Opus 200k would suck, and Opus 1M would be better (as a lower percentage of the context window was used)... but they are the same model, right? However, there are practical inference deployment differences that could change the whole paradigm, right? I am so confused.</p><p>Anthropic says it's the same model [2]. But, Claude Code's own source treats them as distinct variants with separate routing [3]. Closest test I found [4] asserts they're identical below 200K but it never actually A/B tests, correct?</p><p>Inside Claude Code it's probably not testable, right? According to this issue [5], the CLI is non-deterministic for identical inputs, and agent sessions branch on tool-use. Would need a clean API-level test.</p><p><i>The API level test is what I really want to know for the Claude based features in my own apps. Is there a real benchmark for this?</i></p><p>I have reached the limits of my understanding on this problem. If what I am trying to say makes any sense, any help would be greatly appreciated.</p><p>If anyone could help me ask the question better, that would also be appreciated.</p><p>[0] https://arxiv.org/abs/2502.05167</p><p>[1] https://research.trychroma.com/context-rot</p><p>[2] https://claude.com/blog/1m-context-ga</p><p>[3] https://github.com/anthropics/claude-code/issues/35545</p><p>[4] https://www.claudecodecamp.com/p/claude-code-1m-context-window</p><p>[5] https://github.com/anthropics/claude-code/issues/3370</p>