
DZ
Dan Zhao, Karthikeyan Sankaralingam, Christos Kozyrakis, Qijing Huang
· 1 min read
ResearcharXiv cs.AI
PerfReasoning: How Well Do LLMs Reason on Hardware Performance?
arXiv:2609.04476v2 Announce Type: replace
Abstract: Performance modeling is central to hardware design and software optimization, yet constructing these models requires structured reasoning about computation, data reuse, storage, and movement. We introduce PerfReasoning, a benchmark that evaluates LLMs both as direct performance reasoners and as generators of analytical performance-model code. Given workload, architecture, and mapping specifications, models compare mappings and predict off-chip traffic and buffer requirements. The strongest closed-source models exceed 90% on reasoning-based Q&A, and the best open-weight model reaches 82.4%. However, model construction is substantially harder: while GPT-5.6 Sol exceeds 80% pass rate, all other model configurations average below 45% and vary markedly across runs. Task-specific RL raises a 4B model's mapping-reasoning accuracy by 15.7 points, whereas feedback-free multi-round self-revision prompting is not reliably effective. PerfReasoning exposes the gap between plausible architectural reasoning and reliable performance-model construction. We will publicly release the benchmark to support reproducible evaluation and track future progress.
Original source
This story was published by arXiv cs.AI and written by Dan Zhao, Karthikeyan Sankaralingam, Christos Kozyrakis, Qijing Huang. SyncAI.news shows a preview; the complete article is on the publisher's site.
Read the full story on arxiv.org


