Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA

[2608.14560] Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA Skip to main content Search Submit Donate Log in Search arXiv Press Enter to search · Advanced search Computer Science > Distributed, Parallel, and Cluster Computing arXiv:2608.14560 (cs) [Submitted on 25 May 2026] Title: Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA Authors: Mao Luo , Hongbin Li , Feng Lin , Hanling Yi , Zhe Huang View a PDF of the paper titled Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA, by Mao Luo and 4 other authors View PDF HTML (experimental) Abstract: We study whether general-purpose code agents can produce state-of-the-art GPU kernels without any manually written CUDA code. We investigate this question using representative workloads from FlashInfer-Bench, focusing on the Fused MoE, DSA TopK Indexer, and DSA Sparse Attention, and evaluate all generated kernels under the correctness-gated FlashInfer-Bench protocol on NVIDIA B200 GPUs. Starting from the PyTorch implementations, workload definitions, benchmark commands, and a compact set of CUDA optimization skills, we build a kernel optimization workflow in Houmao, a multi-agent orchestration framework for heterogeneous coding agents, to generate, debug, profile, and optimize the kernels. Humans remain strictly in an orchestration role: defining the workflow, enforcing correctness and anti-hacking constraints, supplying key references, and redirecting the search when progress stalls, without reviewing or editing the kernel code itself. Across roughly 1.9 billion agent tokens, the resulting kernels achieve speedups of 92.68x on Fused MoE, 1101.02x on DSA TopK Indexer, and 181.35x on DSA Sparse Attention relative to the PyTorch reference implementations, while also significantly outperforming the corresponding FlashInfer baselines. In the official evaluation of the MLSys 2026 FlashInfer AI Kernel Generation Contest, our generated Fused MoE kernel achieves a 1.71x speedup over the FlashInfer baseline, exceeding the top result of the Fused MoE agent-assisted track, which reports a 1.68x speedup. These results suggest that, under a disciplined correctness-first workflow, code agents can serve as effective autonomous optimizers for modern GPU kernel development. Comments: Technical report on AI code generation for practical GPU kernels on NVIDIA Blackwell GPUs Subjects: Distributed, Parallel, and Cluster Computing (cs.DC) ; Software Engineering (cs.SE) Cite as: arXiv:2608.14560 [cs.DC] (or arXiv:2608.14560v1 [cs.DC] for this version) https://doi.org/10.48550/arXiv.2608.14560 Focus to learn more arXiv-issued DOI via DataCite Submission history From: Feng Lin [ view email ] [v1] Mon, 25 May 2026 02:50:41 UTC (4,728 KB) Full-text links: Access Paper: View a PDF of the paper titled Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA, by Mao Luo and 4 other authors View PDF HTML (experimental) TeX Source view license Current browse context: cs.DC < prev | next > new | recent | 2026-08 Change to browse by: cs cs.SE References & Citations NASA ADS Google Scholar Semantic Scholar export BibTeX citation Loading... BibTeX formatted citation × loading... Data provided by: Bookmark Bibliographic Tools Bibliographic and Citation Tools Bibliographic Explorer Toggle Bibliographic Explorer ( What is the Explorer? ) Connected Papers Toggle Connected Papers ( What is Connected Papers? ) Litmaps Toggle Litmaps ( What is Litmaps? ) scite.ai Toggle scite Smart Citations ( What are Smart Citations? ) Code, Data, Media Code, Data and Media Associated with this Article alphaXiv Toggle alphaXiv ( What is alphaXiv? ) Links to Code Toggle CatalyzeX Code Finder for Papers ( What is CatalyzeX? ) DagsHub Toggle DagsHub ( What is DagsHub? ) GotitPub Toggle Gotit.pub ( What is GotitPub? ) Huggingface Toggle Hugging Face ( What is Huggingface? ) ScienceCast Toggle ScienceCast ( What is ScienceCast? ) Demos Demos Replicate Toggle Replicate ( What is Replicate? ) Spaces Toggle Hugging Face Spaces ( What is Spaces? ) Spaces Toggle TXYZ.AI ( What is TXYZ.AI? ) Related Papers Recommenders and Search Tools Link to Influence Flower Influence Flower ( What are Influence Flowers? ) Core recommender toggle CORE Recommender ( What is CORE? ) Author Venue Institution Topic About arXivLabs arXivLabs: experimental projects with community collaborators arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website. Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them. Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs . Which authors of this paper are endorsers? | Disable MathJax ( What is MathJax? ) We gratefully acknowledge support from our major funders , member institutions , , and all contributors. About · Help · Contact · Subscribe · Copyright · Privacy · Accessibility · Operational Status (opens in new tab) Major funding support from

Related documents

• Agentic Kernel Optimization: Generating State-of-the-Art GPU Kernels Without Hand-Written CUDA#610063
• AMDKernelVault: Large-Scale Datasets and Agentic Training for AMD GPU Kernel Optimization#1069969
• KernelArc: A Multi-Agent Framework for GPU Kernel Optimization#658877
• KernelArc: A Multi-Agent Framework for GPU Kernel Optimization#656741
• GPU-Accelerated First-Order Method with Randomized Sampling for Binary Integer Programs#1062806