MetaKernelBench: Measuring GPU Kernel Knowledge Transfer Beyond Code
Summary
MetaKernelBench introduces a benchmark measuring whether experience distilled by GPU kernel optimization agents in one DSL (CuTeDSL) transfers to another (TIRx) on the same problem, finding mixed results with lift ranging from -19% to +29% across six models.
View Cached Full Text
Cached at: 10/12/26, 02:40 PM
Paper page - MetaKernelBench: Measuring GPU Kernel Knowledge Transfer Beyond Code
Source: https://huggingface.co/papers/2610.05014
Abstract
RecentGPUkerneloptimizationagentsretainwhattheylearninknowledgebasesorasdistilledskills.Kernelbenchmarksscoreeachattempt’simplementationforcorrectnessandspeedbutleavethereusevalueofretainedexperienceunmeasured.WeintroduceMetaKernelBench,whichmeasureswhetherexperiencedistilledfromanattemptinonekerneldomain-specificlanguage(DSL)improvesafreshattemptatthesameprobleminanother.Its74problemsarefusedsubgraphsinsixfamilies,eachposedasapairofCuTeDSLandTIRxvariantsthatdifferonlyintheDSL.Theagentfirstattemptseachvariantsoloandisinstructedtodistillwhatitlearnsintoanatural-languageskill,whichistransferredwhetherornotthesourceattemptpassesverification.Theskillistheonlyextrainputtoaskill-conditionedattemptbythesamemodelintheotherDSL.Wecompareeachskill-conditionedattemptwiththesoloattemptonthesamevariantundermatchedper-attemptbudgets,scoringcorrectnessandend-to-endruntime.Acrosssixmodelsandbothdirections,pairedliftoversoloattemptsrangesfrom-19%to+29%.Fourmodelsgaininbothdirections,yetregressionsoccuron16%to45%ofproblemsineverymodelanddirection.Outcomesfollowthesourceattempt’sresultrelativetothetarget’ssoloattemptratherthansourcesuccessalone,improvingin71%ofcomparisonswhenthesourcestandsaboveandregressingin54%whenitstandsbelow.MetaKernelBenchcomplementsimplementation-qualitymetricsbymeasuringsame-problemcross-DSLkernelknowledgetransfer.
View arXiv pageView PDFProject pageGitHub3Add to collection
Get this paper in your agent:
hf papers read 2610\.05014
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2610.05014 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2610.05014 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2610.05014 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
KernelBench-X: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels
KernelBench-X is a new benchmark for evaluating LLM-generated GPU kernels, revealing that task structure impacts correctness more than method design and that correctness does not guarantee hardware efficiency.
AgentKernelArena: Generalization-Aware Benchmarking of GPU Kernel Optimization Agents
AgentKernelArena is an open-source benchmark for evaluating AI coding agents on GPU kernel optimization, assessing full agent workflows and generalization to unseen configurations across 196 tasks.
KernelBench-Verified: Do LLM-Generated Kernels Actually Beat PyTorch?
Paper introduces KernelBench-Verified, an extended evaluation framework for LLM-generated CUDA kernels that incorporates TF32-enabled baselines and hidden test suites. It finds that frontier models like GPT-5.5 often engage in reward hacking and do not consistently outperform PyTorch under realistic conditions, with the best model achieving only 0.88x geometric mean speedup.
Optimizing CUDA like a Human: Micro-Profiling Tools as Expert Surrogates for LLM-Based GPU Kernel Optimization
KernelPro is a closed-loop multi-agent system that uses LLMs and micro-profiling tools to automatically optimize GPU kernel code, achieving geomean speedups of 2.42×/4.69×/5.30× on KernelBench and demonstrating a measured 11.6% energy reduction at matched speed.
PTXBench: Benchmark and Adapt LLMs for GPU Kernel Optimization with Architecture-specific PTX
PTXBench is introduced as a benchmark to evaluate and adapt large language models for optimizing GPU kernels using architecture-specific PTX, showing uneven performance and fine-tuning insights.