LLMs × databases × GPU kernels
DataKernelBench
Accepted at EMNLP 2026
Can LLMs optimize database queries on GPUs?
GPU-accelerated data processing is expanding as data volumes grow. At the same time, coding LLMs can increasingly generate low-level GPU programs.
DataKernelBench evaluates LLMs on a novel task: translating SQL queries into optimized CUDA and Triton kernels via a validated PyTorch intermediate representation, then measuring whether they execute faster on GPUs.
IBM Research · TU Wien
Benchmark results
How well do current models perform?
We evaluate ten proprietary and open-weight LLMs across CUDA and Triton on all 22 TPC-H queries at SF10 using one NVIDIA H100, at core and full-query optimization. The strongest configuration achieves 2.11× overall speedup with a 100% pass rate.
Beyond GPU memory. On TPC-H SF100, Dask-cuDF loads partitions on demand across four H100 GPUs. All 22 optimized implementations pass, and the complete workload runs 2.54× faster than the partitioned TorchPlan baseline.
The paper examines where these gains come from—and why they vary so much across models and queries.
TPC-H Q6
See one example.
Follow TPC-H Q6 from SQL to its validated TorchPlan and complete LLM-generated Triton and CUDA implementations. Expand any file to inspect the exact source.
01 SQL query
Open to load the complete source.02 TorchPlan
Open to load the complete source.03 Triton kernel
Open to load the complete source.04 CUDA kernel
Open to load the complete source.