metakernel

The measuring instrument for the kernel-design loop, packaged as a kernel. An agent iterating on CUDA candidates is good at writing code and bad at trusting its own measurements; it needs three things it cannot reliably improvise: trustworthy timing, adversarial correctness comparison, and measured facts about the device it is targeting. metakernel ships those as loadable ops, so the generate, build, measure loop has a fixed, verified instrument at its center while the candidate code churns. It runs anywhere get_kernel runs, including headless CI runners where Nsight and CUPTI are unavailable or unprivileged, which is exactly where agent loops live.

Usage

from kernels import get_kernel
mk = get_kernel("phanerozoic/metakernel", version=1, trust_remote_code=True)

dossier = mk.probe_device()                          # measured device facts
report  = mk.bench(lambda: cand.op(x), iters=200,    # throttle-rejected medians
                   bytes=B, flops=F, dossier=dossier)
verdict = mk.compare(ref_op, cand_op, gen=make_inputs)   # pass/fail with ULP evidence
table   = mk.sweep(factory, {"BLOCK": [64, 128, 256]})   # config search

Every op returns plain dicts and tensors, never prose, so the agent parses results instead of scraping stdout.

API

Return schemas

Every op returns plain data with a fixed schema; nothing needs parsing from prose.

probe_device() returns a dict of entries keyed device, clock, dram_bw, l2_bw, smem_bw, transfer_bw, latency, fma_f32, fma_f64, mma, atomics_f32, launch_overhead, grid_barrier, occupancy_blocks_per_sm, occupancy_knees_fma_tflops. Each measurement entry is {value, units, method, clock_ghz, confidence, note?}; value is a scalar or a flat dict of scalars; mma additionally carries verified_k_lanes: {kind: {executed_k_per_iter, vs_nominal}}; device records ecc, driver, and power_limit_w (None when nvidia-smi is unavailable).

bench() returns {median_ms, sustained_ms, median_ci95_ms: [lo, hi], p05_ms, p95_ms, min_ms, mean_ms, samples, rejected_throttled, graph_captured, clock_ghz: {best, worst}} plus graph_fallback_reason? when capture failed, achieved_gbs?/achieved_tflops?/ arithmetic_intensity? when traffic was declared, and roofline?: {bound, ridge_flops_per_byte, pct_of_roof, peak_dtype} when a dossier was supplied. median_ms is over throttle-clean windows; sustained_ms is over all samples, so power-limited parts are characterized by both.

compare() returns {pass, criterion, worst_trial, max_abs, max_rel, trials: [...], ulp_hist_log2}; each trial is {max_ulp, mean_ulp, first_divergent_index, nan_ref, nan_cand, inf_ref, inf_cand, order_band_ulp, band_score, frac_above_band, worst_elements: [{output, index, ref, cand, ulp}]}.

fuzz_shapes() returns {cases: [...], failed, total, context_lost}; each case is {shape, variant, dtype, status, contiguous?, max_ulp?, repro?: {shape, strides, storage_offset, dtype}, error?, stderr_tail?, logical_elems?, exitcode?} with status in ok | mismatch | exception | context_killed | context_lost | crashed_isolated | skipped_vram | skipped_empty.

sweep() returns a list of rows, each the grid point's params plus {median_ms, rejected_throttled, check?, achieved_gbs?, achieved_tflops?, status?, error?} with status in pruned_race | not_run for points not fully run. read_stamps() returns {phases: [{phase, ticks..., us...}], clock_ghz, blocks_reporting?, blocks_total?}. dossier_drift() returns {entries: [{path, current, pinned, rel}], worst_rel, over_tol, rel_tol, compared, pass}.

How it works

Each dossier entry is produced by a dedicated probe kernel rather than a spec sheet: streaming triads at 4/8/16-byte access widths plus read-only, write-only, and random-gather kernels for bandwidth (one-pass launches for DRAM, in-kernel repeated passes over an L2-resident working set for L2), single-lane Sattolo-cycle pointer chases for latency (a 16 KB ring for L1, a 2 MB TLB-resident ring for L2, and a chain long enough to evict itself, two L2 capacities of touched lines, for DRAM), register-resident FMA chains for the f32 and f64 pipes, register-fragment mma_sync loops per dtype for tensor-core rates (fp8 via inline PTX m16n8k32 on sm89+), an atomic hammer swept over distinct-address counts for the contention curve, empty-launch trains for overhead, and a cooperative-groups kernel for grid-barrier cost. The occupancy knee is timed, not just queried: a single dependent FMA chain per thread runs at forced (threads, dynamic smem) points, and because its issue rate is proportional to resident threads until the pipe saturates, the throughput cliff marks where occupancy stops covering latency.

Every number carries the SM clock it was measured under. The clock primitive is clock64 ticks over wall time from an FMA spinner, so throttled samples are labeled instead of silently poisoning the median: bench probes the clock every clock_stride samples and rejects samples from windows that sagged, reporting the rejection count next to the result.

The comparison ops treat the reference as a distribution, not a point: when the caller supplies an order_shuffle that permutes contractual-noise dimensions (for a matmul, permuting K in both operands changes accumulation order but not the mathematical value), the permuted references bound the legitimate numerical band. The band is applied pointwise: each element of the candidate is scored as ulp / (own_band + 2), so a candidate that is uniformly one percent wrong cannot hide behind one legitimately order-sensitive near-zero element, and a genuinely reordered reduction passes. ULP distances come from the monotone integer mapping of the float bit patterns, computed per element on device.

Certification

The test suite asserts measurement invariants that must hold on any correctly functioning device and runs against the published build variants on multiple architectures. L2 bandwidth exceeds device-memory bandwidth and random gather costs more than streaming; dependent-load latency orders shared memory below L2 below DRAM with L1 between the cache levels; tensor-core rates beat the FMA pipe and hold the structural ratios of the silicon (2:4-sparse at 1.5-2.4x dense fp16, fp16 at 1.5-2.6x tf32, int8 at 1.5-4.8x fp16); a random-operand tile through the MMA fragment path reproduces the reference GEMM exactly; a large cuBLAS GEMM lands within [0.30, 1.05] of the dossier fp16 rate; the f64 pipe lands well below f32, and fp8 reports a number exactly when the architecture has it; the atomic curve rises monotonically with distinct addresses and spans at least 4x end to end; the timed occupancy knee shows a real cliff when dynamic smem starves residency; graph-captured timing does not exceed eager timing on a dispatch-bound train; ulp_diff is exact on constructed cases (nextafter = 1 ULP, -0.0 = +0.0, both-NaN = 0); compare passes a reordered reduction and a K-permuted matmul against their own pointwise order bands and fails corrupted candidates through both routes; fuzz_shapes catches a contiguity-cheating op, exercises multi-argument ops across dtypes, and reports the 2^31 overflow case as run or as skipped_vram, never silently; sweep keeps not-run and failed points visible; the chase probe repeats within 20%.

Measured

RTX 6000 Ada (sm89, 142 SMs, 96 MB L2), single tenant, ECC enabled, driver 595.97, 300 W power limit. Values are the release-pinned dossier (_pinned.py); dossier_drift compares fresh measurements against it.

MMA flops derive from the accumulator read-back with every fragment element observed, never the declared loop shape; verified_k_lanes reports executed against nominal lanes, 100.0% for fp16, bf16, tf32, int8, and sparse. mma_tile_check reproduces the reference GEMM exactly through the same fragment path. The fp8 probe feeds 25% of its nominal K-lanes (vs_nominal 0.25); its entry is an executed-work lower bound at low confidence.

A 1 GiB device copy measures 635 GB/s of combined read+write traffic. An fp16 8192x8192x8192 torch.mm measures 118 TFLOP/s at the median and 183 TFLOP/s at the best sample, all 64 samples rejected as throttled (SM clock 0.81-2.65 GHz at the 300 W limit).

GeForce RTX 3070 Ti Laptop (sm86, 46 SMs, 4 MB L2), measured from the published build variants (torch 2.13, cu126):

Limits

No hardware counters: everything is inferred from timing, clocks, and the occupancy API, and timing-differential inference can misattribute (a bank-conflict signature and an ILP stall can look alike), so dossier entries carry a confidence field. Single device, single tenant: the probes measure whatever the GPU is doing, so a contended device surfaces as rejected samples and sagged clocks rather than an error, and a high rejection count indicates the measurement should be repeated on an idle device. launch_overhead includes torch op dispatch, the overhead a torch-launched candidate pays. The MMA numbers are dense-tile register rates with no memory traffic, an upper bound real kernels reach only when staging hides behind compute. In-process fuzz_shapes cannot survive a candidate that corrupts the CUDA context (the report marks how far it got); isolate=True survives by spawning each group, at the cost of a torch startup per child and the requirement that op/ref/make be picklable. A failed CUDA-graph capture can leave the device RNG wedged for the whole process on some torch versions; bench reports the condition in graph_fallback_reason (timing itself uses no RNG), and capturability of untrusted candidates should be probed in a subprocess. Sparse MMA is counted at dense-equivalent flops, the vendor convention. The fp8 probe feeds 25% of its nominal K-lanes, so its entry is an executed-work lower bound, not the device's fp8 ceiling. Device memory is GDDR6 on both certified cards. Published variants are Linux x86_64; on Windows load_local.py JIT-builds the same source and exposes the identical API.

The position in the loop is the point: kernel-builder compiles candidates, kernels loads them, metakernel measures and judges them, and the agent closes the loop conditioned on measured facts. It is the one component that stays fixed while everything it measures is disposable.

License

Apache-2.0.