From Layout Algebra to Intel Xe XMX GEMM

The one picture to keep in your head

CuTe is a C++ template library for describing and operating on hierarchical multidimensional layouts of data and execution. Its central abstractions are Layout and Tensor; Intel's Xe path adds Xe-specific copy and MMA atoms that plug into the same CuTe layout machinery.

CUDA/CUTLASS words vs Intel/SYCL words

The SYCL*TLA documentation inherits substantial terminology from upstream CUTLASS. The table below is a programming analogy, not a claim that NVIDIA and Intel hardware structures are physically identical.

Layout = Shape + Stride = coordinate → index mapping

A CuTe Layout is a pair of Shape and Stride. Semantically, it maps a logical coordinate inside the shape to an index. Shapes and strides can contain static compile-time integers, dynamic runtime integers, or hierarchical tuples of them.

// 4 × 4; mode 0 has stride 1
using L = Layout<
  Shape<_4,_4>,
  Stride<_1,_4>
>;

Why layout algebra matters

Tensor = Engine + Layout

A CuTe Tensor combines a layout with an engine. The engine is a wrapper around an iterator or array; the layout maps logical coordinates to indices used to access that engine. Data may live in global memory, shared/local memory, register memory, or be transformed/generated on the fly.

CuTe GEMM mode convention

For GEMM, CuTe uses the logical tensor modes A(M,K), B(N,K), and C(M,N). This differs from the conventional BLAS view of B as (K,N). Keeping K as the second mode of both inputs lets generic implementations reduce over the same logical mode for A and B. The physical memory layout is still determined by each tensor's strides.

What the Intel Xe path adds to upstream CuTe

DPAS: the matrix operation underneath gemm()

XE_DPAS_TT<M,...> is the current CuTe Xe atom wrapping a DPAS matrix multiply-accumulate. In the documented API, M is 1–8, N is fixed at 16, and K = 256 / max(bit-width(A), bit-width(B)).

D[M × 16] += A[M × K] × B[K × 16]

From one atom to a tiled MMA

// Documented BF16 starting configuration
using WGTile   = Shape<_256,_256,_32>;
using SGLayout = Layout<Shape<_8,_4,_1>,
                        Stride<_4,_1,_0>>;

using Tiled = typename TiledMMAHelper<
    MMA_Atom<XE_DPAS_TT<8, float, bfloat16_t>>,
    Layout<WGTile>,
    SGLayout
>::TiledMMA;

This BF16 starting configuration uses a 256×256 output tile and 32 subgroups arranged as an 8×4 grid over M×N. In the pure xe_gemm.cpp tutorial, however, the K tile is selected programmatically as either op.K or 2*op.K depending on operand type/layout conditions; _32 is therefore a useful BF16 baseline, not a universal K tile.

Walk through the Intel Xe CuTe GEMM dataflow

The K-loop as a pipeline

// Guarded teaching version of the documented mainloop
int prefetch_k = 0;

for (; prefetch_k < min(prefetch_dist, k_tile_count); ++prefetch_k) {
  prefetch(prefetch_a, A_prefetch_tile(prefetch_k));
  prefetch(prefetch_b, B_prefetch_tile(prefetch_k));
}

for (int k_tile = 0; k_tile < k_tile_count; ++k_tile, ++prefetch_k) {
  barrier_arrive(barrier_scope);

  copy(copy_a, A_tile(k_tile), a_copy_regs);
  copy(copy_b, B_tile(k_tile), b_copy_regs);

  if (prefetch_k < k_tile_count) {
    prefetch(prefetch_a, A_prefetch_tile(prefetch_k));
    prefetch(prefetch_b, B_prefetch_tile(prefetch_k));
  }

  reorder(a_copy_regs, a_mma_regs);
  reorder(b_copy_regs, b_mma_regs);
  gemm(mma, a_mma_regs, b_mma_regs, accum);

  barrier_wait(barrier_scope);
}

copy(copy_c, accum, C_partition);

Understand Xe 2D block copies before tuning GEMM

Xe 2D block I/O is subgroup-cooperative: 16 work-items cooperate on a rectangular memory region. CuTe exposes these operations through parameterized copy atoms and maps their results into subgroup-distributed register fragments.

Current 2D-operation limits highlighted by the performance guide

Tune only after you can explain the dataflow

Correctness-first tuning sequence

1. Verify the required 16-work-item subgroup configuration.
2. Verify 2D-copy base-pointer, pitch, width, and shape constraints.
3. Profile whether the kernel is bandwidth-bound or compute-bound.
4. Adjust M/N/K tile dimensions one change at a time and watch GRF spill.
5. Adjust prefetch depth and measure rather than assuming more stages are better.
6. Inspect whether reorder() is compiled away or remains a shuffle.

How to read examples/cute/tutorial/xe_gemm.cpp

For a first pass, follow the dataflow instead of reading every template declaration in file order:

Three source-faithful lines that unlock the file

auto wg_tile = mma.tile_mnk();          // work-group M/N/K tile
auto thr_mma = mma.get_slice(local_id); // this work-item's MMA slice
gemm(mma, tCrA, tCrB, tCrC);           // DPAS-backed accumulation

Source trail and audit scope

Technical statements in this page were rechecked against the current main branch of Intel SYCL*TLA on August 17, 2026, plus the upstream CuTe GEMM convention where needed. The code blocks labeled as teaching versions are intentionally simplified; source-faithful snippets are labeled explicitly.

• Intel — CuTe in SYCL*TLA: Intel Overview
• Intel — Intel Xe GPU GEMM Companion
• Intel — Performance Tuning Guide for CuTe on Xe
• CuTe Layout tutorial
• CuTe Layout Algebra tutorial
• CuTe Tensor tutorial
• Intel Xe 2D Copy Operations (legacy naming, still useful for data distribution)
• Reference source — examples/cute/tutorial/xe_gemm.cpp
• Current Xe MMA atom definitions — mma_xe.hpp
• Current parameterized Xe 2D copy atoms — copy_xe_2d.hpp
• Xe 2D copy selection/traits — copy_traits_xe_2d.hpp
• Xe register reorder implementation — reorder_xe.hpp
• Upstream CuTe dense GEMM tutorial — A(M,K), B(N,K), C(M,N) convention
• Khronos — SPV_INTEL_2d_block_io