Welcome to the D3D12 LinAlg Matrix Preview release!<\/p>\n
Today, we are excited to announce the preview release for the D3D12 Linear Algebra APIs! This feature set unlocks comprehensive hardware acceleration for Matrix-oriented operations across various use cases. Previously, we announced the WaveMMA<\/a> and Cooperative Vectors<\/a> features which supported narrow matrix operation use cases; the LinAlg feature set being announced today subsumes these APIs into a singular set of orthogonal APIs. With today’s announcement, we are enabling developers to both efficiently drive neural rendering techniques directly from individual shader threads in real-time graphics pipelines and utilize higher bandwidth matrix MMA operations for ML and image processing applications, all in a singular combined API.<\/p>\n The application of machine learning techniques is now ubiquitous across the industry. For graphics development, neural network based rendering methods, which we\u2019ve been calling neural rendering, are quickly growing in popularity. At the same time, offloading high bandwidth matrix compute onto the GPU is unquestionably at an all-time high. As such, GPU vendors continue to adopt and expand specialized hardware for matrix operations, and the new LinAlg Matrix APIs put the power of that hardware into your hands!<\/p>\n This blog post is part of the larger SM6.10 preview announcement. See the parent blog post<\/a> for the full feature set. Also\u00a0see the GDC 2026 blog putting this feature in context of the overall ML story for DirectX\u00a0<\/span><\/span>here<\/span><\/span><\/a>.<\/span><\/span><\/p>\n Unlocking efficient use of the GPU\u2019s specialized matrix hardware is the core motivation for the introduction of the LinAlg Matrix APIs. Thanks to the preview process, we were able to go back to the drawing board and evolve the previous design. Thank you for all the feedback and please keep it coming! We will continue to evolve the LinAlg Matrix APIs over the preview period in response to real world feedback.<\/p>\n The new API supports three modes of operations (called Matrix Scopes), motivating different key matrix use cases:<\/p>\n MatrixScope::Thread (previously previewed as Cooperative Vectors)<\/span><\/p>\n A thread-scope matrix is able to fit into a traditional rendering pipeline in place of another shader. A potential example of such is running inference on a neural network trained to compute lighting. Replacing only the single shader enables easy adoption of ML techniques. Inference is understood at a high level by the driver, so it can be mapped to dedicated hardware acceleration.<\/p>\n MatrixScope::Wave (previously previewed as WaveMMA)<\/span><\/p>\n High bandwidth dedicated matrix multiplication hardware is increasingly available in contemporary GPUs. A wave-scope matrix surfaces access to this hardware for complex machine learning and image processing applications. A typical application may employ smaller matrices or manually tiled larger matrices for hardware accelerated matrix-matrix multiplications.<\/p>\n MatrixScope::ThreadGroup<\/span><\/p>\n MatrixScope::ThreadGroup is new to the LinAlg Matrix API.<\/span> It is compatible with all the operations of a wave-scope matrix above serving a different use case. The inputs and weight matrices used in LLM-like networks are much larger than allowed sizes for wave-scope matrices. To serve this case with a wave-scope matrix, manual tiling is mandatory, and for cross-hardware performance, multiple different kernels would be required. Conversely, a threadgroup-scope matrix’s larger size avoids manual tiling. The tiling decision is shifted to the driver, allowing you to ship a single implementation while still retaining optimal tiling.<\/p>\n Shader Model 6.10 introduces high level linear algebra APIs building on top of the Long Vectors<\/a> and Native DXIL Vectors<\/a> features released as part of SM6.9. The high level API is converted into a “mid level” API consumed by the driver.\u00a0 The mid level API maintains high levels of context, enabling the driver to take advantage of the underlying hardware capabilities. Meanwhile, the high level API enables better source level usage rules and fast iteration. This API is centered on the new Matrix type provided as a permissively licensed HLSL source header. Depending on the declaration of a specific Matrix instance, various operations are enabled or disabled at compilation time. For example, MatrixScope::Thread<\/span> is roughly limited to matrix-vector operations, while MatrixScope::Wave<\/span> and MatrixScope::<\/span>ThreadGroup are roughly limited to matrix-matrix operations. You can view the full table of available operations in the LinAlg spec<\/a>.<\/p>\n Below are examples of primary use cases for the There are a couple of D3D methods for converting weight and bias matrix data between formats:<\/p>\n For instance,\u00a0 See\u00a0 LinAlg is part of Shader Model 6.10, currently in preview. This requires:<\/p>\n Device Support<\/strong>:<\/p>\n
\nMotivation<\/h3>\n
\nFeature Overview<\/h3>\n
Code Examples<\/h3>\n
Matrix<\/code> header, each serving a different goal with different available operations. You can find some of these examples\u00a0here<\/a> on GitHub.<\/p>\nCooperative Vectors Example<\/h4>\n
\/\/ Compiled with the line below\r\n\/\/ bin\/dxc -I .\/include\/hlsl -T cs_6_10 -enable-16bit-types coop-vec-example.hlsl\r\n\r\n\/\/ System header containing the LinAlg Matrix APIs\r\n#include <dx\/linalg.h>\r\n\r\n\/\/ The API is nested under dx::linalg. Simplify the example by using it\r\nusing namespace dx::linalg;\r\n\r\n\/\/ Byte Address Buffer to load\/store the matrices\r\nByteAddressBuffer InBuff : register(t0);\r\n\r\n[numthreads(8, 1, 1)]\r\n[shader(\"compute\")]\r\nvoid main() {\r\n \/\/ The Matrix type names can get quite long. Alias them for readability\r\n \/\/ Looking at the template arguments we have:\r\n \/\/ ComponentType::F16 - The matrix holds and F16 type\r\n \/\/ 16 - The M dimension of the matrix is 16\r\n \/\/ 16 - The N dimension of the matrix is 16\r\n \/\/ MatrixUse::A - The Matrix is an \"A\" matrix, so it only fits into the \"A\"\r\n \/\/ slot of various functions\r\n \/\/ MatrixScope::Thread - The Matrix is a \"Thread\" matrix, so it may only be\r\n \/\/ used with \"Thread Matrix\" operations. These are the operations \r\n \/\/ previously covered under the Cooperative Vector API\r\n using MatrixATy =\r\n Matrix<ComponentType::F16, 16, 16, MatrixUse::A, MatrixScope::Thread>;\r\n\r\n \/\/ Setup data for later by loading the matrix and creating null vectors\r\n vector<float16_t, 16> Vec = (vector<float16_t, 16>)0;\r\n vector<float16_t, 16> Bias = (vector<float16_t, 16>)0;\r\n MatrixATy MatA = MatrixATy::Load<MatrixLayout::RowMajor>(\r\n InBuff, 0, \/* Row stride = number of columns * element size *\/ 16 * 2);\r\n\r\n \/\/ Do a F16 Matrix x Vector multiply\r\n vector<float16_t, 16> Layer1 = Multiply<float16_t>(MatA, Vec);\r\n\r\n \/\/ Do a F16 Matrix x Vector multiply with a bias Vector\r\n vector<float16_t, 16> Layer2 = MultiplyAdd<float16_t>(MatA, Layer1, Bias);\r\n\r\n \/\/ Create a reference to an in-memory vector at offset 4096 in InBuff\r\n \/\/ without actually loading it in\r\n VectorRef<ComponentType::F8_E4M3FN, 16> MemBias = {InBuff,\r\n \/*start offset*\/ 4096};\r\n\r\n \/\/ Do a F16 Matrix x Vector multiply with a bias vector stored in memory\r\n vector<float16_t, 16> Layer3 = MultiplyAdd<float16_t>(MatA, Layer2, MemBias);\r\n\r\n \/\/ Create some packed data\r\n vector<uint8_t4_packed, 4> SomeData = (vector<uint8_t4_packed, 4>)0;\r\n\r\n \/\/ Do a MatVecMulAdd but reinterpret the Vec data as F8_F8_E4M3FN with a bias\r\n \/\/ stored in memory\r\n vector<float16_t, 16> Layer4 = MultiplyAdd<float16_t>(\r\n MatA, MakeInterpretedVector<ComponentType::F8_E4M3FN>(SomeData), MemBias);\r\n \/\/ Do a MatVecMulAdd but reinterpret the Vec data as F8_E4M3FN with a regular\r\n \/\/ bias vector\r\n vector<float16_t, 16> Layer5 = MultiplyAdd<float16_t>(\r\n MatA, MakeInterpretedVector<ComponentType::F8_E4M3FN>(SomeData), Bias);\r\n\r\n \/\/ Create some uint data\r\n vector<uint, 16> SomeData2 = (vector<uint, 16>)0;\r\n\r\n \/\/ Do a MatVecMulAdd but convert SomeData2 from a U32 to a F8_E4M3FN first\r\n vector<float16_t, 16> Layer6 = MultiplyAdd<float16_t>(\r\n MatA, Convert<ComponentType::F8_E4M3FN, ComponentType::U32>(SomeData2),\r\n MemBias);\r\n}<\/code><\/pre>\nOuterProduct and InterlockedAccumulate Example<\/h4>\n
\/\/ Compiled with the line below\r\n\/\/ bin\/dxc -I .\/include\/hlsl -T cs_6_10 -enable-16bit-types outerproduct-example.hlsl\r\n\r\n\/\/ System header containing the LinAlg Matrix APIs\r\n#include <dx\/linalg.h>\r\n\r\n\/\/ The API is nested under dx::linalg. Simplify the example by using it\r\nusing namespace dx::linalg;\r\n\r\n\/\/ Byte Address Buffer to load\/store from\r\nRWByteAddressBuffer OutBuff : register(u0);\r\n\r\n[numthreads(8, 1, 1)]\r\n[shader(\"compute\")]\r\nvoid main() {\r\n \/\/ The Matrix type names can get quite long. Alias them for readability\r\n \/\/ Looking at the template arguments we have:\r\n \/\/ ComponentType::F16 - The matrix holds and F16 type\r\n \/\/ 16 - The M dimension of the matrix is 16\r\n \/\/ 8 - The N dimension of the matrix is 8\r\n \/\/ MatrixUse::Accumulator - The Matrix is an \"Accumulator\" matrix, so it\r\n \/\/ only fits into the \"Accumulator\" slot of various functions\r\n \/\/ MatrixScope::Thread - The Matrix is a \"Thread\" matrix, so it may only be\r\n \/\/ used with \"Thread Matrix\" operations. \r\n using MatrixAccumTy = Matrix<ComponentType::F16, 16, 8,\r\n MatrixUse::Accumulator, MatrixScope::Thread>;\r\n\r\n \/\/ Create some F16 vectors with placeholder data\r\n vector<float16_t, 16> VecA = (vector<float16_t, 16>)0;\r\n vector<float16_t, 8> VecB = (vector<float16_t, 8>)0;\r\n\r\n \/\/ Create an Accum matrix by outer producting the two vectors\r\n MatrixAccumTy MatAcc =\r\n OuterProduct<ComponentType::F16>(VecA, VecB);\r\n\r\n \/\/ Atomically accumulate the result into the output buffer\r\n MatAcc.InterlockedAccumulate(OutBuff, 0);\r\n}<\/code><\/pre>\nWave Matrix Example<\/h4>\n
\/\/ Compiled with the line below\r\n\/\/ bin\/dxc -I .\/include\/hlsl -T cs_6_10 -enable-16bit-types linalg-wave.hlsl\r\n\r\n\/\/ System header containing the LinAlg Matrix APIs\r\n#include <dx\/linalg.h>\r\n\r\n\/\/ The API is nested under dx::linalg. Simplify the example by using it\r\nusing namespace dx::linalg;\r\n\r\n\/\/ This shader performs matrix multiplication C = \u03b1*A*B + \u03b2*C\r\n\/\/ where A, B, and C are matrices of dimensions MxK, KxN, and MxN respectively.\r\n\/\/ The shader uses wave-level parallelism to compute tiles of the output matrix\r\n\/\/ C. Each wave computes a TILE_SIZExTILE_SIZE tile of C. The dispatch must\r\n\/\/ allocate waves for each tile of the MxN output matrix.\r\n\r\n\/\/ GEMM constants\r\ncbuffer GemmConstants : register(b0)\r\n{\r\n float alpha; \/\/ Scalar multiplier for A*B\r\n float beta; \/\/ Scalar multiplier for existing C\r\n}\r\n\r\nByteAddressBuffer MatrixA;\r\nByteAddressBuffer MatrixB;\r\nRWByteAddressBuffer MatrixC;\r\n\r\n\/\/ Matrix dimensions - can be configured as needed\r\n#define M 1024 \/\/ Rows in A and C\r\n#define N 1024 \/\/ Columns in B and C\r\n#define K 1024 \/\/ Columns in A, rows in B\r\n#define TILE_SIZE 16\r\n\r\n\/\/ Optimized GEMM using wave-level parallelism\r\n[numthreads(TILE_SIZE, 1, 1)]\r\nvoid main(uint3 group_id : SV_GroupID)\r\n{\r\n \/\/ Matrix type definitions for wave scope\r\n using MatrixATy = Matrix<ComponentType::F16, TILE_SIZE, TILE_SIZE, MatrixUse::A, MatrixScope::Wave>;\r\n using MatrixBTy = Matrix<ComponentType::F16, TILE_SIZE, TILE_SIZE, MatrixUse::B, MatrixScope::Wave>;\r\n using MatrixResultTy = Matrix<ComponentType::F32, TILE_SIZE, TILE_SIZE, MatrixUse::Accumulator, MatrixScope::Wave>;\r\n\r\n \/\/ Calculate tile coordinates for this thread group\r\n uint tile_row = group_id.y;\r\n uint tile_col = group_id.x;\r\n\r\n \/\/ Initialize accumulator\r\n MatrixResultTy c_tile = MatrixResultTy::Splat(0.0f);\r\n\r\n \/\/ Perform tiled matrix multiplication across K dimension\r\n for (uint k = 0; k < K; k += TILE_SIZE)\r\n {\r\n \/\/ Calculate byte offsets for A and B tiles\r\n uint a_offset = ((tile_row * TILE_SIZE) * K + k) * sizeof(half);\r\n uint b_offset = (k * N + (tile_col * TILE_SIZE)) * sizeof(half);\r\n\r\n \/\/ Load A and B tiles for this K iteration using ByteAddressBuffer\r\n MatrixATy a_k_tile = MatrixATy::Load(\r\n MatrixA, a_offset, K * sizeof(half), MatrixLayout::RowMajor);\r\n\r\n MatrixBTy b_k_tile = MatrixBTy::Load(\r\n MatrixB, b_offset, N * sizeof(half), MatrixLayout::RowMajor);\r\n\r\n \/\/ Multiply and accumulate with mixed precision (half inputs -> float accumulation)\r\n c_tile.MultiplyAccumulate(a_k_tile, b_k_tile);\r\n }\r\n\r\n \/\/ Calculate output offset for GEMM equation: C = \u03b1*A*B + \u03b2*C\r\n uint c_offset = ((tile_row * TILE_SIZE) * N + (tile_col * TILE_SIZE)) * sizeof(float);\r\n\r\n \/\/ Load existing C tile\r\n MatrixResultTy c_existing = MatrixResultTy::Load(MatrixC, c_offset, N * sizeof(float), MatrixLayout::RowMajor);\r\n\r\n \/\/ Apply GEMM scaling element-wise: \u03b1*A*B + \u03b2*C\r\n for (uint i = 0; i < c_tile.Length(); i++) {\r\n float ab_val = c_tile.Get(i);\r\n float c_val = c_existing.Get(i);\r\n float result = alpha * ab_val + beta * c_val;\r\n c_tile.Set(i, result);\r\n }\r\n\r\n c_tile.Store(MatrixC, c_offset, N * sizeof(float), MatrixLayout::RowMajor);\r\n}<\/code><\/pre>\nThreadGroup Matrix Example<\/h4>\n
\/\/ Compiled with the line below\r\n\/\/ bin\/dxc -I .\/include\/hlsl -T cs_6_10 -enable-16bit-types linalg-threadgroup.hlsl\r\n\r\n\/\/ System header containing the LinAlg Matrix APIs\r\n#include <dx\/linalg.h>\r\n\r\n\/\/ The API is nested under dx::linalg. Simplify the example by using it\r\nusing namespace dx::linalg;\r\n\r\n\/\/ This shader performs matrix multiplication C = \u03b1*A*B + \u03b2*C\r\n\/\/ where A, B, and C are matrices of dimensions MxK, KxN, and MxN respectively.\r\n\/\/ The shader uses threadgroup-level parallelism to compute tiles of the output\r\n\/\/ matrix C. The GPU driver will generate code to split the matrix into optimal\r\n\/\/ tiles based on the hardware capabilities.\r\n\r\n\/\/ GEMM constants\r\ncbuffer GemmConstants : register(b0)\r\n{\r\n float alpha; \/\/ Scalar multiplier for A*B\r\n float beta; \/\/ Scalar multiplier for existing C\r\n}\r\n\r\nByteAddressBuffer MatrixA;\r\nByteAddressBuffer MatrixB;\r\nRWByteAddressBuffer MatrixC;\r\n\r\n\/\/ Matrix dimensions - can be configured as needed\r\n#define M 1024 \/\/ Rows in A and C\r\n#define N 1024 \/\/ Columns in B and C\r\n#define K 1024 \/\/ Columns in A, rows in B\r\n\r\n\/\/ Optimized GEMM using threadgroup-level parallelism\r\n[numthreads(1024, 1, 1)]\r\nvoid main()\r\n{\r\n \/\/ Matrix type definitions for threadgroup scope\r\n using MatrixATy = Matrix<ComponentType::F16, M, K, MatrixUse::A, MatrixScope::ThreadGroup>;\r\n using MatrixBTy = Matrix<ComponentType::F16, N, K, MatrixUse::B, MatrixScope::ThreadGroup>;\r\n using MatrixResultTy = Matrix<ComponentType::F32, M, N, MatrixUse::Accumulator, MatrixScope::ThreadGroup>;\r\n\r\n MatrixATy a_matrix = MatrixATy::Load(MatrixA, 0, K * sizeof(half), MatrixLayout::RowMajor);\r\n\r\n MatrixBTy b_matrix = MatrixBTy::Load(MatrixB, 0, N * sizeof(half), MatrixLayout::RowMajor);\r\n\r\n \/\/ Load existing C matrix for GEMM equation: C = \u03b1*A*B + \u03b2*C\r\n MatrixResultTy c_existing = MatrixResultTy::Load(MatrixC, 0, N * sizeof(float), MatrixLayout::RowMajor);\r\n\r\n \/\/ Compute A*B\r\n MatrixResultTy ab_result = Multiply<ComponentType::F32>(a_matrix, b_matrix);\r\n\r\n \/\/ Apply GEMM scaling element-wise: \u03b1*A*B + \u03b2*C\r\n for (uint i = 0; i < ab_result.Length(); i++) {\r\n float ab_val = ab_result.Get(i);\r\n float c_val = c_existing.Get(i);\r\n float result = alpha * ab_val + beta * c_val;\r\n ab_result.Set(i, result);\r\n }\r\n\r\n ab_result.Store(MatrixC, 0, N * sizeof(float), MatrixLayout::RowMajor);\r\n}<\/code><\/pre>\n
\nData Preparation<\/h3>\n
enum D3D12_LINEAR_ALGEBRA_MATRIX_LAYOUT {\r\n\u00a0 \u00a0 D3D12_LINEAR_ALGEBRA_MATRIX_LAYOUT_ROW_MAJOR,\r\n\u00a0 \u00a0 D3D12_LINEAR_ALGEBRA_MATRIX_LAYOUT_COLUMN_MAJOR,\r\n\u00a0 \u00a0 D3D12_LINEAR_ALGEBRA_MATRIX_LAYOUT_MUL_OPTIMAL,\r\n\u00a0 \u00a0 D3D12_LINEAR_ALGEBRA_MATRIX_LAYOUT_OUTER_PRODUCT_OPTIMAL\r\n}<\/code><\/pre>\nD3D12_LINEAR_ALGEBRA_MATRIX_LAYOUT_MUL_OPTIMAL<\/span><\/code> is a device-specific layout for optimal use with the Matrix-Vector operations such as MultiplyAdd<\/span><\/code>\u00a0in the code example above.<\/p>\nID3D12DevicePreview<\/span>::<\/span>GetLinearAlgebraMatrixConversionDestinationInfo<\/span>()<\/span><\/code>\u00a0and\u00a0ID3D12CommandListPreview<\/span>::<\/span>ConvertLinearAlgebraMatrix<\/span>()<\/span><\/code> in the D3D LinAlg spec here<\/a>.<\/p>\n
\nGet Running<\/h3>\n
\n