diff --git a/docs/gpu-arch/gpu-arch.md b/docs/gpu-arch/gpu-arch.md deleted file mode 100644 index a1ae28b4..00000000 --- a/docs/gpu-arch/gpu-arch.md +++ /dev/null @@ -1,84 +0,0 @@ - - - - - - -(gpu-arch-documentation)= - -# GPU architecture documentation - -:::::{grid} 1 1 2 2 -:gutter: 1 - -:::{grid-item-card} -**AMD Instinct MI350 series** - -Review hardware aspects of the AMD Instinct™ MI350 series of GPU accelerators and the CDNA™ 4 -architecture. - -* AMD Instinct™ MI350 microarchitecture **Coming Soon!** -* [AMD Instinct MI350/CDNA4 ISA](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-cdna4-instruction-set-architecture.pdf) -* [White paper](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-4-architecture-whitepaper.pdf) -* Performance counters **Coming Soon!** -::: - -:::{grid-item-card} -**AMD Instinct MI300 series** - -Review hardware aspects of the AMD Instinct™ MI300 series of GPU accelerators and the CDNA™ 3 -architecture. - -* [AMD Instinct™ MI300 microarchitecture](./mi300.md) -* [AMD Instinct MI300/CDNA3 ISA](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-mi300-cdna3-instruction-set-architecture.pdf) -* [White paper](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-3-white-paper.pdf) -* [Performance counters](./mi300-mi200-performance-counters.rst) -::: - -:::{grid-item-card} -**AMD Instinct MI200 series** - -Review hardware aspects of the AMD Instinct™ MI200 series of GPU accelerators and the CDNA™ 2 -architecture. - -* [AMD Instinct™ MI250 microarchitecture](./mi250.md) -* [AMD Instinct MI200/CDNA2 ISA](https://www.amd.com/system/files/TechDocs/instinct-mi200-cdna2-instruction-set-architecture.pdf) -* [White paper](https://www.amd.com/content/dam/amd/en/documents/instinct-business-docs/white-papers/amd-cdna2-white-paper.pdf) -* [Performance counters](./mi300-mi200-performance-counters.rst) - -::: - -:::{grid-item-card} -**AMD Instinct MI100** - -Review hardware aspects of the AMD Instinct™ MI100 series of GPU accelerators and the CDNA™ 1 -architecture. - -* [AMD Instinct™ MI100 microarchitecture](./mi100.md) -* [AMD Instinct MI100/CDNA1 ISA](https://www.amd.com/system/files/TechDocs/instinct-mi100-cdna1-shader-instruction-set-architecture%C2%A0.pdf) -* [White paper](https://www.amd.com/content/dam/amd/en/documents/instinct-business-docs/white-papers/amd-cdna-white-paper.pdf) - -::: - -:::{grid-item-card} -**RDNA** - -* [AMD RDNA4 ISA](https://www.amd.com/content/dam/amd/en/documents/radeon-tech-docs/instruction-set-architectures/rdna4-instruction-set-architecture.pdf) -* [AMD RDNA3 ISA](https://www.amd.com/system/files/TechDocs/rdna3-shader-instruction-set-architecture-feb-2023_0.pdf) -* [AMD RDNA2 ISA](https://www.amd.com/system/files/TechDocs/rdna2-shader-instruction-set-architecture.pdf) -* [AMD RDNA ISA](https://www.amd.com/system/files/TechDocs/rdna-shader-instruction-set-architecture.pdf) - -::: - -:::{grid-item-card} -**Older architectures** - -* [AMD Instinct MI50/Vega 7nm ISA](https://www.amd.com/system/files/TechDocs/vega-7nm-shader-instruction-set-architecture.pdf) -* [AMD Instinct MI25/Vega ISA](https://www.amd.com/system/files/TechDocs/vega-shader-instruction-set-architecture.pdf) -* [AMD GCN3 ISA](https://www.amd.com/system/files/TechDocs/gcn3-instruction-set-architecture.pdf) -* [AMD Vega Architecture White Paper](https://en.wikichip.org/w/images/a/a1/vega-whitepaper.pdf) - -::: - -::::: diff --git a/docs/gpu-arch/mi100.md b/docs/gpu-arch/mi100.md deleted file mode 100644 index ae87b505..00000000 --- a/docs/gpu-arch/mi100.md +++ /dev/null @@ -1,95 +0,0 @@ ---- -myst: - html_meta: - "description lang=en": "Learn about the AMD Instinct MI100 series architecture." - "keywords": "Instinct, MI100, microarchitecture, AMD, ROCm" ---- - -# AMD Instinct™ MI100 microarchitecture - -The following image shows the node-level architecture of a system that -comprises two AMD EPYC™ processors and (up to) eight AMD Instinct™ accelerators. -The two EPYC processors are connected to each other with the AMD Infinity™ -fabric which provides a high-bandwidth (up to 18 GT/sec) and coherent links such -that each processor can access the available node memory as a single -shared-memory domain in a non-uniform memory architecture (NUMA) fashion. In a -2P, or dual-socket, configuration, three AMD Infinity™ fabric links are -available to connect the processors plus one PCIe Gen 4 x16 link per processor -can attach additional I/O devices such as the host adapters for the network -fabric. - -![Structure of a single GCD in the AMD Instinct MI100 accelerator](../images/gpu-arch/image004.png "Node-level system architecture with two AMD EPYC™ processors and eight AMD Instinct™ accelerators.") - -In a typical node configuration, each processor can host up to four AMD -Instinct™ accelerators that are attached using PCIe Gen 4 links at 16 GT/sec, -which corresponds to a peak bidirectional link bandwidth of 32 GB/sec. Each hive -of four accelerators can participate in a fully connected, coherent AMD -Instinct™ fabric that connects the four accelerators using 23 GT/sec AMD -Infinity fabric links that run at a higher frequency than the inter-processor -links. This inter-GPU link can be established in certified server systems if the -GPUs are mounted in neighboring PCIe slots by installing the AMD Infinity -Fabric™ bridge for the AMD Instinct™ accelerators. - -## Microarchitecture - -The microarchitecture of the AMD Instinct accelerators is based on the AMD CDNA -architecture, which targets compute applications such as high-performance -computing (HPC) and AI & machine learning (ML) that run on everything from -individual servers to the world's largest exascale supercomputers. The overall -system architecture is designed for extreme scalability and compute performance. - -![Structure of the AMD Instinct accelerator (MI100 generation)](../images/gpu-arch/image005.png "Structure of the AMD Instinct accelerator (MI100 generation)") - -The above image shows the AMD Instinct accelerator with its PCIe Gen 4 x16 -link (16 GT/sec, at the bottom) that connects the GPU to (one of) the host -processor(s). It also shows the three AMD Infinity Fabric ports that provide -high-speed links (23 GT/sec, also at the bottom) to the other GPUs of the local -hive. - -On the left and right of the floor plan, the High Bandwidth Memory (HBM) -attaches via the GPU memory controller. The MI100 generation of the AMD -Instinct accelerator offers four stacks of HBM generation 2 (HBM2) for a total -of 32GB with a 4,096bit-wide memory interface. The peak memory bandwidth of the -attached HBM2 is 1.228 TB/sec at a memory clock frequency of 1.2 GHz. - -The execution units of the GPU are depicted in the above image as Compute -Units (CU). There are a total 120 compute units that are physically organized -into eight Shader Engines (SE) with fifteen compute units per shader engine. -Each compute unit is further sub-divided into four SIMD units that process SIMD -instructions of 16 data elements per instruction. This enables the CU to process -64 data elements (a so-called 'wavefront') at a peak clock frequency of 1.5 GHz. -Therefore, the theoretical maximum FP64 peak performance is 11.5 TFLOPS -(`4 [SIMD units] x 16 [elements per instruction] x 120 [CU] x 1.5 [GHz]`). - -![Block diagram of an MI100 compute unit with detailed SIMD view of the AMD CDNA architecture](../images/gpu-arch/image006.png "An MI100 compute unit with detailed SIMD view of the AMD CDNA architecture") - -The preceding image shows the block diagram of a single CU of an AMD Instinct™ -MI100 accelerator and summarizes how instructions flow through the execution -engines. The CU fetches the instructions via a 32KB instruction cache and moves -them forward to execution via a dispatcher. The CU can handle up to ten -wavefronts at a time and feed their instructions into the execution unit. The -execution unit contains 256 vector general-purpose registers (VGPR) and 800 -scalar general-purpose registers (SGPR). The VGPR and SGPR are dynamically -allocated to the executing wavefronts. A wavefront can access a maximum of 102 -scalar registers. Excess scalar-register usage will cause register spilling and -thus may affect execution performance. - -A wavefront can occupy any number of VGPRs from 0 to 256, directly affecting -occupancy; that is, the number of concurrently active wavefronts in the CU. For -instance, with 119 VGPRs used, only two wavefronts can be active in the CU at -the same time. With the instruction latency of four cycles per SIMD instruction, -the occupancy should be as high as possible such that the compute unit can -improve execution efficiency by scheduling instructions from multiple -wavefronts. - -:::{table} Peak-performance capabilities of MI100 for different data types. -:name: mi100-perf -| Computation and Data Type | FLOPS/CLOCK/CU | Peak TFLOPS | -| :------------------------ | :------------: | ----------: | -| Vector FP64 | 64 | 11.5 | -| Matrix FP32 | 256 | 46.1 | -| Vector FP32 | 128 | 23.1 | -| Matrix FP16 | 1024 | 184.6 | -| Matrix BF16 | 512 | 92.3 | - -::: diff --git a/docs/gpu-arch/mi250.md b/docs/gpu-arch/mi250.md deleted file mode 100644 index 9da0d965..00000000 --- a/docs/gpu-arch/mi250.md +++ /dev/null @@ -1,134 +0,0 @@ ---- -myst: - html_meta: - "description lang=en": "Learn about the AMD Instinct MI250 series architecture." - "keywords": "Instinct, MI250, microarchitecture, AMD, ROCm" ---- - -# AMD Instinct™ MI250 microarchitecture - -The microarchitecture of the AMD Instinct MI250 accelerators is based on the -AMD CDNA 2 architecture that targets compute applications such as HPC, -artificial intelligence (AI), and machine learning (ML) and that run on -everything from individual servers to the world’s largest exascale -supercomputers. The overall system architecture is designed for extreme -scalability and compute performance. - -The following image shows the components of a single Graphics Compute Die (GCD) of the CDNA 2 architecture. On the top and the bottom are AMD Infinity Fabric™ -interfaces and their physical links that are used to connect the GPU die to the -other system-level components of the node (see also Section 2.2). Both -interfaces can drive four AMD Infinity Fabric links. One of the AMD Infinity -Fabric links of the controller at the bottom can be configured as a PCIe link. -Each of the AMD Infinity Fabric links between GPUs can run at up to 25 GT/sec, -which correlates to a peak transfer bandwidth of 50 GB/sec for a 16-wide link ( -two bytes per transaction). Section 2.2 has more details on the number of AMD -Infinity Fabric links and the resulting transfer rates between the system-level -components. - -To the left and the right are memory controllers that attach the High Bandwidth -Memory (HBM) modules to the GCD. AMD Instinct MI250 GPUs use HBM2e, which offers -a peak memory bandwidth of 1.6 TB/sec per GCD. - -The execution units of the GPU are depicted in the following image as Compute -Units (CU). The MI250 GCD has 104 active CUs. Each compute unit is further -subdivided into four SIMD units that process SIMD instructions of 16 data -elements per instruction (for the FP64 data type). This enables the CU to -process 64 work items (a so-called “wavefront”) at a peak clock frequency of 1.7 -GHz. Therefore, the theoretical maximum FP64 peak performance per GCD is 22.6 -TFLOPS for vector instructions. This equates to 45.3 TFLOPS for vector instructions for both GCDs together. The MI250 compute units also provide specialized -execution units (also called matrix cores), which are geared toward executing -matrix operations like matrix-matrix multiplications. For FP64, the peak -performance of these units amounts to 90.5 TFLOPS. - -![Structure of a single GCD in the AMD Instinct MI250 accelerator.](../images/gpu-arch/image001.png "Structure of a single GCD in the AMD Instinct MI250 accelerator.") - -```{list-table} Peak-performance capabilities of the MI250 OAM for different data types. -:header-rows: 1 -:name: mi250-perf-table - -* - - Computation and Data Type - - FLOPS/CLOCK/CU - - Peak TFLOPS -* - - Matrix FP64 - - 256 - - 90.5 -* - - Vector FP64 - - 128 - - 45.3 -* - - Matrix FP32 - - 256 - - 90.5 -* - - Packed FP32 - - 256 - - 90.5 -* - - Vector FP32 - - 128 - - 45.3 -* - - Matrix FP16 - - 1024 - - 362.1 -* - - Matrix BF16 - - 1024 - - 362.1 -* - - Matrix INT8 - - 1024 - - 362.1 -``` - -The above table summarizes the aggregated peak performance of the AMD -Instinct MI250 OCP Open Accelerator Modules (OAM, OCP is short for Open Compute -Platform) and its two GCDs for different data types and execution units. The -middle column lists the peak performance (number of data elements processed in a -single instruction) of a single compute unit if a SIMD (or matrix) instruction -is being retired in each clock cycle. The third column lists the theoretical -peak performance of the OAM module. The theoretical aggregated peak memory -bandwidth of the GPU is 3.2 TB/sec (1.6 TB/sec per GCD). - -![Dual-GCD architecture of the AMD Instinct MI250 accelerators](../images/gpu-arch/image002.png "Dual-GCD architecture of the AMD Instinct MI250 accelerators") - -The following image shows the block diagram of an OAM package that consists -of two GCDs, each of which constitutes one GPU device in the system. The two -GCDs in the package are connected via four AMD Infinity Fabric links running at -a theoretical peak rate of 25 GT/sec, giving 200 GB/sec peak transfer bandwidth -between the two GCDs of an OAM, or a bidirectional peak transfer bandwidth of -400 GB/sec for the same. - -## Node-level architecture - -The following image shows the node-level architecture of a system that is -based on the AMD Instinct MI250 accelerator. The MI250 OAMs attach to the host -system via PCIe Gen 4 x16 links (yellow lines). Each GCD maintains its own PCIe -x16 link to the host part of the system. Depending on the server platform, the -GCD can attach to the AMD EPYC processor directly or via an optional PCIe switch -. Note that some platforms may offer an x8 interface to the GCDs, which reduces -the available host-to-GPU bandwidth. - -![Block diagram of AMD Instinct MI250 Accelerators with 3rd Generation AMD EPYC processor](../images/gpu-arch/image003.png "Block diagram of AMD Instinct MI250 Accelerators with 3rd Generation AMD EPYC processor") - -The preceding image shows the node-level architecture of a system with AMD -EPYC processors in a dual-socket configuration and four AMD Instinct MI250 -accelerators. The MI250 OAMs attach to the host processors system via PCIe Gen 4 -x16 links (yellow lines). Depending on the system design, a PCIe switch may -exist to make more PCIe lanes available for additional components like network -interfaces and/or storage devices. Each GCD maintains its own PCIe x16 link to -the host part of the system or to the PCIe switch. Please note, some platforms -may offer an x8 interface to the GCDs, which will reduce the available -host-to-GPU bandwidth. - -Between the OAMs and their respective GCDs, a peer-to-peer (P2P) network allows -for direct data exchange between the GPU dies via AMD Infinity Fabric links ( -black, green, and red lines). Each of these 16-wide links connects to one of the -two GPU dies in the MI250 OAM and operates at 25 GT/sec, which corresponds to a -theoretical peak transfer rate of 50 GB/sec per link (or 100 GB/sec -bidirectional peak transfer bandwidth). The GCD pairs 2 and 6 as well as GCDs 0 -and 4 connect via two XGMI links, which is indicated by the thicker red line in -the preceding image. diff --git a/docs/gpu-arch/mi300-mi200-performance-counters.rst b/docs/gpu-arch/mi300-mi200-performance-counters.rst deleted file mode 100644 index f35afb07..00000000 --- a/docs/gpu-arch/mi300-mi200-performance-counters.rst +++ /dev/null @@ -1,757 +0,0 @@ -.. meta:: - :description: MI300 and MI200 series performance counters and metrics - :keywords: MI300, MI200, performance counters, command processor counters - -*************************************************************************************************** -MI300 and MI200 series performance counters and metrics -*************************************************************************************************** - -This document lists and describes the hardware performance counters and derived metrics available -for the AMD Instinct™ MI300 and MI200 GPU. You can also access this information using the -:doc:`ROCProfiler tool `. - -MI300 and MI200 series performance counters -=============================================================== - -Series performance counters include the following categories: - -* :ref:`command-processor-counters` -* :ref:`graphics-register-bus-manager-counters` -* :ref:`spi-counters` -* :ref:`compute-unit-counters` -* :ref:`l1i-and-sl1d-cache-counters` -* :ref:`vector-l1-cache-subsystem-counters` -* :ref:`l2-cache-access-counters` - -The following sections provide additional details for each category. - -.. note:: - - Preliminary validation of all MI300 and MI200 series performance counters is in progress. Those with - an asterisk (*) require further evaluation. - -.. _command-processor-counters: - -Command processor counters ---------------------------------------------------------------------------------------------------------------- - -Command processor counters are further classified into command processor-fetcher and command -processor-compute. - -Command processor-fetcher counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``CPF_CMP_UTCL1_STALL_ON_TRANSLATION``", "Cycles", "Number of cycles one of the compute unified translation caches (L1) is stalled waiting on translation" - "``CPF_CPF_STAT_BUSY``", "Cycles", "Number of cycles command processor-fetcher is busy" - "``CPF_CPF_STAT_IDLE``", "Cycles", "Number of cycles command processor-fetcher is idle" - "``CPF_CPF_STAT_STALL``", "Cycles", "Number of cycles command processor-fetcher is stalled" - "``CPF_CPF_TCIU_BUSY``", "Cycles", "Number of cycles command processor-fetcher texture cache interface unit interface is busy" - "``CPF_CPF_TCIU_IDLE``", "Cycles", "Number of cycles command processor-fetcher texture cache interface unit interface is idle" - "``CPF_CPF_TCIU_STALL``", "Cycles", "Number of cycles command processor-fetcher texture cache interface unit interface is stalled waiting on free tags" - -The texture cache interface unit is the interface between the command processor and the memory -system. - -Command processor-compute counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``CPC_ME1_BUSY_FOR_PACKET_DECODE``", "Cycles", "Number of cycles command processor-compute micro engine is busy decoding packets" - "``CPC_UTCL1_STALL_ON_TRANSLATION``", "Cycles", "Number of cycles one of the unified translation caches (L1) is stalled waiting on translation" - "``CPC_CPC_STAT_BUSY``", "Cycles", "Number of cycles command processor-compute is busy" - "``CPC_CPC_STAT_IDLE``", "Cycles", "Number of cycles command processor-compute is idle" - "``CPC_CPC_STAT_STALL``", "Cycles", "Number of cycles command processor-compute is stalled" - "``CPC_CPC_TCIU_BUSY``", "Cycles", "Number of cycles command processor-compute texture cache interface unit interface is busy" - "``CPC_CPC_TCIU_IDLE``", "Cycles", "Number of cycles command processor-compute texture cache interface unit interface is idle" - "``CPC_CPC_UTCL2IU_BUSY``", "Cycles", "Number of cycles command processor-compute unified translation cache (L2) interface is busy" - "``CPC_CPC_UTCL2IU_IDLE``", "Cycles", "Number of cycles command processor-compute unified translation cache (L2) interface is idle" - "``CPC_CPC_UTCL2IU_STALL``", "Cycles", "Number of cycles command processor-compute unified translation cache (L2) interface is stalled" - "``CPC_ME1_DC0_SPI_BUSY``", "Cycles", "Number of cycles command processor-compute micro engine processor is busy" - -The micro engine runs packet-processing firmware on the command processor-compute counter. - -.. _graphics-register-bus-manager-counters: - -Graphics register bus manager counters ---------------------------------------------------------------------------------------------------------------- - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``GRBM_COUNT``", "Cycles","Number of free-running GPU cycles" - "``GRBM_GUI_ACTIVE``", "Cycles", "Number of GPU active cycles" - "``GRBM_CP_BUSY``", "Cycles", "Number of cycles any of the command processor blocks are busy" - "``GRBM_SPI_BUSY``", "Cycles", "Number of cycles any of the shader processor input is busy in the shader engines" - "``GRBM_TA_BUSY``", "Cycles", "Number of cycles any of the texture addressing unit is busy in the shader engines" - "``GRBM_TC_BUSY``", "Cycles", "Number of cycles any of the texture cache blocks are busy" - "``GRBM_CPC_BUSY``", "Cycles", "Number of cycles the command processor-compute is busy" - "``GRBM_CPF_BUSY``", "Cycles", "Number of cycles the command processor-fetcher is busy" - "``GRBM_UTCL2_BUSY``", "Cycles", "Number of cycles the unified translation cache (Level 2 [L2]) block is busy" - "``GRBM_EA_BUSY``", "Cycles", "Number of cycles the efficiency arbiter block is busy" - -Texture cache blocks include: - -* Texture cache arbiter -* Texture cache per pipe, also known as vector Level 1 (L1) cache -* Texture cache per channel, also known as known as L2 cache -* Texture cache interface - -.. _spi-counters: - -Shader processor input counters ---------------------------------------------------------------------------------------------------------------- - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SPI_CSN_BUSY``", "Cycles", "Number of cycles with outstanding waves" - "``SPI_CSN_WINDOW_VALID``", "Cycles", "Number of cycles enabled by ``perfcounter_start`` event" - "``SPI_CSN_NUM_THREADGROUPS``", "Workgroups", "Number of dispatched workgroups" - "``SPI_CSN_WAVE``", "Wavefronts", "Number of dispatched wavefronts" - "``SPI_RA_REQ_NO_ALLOC``", "Cycles", "Number of arbiter cycles with requests but no allocation" - "``SPI_RA_REQ_NO_ALLOC_CSN``", "Cycles", "Number of arbiter cycles with compute shader (n\ :sup:`th` pipe) requests but no compute shader (n\ :sup:`th` pipe) allocation" - "``SPI_RA_RES_STALL_CSN``", "Cycles", "Number of arbiter stall cycles due to shortage of compute shader (n\ :sup:`th` pipe) pipeline slots" - "``SPI_RA_TMP_STALL_CSN``", "Cycles", "Number of stall cycles due to shortage of temp space" - "``SPI_RA_WAVE_SIMD_FULL_CSN``", "SIMD-cycles", "Accumulated number of single instruction, multiple data (SIMD) per cycle affected by shortage of wave slots for compute shader (n\ :sup:`th` pipe) wave dispatch" - "``SPI_RA_VGPR_SIMD_FULL_CSN``", "SIMD-cycles", "Accumulated number of SIMDs per cycle affected by shortage of vector general-purpose register (VGPR) slots for compute shader (n\ :sup:`th` pipe) wave dispatch" - "``SPI_RA_SGPR_SIMD_FULL_CSN``", "SIMD-cycles", "Accumulated number of SIMDs per cycle affected by shortage of scalar general-purpose register (SGPR) slots for compute shader (n\ :sup:`th` pipe) wave dispatch" - "``SPI_RA_LDS_CU_FULL_CSN``", "CU", "Number of compute units affected by shortage of local data share (LDS) space for compute shader (n\ :sup:`th` pipe) wave dispatch" - "``SPI_RA_BAR_CU_FULL_CSN``", "CU", "Number of compute units with compute shader (n\ :sup:`th` pipe) waves waiting at a BARRIER" - "``SPI_RA_BULKY_CU_FULL_CSN``", "CU", "Number of compute units with compute shader (n\ :sup:`th` pipe) waves waiting for BULKY resource" - "``SPI_RA_TGLIM_CU_FULL_CSN``", "Cycles", "Number of compute shader (n\ :sup:`th` pipe) wave stall cycles due to restriction of ``tg_limit`` for thread group size" - "``SPI_RA_WVLIM_STALL_CSN``", "Cycles", "Number of cycles compute shader (n\ :sup:`th` pipe) is stalled due to ``WAVE_LIMIT``" - "``SPI_VWC_CSC_WR``", "Qcycles", "Number of quad-cycles taken to initialize VGPRs when launching waves" - "``SPI_SWC_CSC_WR``", "Qcycles", "Number of quad-cycles taken to initialize SGPRs when launching waves" - -.. _compute-unit-counters: - -Compute unit counters ---------------------------------------------------------------------------------------------------------------- - -The compute unit counters are further classified into instruction mix, matrix fused multiply-add (FMA) -operation counters, level counters, wavefront counters, wavefront cycle counters, and LDS counters. - -Instruction mix -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_INSTS``", "Instr", "Number of instructions issued" - "``SQ_INSTS_VALU``", "Instr", "Number of vector arithmetic logic unit (VALU) instructions including matrix FMA issued" - "``SQ_INSTS_VALU_ADD_F16``", "Instr", "Number of VALU half-precision floating-point (F16) ``ADD`` or ``SUB`` instructions issued" - "``SQ_INSTS_VALU_MUL_F16``", "Instr", "Number of VALU F16 Multiply instructions issued" - "``SQ_INSTS_VALU_FMA_F16``", "Instr", "Number of VALU F16 FMA or multiply-add instructions issued" - "``SQ_INSTS_VALU_TRANS_F16``", "Instr", "Number of VALU F16 Transcendental instructions issued" - "``SQ_INSTS_VALU_ADD_F32``", "Instr", "Number of VALU full-precision floating-point (F32) ``ADD`` or ``SUB`` instructions issued" - "``SQ_INSTS_VALU_MUL_F32``", "Instr", "Number of VALU F32 Multiply instructions issued" - "``SQ_INSTS_VALU_FMA_F32``", "Instr", "Number of VALU F32 FMAor multiply-add instructions issued" - "``SQ_INSTS_VALU_TRANS_F32``", "Instr", "Number of VALU F32 Transcendental instructions issued" - "``SQ_INSTS_VALU_ADD_F64``", "Instr", "Number of VALU F64 ``ADD`` or ``SUB`` instructions issued" - "``SQ_INSTS_VALU_MUL_F64``", "Instr", "Number of VALU F64 Multiply instructions issued" - "``SQ_INSTS_VALU_FMA_F64``", "Instr", "Number of VALU F64 FMA or multiply-add instructions issued" - "``SQ_INSTS_VALU_TRANS_F64``", "Instr", "Number of VALU F64 Transcendental instructions issued" - "``SQ_INSTS_VALU_INT32``", "Instr", "Number of VALU 32-bit integer instructions (signed or unsigned) issued" - "``SQ_INSTS_VALU_INT64``", "Instr", "Number of VALU 64-bit integer instructions (signed or unsigned) issued" - "``SQ_INSTS_VALU_CVT``", "Instr", "Number of VALU Conversion instructions issued" - "``SQ_INSTS_VALU_MFMA_I8``", "Instr", "Number of 8-bit Integer matrix FMA instructions issued" - "``SQ_INSTS_VALU_MFMA_F16``", "Instr", "Number of F16 matrix FMA instructions issued" - "``SQ_INSTS_VALU_MFMA_F32``", "Instr", "Number of F32 matrix FMA instructions issued" - "``SQ_INSTS_VALU_MFMA_F64``", "Instr", "Number of F64 matrix FMA instructions issued" - "``SQ_INSTS_MFMA``", "Instr", "Number of matrix FMA instructions issued" - "``SQ_INSTS_VMEM_WR``", "Instr", "Number of vector memory write instructions (including flat) issued" - "``SQ_INSTS_VMEM_RD``", "Instr", "Number of vector memory read instructions (including flat) issued" - "``SQ_INSTS_VMEM``", "Instr", "Number of vector memory instructions issued, including both flat and buffer instructions" - "``SQ_INSTS_SALU``", "Instr", "Number of scalar arithmetic logic unit (SALU) instructions issued" - "``SQ_INSTS_SMEM``", "Instr", "Number of scalar memory instructions issued" - "``SQ_INSTS_SMEM_NORM``", "Instr", "Number of scalar memory instructions normalized to match ``smem_level`` issued" - "``SQ_INSTS_FLAT``", "Instr", "Number of flat instructions issued" - "``SQ_INSTS_FLAT_LDS_ONLY``", "Instr", "**MI200 series only** Number of FLAT instructions that read/write only from/to LDS issued. Works only if ``EARLY_TA_DONE`` is enabled." - "``SQ_INSTS_LDS``", "Instr", "Number of LDS instructions issued **(MI200: includes flat; MI300: does not include flat)**" - "``SQ_INSTS_GDS``", "Instr", "Number of global data share instructions issued" - "``SQ_INSTS_EXP_GDS``", "Instr", "Number of EXP and global data share instructions excluding skipped export instructions issued" - "``SQ_INSTS_BRANCH``", "Instr", "Number of branch instructions issued" - "``SQ_INSTS_SENDMSG``", "Instr", "Number of ``SENDMSG`` instructions including ``s_endpgm`` issued" - "``SQ_INSTS_VSKIPPED``", "Instr", "Number of vector instructions skipped" - -Flat instructions allow read, write, and atomic access to a generic memory address pointer that can -resolve to any of the following physical memories: - -* Global Memory -* Scratch ("private") -* LDS ("shared") -* Invalid - ``MEM_VIOL`` TrapStatus - -Matrix fused multiply-add operation counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_INSTS_VALU_MFMA_MOPS_I8``", "IOP", "Number of 8-bit integer matrix FMA ops in the unit of 512" - "``SQ_INSTS_VALU_MFMA_MOPS_F16``", "FLOP", "Number of F16 floating matrix FMA ops in the unit of 512" - "``SQ_INSTS_VALU_MFMA_MOPS_BF16``", "FLOP", "Number of BF16 floating matrix FMA ops in the unit of 512" - "``SQ_INSTS_VALU_MFMA_MOPS_F32``", "FLOP", "Number of F32 floating matrix FMA ops in the unit of 512" - "``SQ_INSTS_VALU_MFMA_MOPS_F64``", "FLOP", "Number of F64 floating matrix FMA ops in the unit of 512" - -Level counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. note:: - - All level counters must be followed by ``SQ_ACCUM_PREV_HIRES`` counter to measure average latency. - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_ACCUM_PREV``", "Count", "Accumulated counter sample value where accumulation takes place once every four cycles" - "``SQ_ACCUM_PREV_HIRES``", "Count", "Accumulated counter sample value where accumulation takes place once every cycle" - "``SQ_LEVEL_WAVES``", "Waves", "Number of inflight waves" - "``SQ_INST_LEVEL_VMEM``", "Instr", "Number of inflight vector memory (including flat) instructions" - "``SQ_INST_LEVEL_SMEM``", "Instr", "Number of inflight scalar memory instructions" - "``SQ_INST_LEVEL_LDS``", "Instr", "Number of inflight LDS (including flat) instructions" - "``SQ_IFETCH_LEVEL``", "Instr", "Number of inflight instruction fetch requests from the cache" - -Use the following formulae to calculate latencies: - -* Vector memory latency = ``SQ_ACCUM_PREV_HIRES`` divided by ``SQ_INSTS_VMEM`` -* Wave latency = ``SQ_ACCUM_PREV_HIRES`` divided by ``SQ_WAVE`` -* LDS latency = ``SQ_ACCUM_PREV_HIRES`` divided by ``SQ_INSTS_LDS`` -* Scalar memory latency = ``SQ_ACCUM_PREV_HIRES`` divided by ``SQ_INSTS_SMEM_NORM`` -* Instruction fetch latency = ``SQ_ACCUM_PREV_HIRES`` divided by ``SQ_IFETCH`` - -Wavefront counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_WAVES``", "Waves", "Number of wavefronts dispatched to sequencers, including both new and restored wavefronts" - "``SQ_WAVES_SAVED``", "Waves", "Number of context-saved waves" - "``SQ_WAVES_RESTORED``", "Waves", "Number of context-restored waves sent to sequencers" - "``SQ_WAVES_EQ_64``", "Waves", "Number of wavefronts with exactly 64 active threads sent to sequencers" - "``SQ_WAVES_LT_64``", "Waves", "Number of wavefronts with less than 64 active threads sent to sequencers" - "``SQ_WAVES_LT_48``", "Waves", "Number of wavefronts with less than 48 active threads sent to sequencers" - "``SQ_WAVES_LT_32``", "Waves", "Number of wavefronts with less than 32 active threads sent to sequencers" - "``SQ_WAVES_LT_16``", "Waves", "Number of wavefronts with less than 16 active threads sent to sequencers" - -Wavefront cycle counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_CYCLES``", "Cycles", "Clock cycles" - "``SQ_BUSY_CYCLES``", "Cycles", "Number of cycles while sequencers reports it to be busy" - "``SQ_BUSY_CU_CYCLES``", "Qcycles", "Number of quad-cycles each compute unit is busy" - "``SQ_VALU_MFMA_BUSY_CYCLES``", "Cycles", "Number of cycles the matrix FMA arithmetic logic unit (ALU) is busy" - "``SQ_WAVE_CYCLES``", "Qcycles", "Number of quad-cycles spent by waves in the compute units" - "``SQ_WAIT_ANY``", "Qcycles", "Number of quad-cycles spent waiting for anything" - "``SQ_WAIT_INST_ANY``", "Qcycles", "Number of quad-cycles spent waiting for any instruction to be issued" - "``SQ_ACTIVE_INST_ANY``", "Qcycles", "Number of quad-cycles spent by each wave to work on an instruction" - "``SQ_ACTIVE_INST_VMEM``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on a vector memory instruction" - "``SQ_ACTIVE_INST_LDS``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on an LDS instruction" - "``SQ_ACTIVE_INST_VALU``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on a VALU instruction" - "``SQ_ACTIVE_INST_SCA``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on a SALU or scalar memory instruction" - "``SQ_ACTIVE_INST_EXP_GDS``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on an ``EXPORT`` or ``GDS`` instruction" - "``SQ_ACTIVE_INST_MISC``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on a ``BRANCH`` or ``SENDMSG`` instruction" - "``SQ_ACTIVE_INST_FLAT``", "Qcycles", "Number of quad-cycles spent by the sequencer instruction arbiter to work on a flat instruction" - "``SQ_INST_CYCLES_VMEM_WR``", "Qcycles", "Number of quad-cycles spent to send addr and cmd data for vector memory write instructions" - "``SQ_INST_CYCLES_VMEM_RD``", "Qcycles", "Number of quad-cycles spent to send addr and cmd data for vector memory read instructions" - "``SQ_INST_CYCLES_SMEM``", "Qcycles", "Number of quad-cycles spent to execute scalar memory reads" - "``SQ_INST_CYCLES_SALU``", "Qcycles", "Number of quad-cycles spent to execute non-memory read scalar operations" - "``SQ_THREAD_CYCLES_VALU``", "Qcycles", "Number of quad-cycles spent to execute VALU operations on active threads" - "``SQ_WAIT_INST_LDS``", "Qcycles", "Number of quad-cycles spent waiting for LDS instruction to be issued" - -``SQ_THREAD_CYCLES_VALU`` is similar to ``INST_CYCLES_VALU``, but it's multiplied by the number of -active threads. - -LDS counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_LDS_ATOMIC_RETURN``", "Cycles", "Number of atomic return cycles in LDS" - "``SQ_LDS_BANK_CONFLICT``", "Cycles", "Number of cycles LDS is stalled by bank conflicts" - "``SQ_LDS_ADDR_CONFLICT``", "Cycles", "Number of cycles LDS is stalled by address conflicts" - "``SQ_LDS_UNALIGNED_STALL``", "Cycles", "Number of cycles LDS is stalled processing flat unaligned load or store operations" - "``SQ_LDS_MEM_VIOLATIONS``", "Count", "Number of threads that have a memory violation in the LDS" - "``SQ_LDS_IDX_ACTIVE``", "Cycles", "Number of cycles LDS is used for indexed operations" - -Miscellaneous counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQ_IFETCH``", "Count", "Number of instruction fetch requests from L1i, in 32-byte width" - "``SQ_ITEMS``", "Threads", "Number of valid items per wave" - -.. _l1i-and-sl1d-cache-counters: - -L1 instruction cache (L1i) and scalar L1 data cache (L1d) counters ---------------------------------------------------------------------------------------------------------------- - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition" - - "``SQC_ICACHE_REQ``", "Req", "Number of L1 instruction (L1i) cache requests" - "``SQC_ICACHE_HITS``", "Count", "Number of L1i cache hits" - "``SQC_ICACHE_MISSES``", "Count", "Number of non-duplicate L1i cache misses including uncached requests" - "``SQC_ICACHE_MISSES_DUPLICATE``", "Count", "Number of duplicate L1i cache misses whose previous lookup miss on the same cache line is not fulfilled yet" - "``SQC_DCACHE_REQ``", "Req", "Number of scalar L1d requests" - "``SQC_DCACHE_INPUT_VALID_READYB``", "Cycles", "Number of cycles while sequencer input is valid but scalar L1d is not ready" - "``SQC_DCACHE_HITS``", "Count", "Number of scalar L1d hits" - "``SQC_DCACHE_MISSES``", "Count", "Number of non-duplicate scalar L1d misses including uncached requests" - "``SQC_DCACHE_MISSES_DUPLICATE``", "Count", "Number of duplicate scalar L1d misses" - "``SQC_DCACHE_REQ_READ_1``", "Req", "Number of constant cache read requests in a single 32-bit data word" - "``SQC_DCACHE_REQ_READ_2``", "Req", "Number of constant cache read requests in two 32-bit data words" - "``SQC_DCACHE_REQ_READ_4``", "Req", "Number of constant cache read requests in four 32-bit data words" - "``SQC_DCACHE_REQ_READ_8``", "Req", "Number of constant cache read requests in eight 32-bit data words" - "``SQC_DCACHE_REQ_READ_16``", "Req", "Number of constant cache read requests in 16 32-bit data words" - "``SQC_DCACHE_ATOMIC``", "Req", "Number of atomic requests" - "``SQC_TC_REQ``", "Req", "Number of texture cache requests that were issued by instruction and constant caches" - "``SQC_TC_INST_REQ``", "Req", "Number of instruction requests to the L2 cache" - "``SQC_TC_DATA_READ_REQ``", "Req", "Number of data Read requests to the L2 cache" - "``SQC_TC_DATA_WRITE_REQ``", "Req", "Number of data write requests to the L2 cache" - "``SQC_TC_DATA_ATOMIC_REQ``", "Req", "Number of data atomic requests to the L2 cache" - "``SQC_TC_STALL``", "Cycles", "Number of cycles while the valid requests to the L2 cache are stalled" - -.. _vector-l1-cache-subsystem-counters: - -Vector L1 cache subsystem counters ---------------------------------------------------------------------------------------------------------------- - -The vector L1 cache subsystem counters are further classified into texture addressing unit, texture data -unit, vector L1d or texture cache per pipe, and texture cache arbiter counters. - -Texture addressing unit counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition", "Value range for ``n``" - - "``TA_TA_BUSY[n]``", "Cycles", "Texture addressing unit busy cycles", "0-15" - "``TA_TOTAL_WAVEFRONTS[n]``", "Instr", "Number of wavefronts processed by texture addressing unit", "0-15" - "``TA_BUFFER_WAVEFRONTS[n]``", "Instr", "Number of buffer wavefronts processed by texture addressing unit", "0-15" - "``TA_BUFFER_READ_WAVEFRONTS[n]``", "Instr", "Number of buffer read wavefronts processed by texture addressing unit", "0-15" - "``TA_BUFFER_WRITE_WAVEFRONTS[n]``", "Instr", "Number of buffer write wavefronts processed by texture addressing unit", "0-15" - "``TA_BUFFER_ATOMIC_WAVEFRONTS[n]``", "Instr", "Number of buffer atomic wavefronts processed by texture addressing unit", "0-15" - "``TA_BUFFER_TOTAL_CYCLES[n]``", "Cycles", "Number of buffer cycles (including read and write) issued to texture cache", "0-15" - "``TA_BUFFER_COALESCED_READ_CYCLES[n]``", "Cycles", "Number of coalesced buffer read cycles issued to texture cache", "0-15" - "``TA_BUFFER_COALESCED_WRITE_CYCLES[n]``", "Cycles", "Number of coalesced buffer write cycles issued to texture cache", "0-15" - "``TA_ADDR_STALLED_BY_TC_CYCLES[n]``", "Cycles", "Number of cycles texture addressing unit address path is stalled by texture cache", "0-15" - "``TA_DATA_STALLED_BY_TC_CYCLES[n]``", "Cycles", "Number of cycles texture addressing unit data path is stalled by texture cache", "0-15" - "``TA_ADDR_STALLED_BY_TD_CYCLES[n]``", "Cycles", "Number of cycles texture addressing unit address path is stalled by texture data unit", "0-15" - "``TA_FLAT_WAVEFRONTS[n]``", "Instr", "Number of flat opcode wavefronts processed by texture addressing unit", "0-15" - "``TA_FLAT_READ_WAVEFRONTS[n]``", "Instr", "Number of flat opcode read wavefronts processed by texture addressing unit", "0-15" - "``TA_FLAT_WRITE_WAVEFRONTS[n]``", "Instr", "Number of flat opcode write wavefronts processed by texture addressing unit", "0-15" - "``TA_FLAT_ATOMIC_WAVEFRONTS[n]``", "Instr", "Number of flat opcode atomic wavefronts processed by texture addressing unit", "0-15" - -Texture data unit counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition", "Value range for ``n``" - - "``TD_TD_BUSY[n]``", "Cycle", "Texture data unit busy cycles while it is processing or waiting for data", "0-15" - "``TD_TC_STALL[n]``", "Cycle", "Number of cycles texture data unit is stalled waiting for texture cache data", "0-15" - "``TD_SPI_STALL[n]``", "Cycle", "Number of cycles texture data unit is stalled by shader processor input", "0-15" - "``TD_LOAD_WAVEFRONT[n]``", "Instr", "Number of wavefront instructions (read, write, atomic)", "0-15" - "``TD_STORE_WAVEFRONT[n]``", "Instr", "Number of write wavefront instructions", "0-15" - "``TD_ATOMIC_WAVEFRONT[n]``", "Instr", "Number of atomic wavefront instructions", "0-15" - "``TD_COALESCABLE_WAVEFRONT[n]``", "Instr", "Number of coalescable wavefronts according to texture addressing unit", "0-15" - -Texture cache per pipe counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition", "Value range for ``n``" - - "``TCP_GATE_EN1[n]``", "Cycles", "Number of cycles vector L1d interface clocks are turned on", "0-15" - "``TCP_GATE_EN2[n]``", "Cycles", "Number of cycles vector L1d core clocks are turned on", "0-15" - "``TCP_TD_TCP_STALL_CYCLES[n]``", "Cycles", "Number of cycles texture data unit stalls vector L1d", "0-15" - "``TCP_TCR_TCP_STALL_CYCLES[n]``", "Cycles", "Number of cycles texture cache router stalls vector L1d", "0-15" - "``TCP_READ_TAGCONFLICT_STALL_CYCLES[n]``", "Cycles", "Number of cycles tag RAM conflict stalls on a read", "0-15" - "``TCP_WRITE_TAGCONFLICT_STALL_CYCLES[n]``", "Cycles", "Number of cycles tag RAM conflict stalls on a write", "0-15" - "``TCP_ATOMIC_TAGCONFLICT_STALL_CYCLES[n]``", "Cycles", "Number of cycles tag RAM conflict stalls on an atomic", "0-15" - "``TCP_PENDING_STALL_CYCLES[n]``", "Cycles", "Number of cycles vector L1d is stalled due to data pending from L2 Cache", "0-15" - "``TCP_TCP_TA_DATA_STALL_CYCLES``", "Cycles", "Number of cycles texture cache per pipe stalls texture addressing unit data interface", "NA" - "``TCP_TA_TCP_STATE_READ[n]``", "Req", "Number of state reads", "0-15" - "``TCP_VOLATILE[n]``", "Req", "Number of L1 volatile pixels or buffers from texture addressing unit", "0-15" - "``TCP_TOTAL_ACCESSES[n]``", "Req", "Number of vector L1d accesses. Equals ``TCP_PERF_SEL_TOTAL_READ`+`TCP_PERF_SEL_TOTAL_NONREAD``", "0-15" - "``TCP_TOTAL_READ[n]``", "Req", "Number of vector L1d read accesses", "0-15" - "``TCP_TOTAL_WRITE[n]``", "Req", "Number of vector L1d write accesses", "0-15" - "``TCP_TOTAL_ATOMIC_WITH_RET[n]``", "Req", "Number of vector L1d atomic requests with return", "0-15" - "``TCP_TOTAL_ATOMIC_WITHOUT_RET[n]``", "Req", "Number of vector L1d atomic without return", "0-15" - "``TCP_TOTAL_WRITEBACK_INVALIDATES[n]``", "Count", "Total number of vector L1d writebacks and invalidates", "0-15" - "``TCP_UTCL1_REQUEST[n]``", "Req", "Number of address translation requests to unified translation cache (L1)", "0-15" - "``TCP_UTCL1_TRANSLATION_HIT[n]``", "Req", "Number of unified translation cache (L1) translation hits", "0-15" - "``TCP_UTCL1_TRANSLATION_MISS[n]``", "Req", "Number of unified translation cache (L1) translation misses", "0-15" - "``TCP_UTCL1_PERMISSION_MISS[n]``", "Req", "Number of unified translation cache (L1) permission misses", "0-15" - "``TCP_TOTAL_CACHE_ACCESSES[n]``", "Req", "Number of vector L1d cache accesses including hits and misses", "0-15" - "``TCP_TCP_LATENCY[n]``", "Cycles", "**MI200 series only** Accumulated wave access latency to vL1D over all wavefronts", "0-15" - "``TCP_TCC_READ_REQ_LATENCY[n]``", "Cycles", "**MI200 series only** Total vL1D to L2 request latency over all wavefronts for reads and atomics with return", "0-15" - "``TCP_TCC_WRITE_REQ_LATENCY[n]``", "Cycles", "**MI200 series only** Total vL1D to L2 request latency over all wavefronts for writes and atomics without return", "0-15" - "``TCP_TCC_READ_REQ[n]``", "Req", "Number of read requests to L2 cache", "0-15" - "``TCP_TCC_WRITE_REQ[n]``", "Req", "Number of write requests to L2 cache", "0-15" - "``TCP_TCC_ATOMIC_WITH_RET_REQ[n]``", "Req", "Number of atomic requests to L2 cache with return", "0-15" - "``TCP_TCC_ATOMIC_WITHOUT_RET_REQ[n]``", "Req", "Number of atomic requests to L2 cache without return", "0-15" - "``TCP_TCC_NC_READ_REQ[n]``", "Req", "Number of non-coherently cached read requests to L2 cache", "0-15" - "``TCP_TCC_UC_READ_REQ[n]``", "Req", "Number of uncached read requests to L2 cache", "0-15" - "``TCP_TCC_CC_READ_REQ[n]``", "Req", "Number of coherently cached read requests to L2 cache", "0-15" - "``TCP_TCC_RW_READ_REQ[n]``", "Req", "Number of coherently cached with write read requests to L2 cache", "0-15" - "``TCP_TCC_NC_WRITE_REQ[n]``", "Req", "Number of non-coherently cached write requests to L2 cache", "0-15" - "``TCP_TCC_UC_WRITE_REQ[n]``", "Req", "Number of uncached write requests to L2 cache", "0-15" - "``TCP_TCC_CC_WRITE_REQ[n]``", "Req", "Number of coherently cached write requests to L2 cache", "0-15" - "``TCP_TCC_RW_WRITE_REQ[n]``", "Req", "Number of coherently cached with write write requests to L2 cache", "0-15" - "``TCP_TCC_NC_ATOMIC_REQ[n]``", "Req", "Number of non-coherently cached atomic requests to L2 cache", "0-15" - "``TCP_TCC_UC_ATOMIC_REQ[n]``", "Req", "Number of uncached atomic requests to L2 cache", "0-15" - "``TCP_TCC_CC_ATOMIC_REQ[n]``", "Req", "Number of coherently cached atomic requests to L2 cache", "0-15" - "``TCP_TCC_RW_ATOMIC_REQ[n]``", "Req", "Number of coherently cached with write atomic requests to L2 cache", "0-15" - -Note that: - -* ``TCP_TOTAL_READ[n]`` = ``TCP_PERF_SEL_TOTAL_HIT_LRU_READ`` + ``TCP_PERF_SEL_TOTAL_MISS_LRU_READ`` + ``TCP_PERF_SEL_TOTAL_MISS_EVICT_READ`` -* ``TCP_TOTAL_WRITE[n]`` = ``TCP_PERF_SEL_TOTAL_MISS_LRU_WRITE``+ ``TCP_PERF_SEL_TOTAL_MISS_EVICT_WRITE`` -* ``TCP_TOTAL_WRITEBACK_INVALIDATES[n]`` = ``TCP_PERF_SEL_TOTAL_WBINVL1``+ ``TCP_PERF_SEL_TOTAL_WBINVL1_VOL``+ ``TCP_PERF_SEL_CP_TCP_INVALIDATE``+ ``TCP_PERF_SEL_SQ_TCP_INVALIDATE_VOL`` - -Texture cache arbiter counters -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -.. csv-table:: - :header: "Hardware counter", "Unit", "Definition", "Value range for ``n``" - - "``TCA_CYCLE[n]``", "Cycles", "Number of texture cache arbiter cycles", "0-31" - "``TCA_BUSY[n]``", "Cycles", "Number of cycles texture cache arbiter has a pending request", "0-31" - -.. _l2-cache-access-counters: - -L2 cache access counters ---------------------------------------------------------------------------------------------------------------- - -L2 cache is also known as texture cache per channel. - -.. tab-set:: - - .. tab-item:: MI300 hardware counter - - .. csv-table:: - :header: "Hardware counter", "Unit", "Definition", "Value range for ``n``" - - "``TCC_CYCLE[n]``", "Cycles", "Number of L2 cache free-running clocks", "0-31" - "``TCC_BUSY[n]``", "Cycles", "Number of L2 cache busy cycles", "0-31" - "``TCC_REQ[n]``", "Req", "Number of L2 cache requests of all types (measured at the tag block)", "0-31" - "``TCC_STREAMING_REQ[n]``", "Req", "Number of L2 cache streaming requests (measured at the tag block)", "0-31" - "``TCC_NC_REQ[n]``", "Req", "Number of non-coherently cached requests (measured at the tag block)", "0-31" - "``TCC_UC_REQ[n]``", "Req", "Number of uncached requests. This is measured at the tag block", "0-31" - "``TCC_CC_REQ[n]``", "Req", "Number of coherently cached requests. This is measured at the tag block", "0-31" - "``TCC_RW_REQ[n]``", "Req", "Number of coherently cached with write requests. This is measured at the tag block", "0-31" - "``TCC_PROBE[n]``", "Req", "Number of probe requests", "0-31" - "``TCC_PROBE_ALL[n]``", "Req", "Number of external probe requests with ``EA_TCC_preq_all == 1``", "0-31" - "``TCC_READ[n]``", "Req", "Number of L2 cache read requests (includes compressed reads but not metadata reads)", "0-31" - "``TCC_WRITE[n]``", "Req", "Number of L2 cache write requests", "0-31" - "``TCC_ATOMIC[n]``", "Req", "Number of L2 cache atomic requests of all types", "0-31" - "``TCC_HIT[n]``", "Req", "Number of L2 cache hits", "0-31" - "``TCC_MISS[n]``", "Req", "Number of L2 cache misses", "0-31" - "``TCC_WRITEBACK[n]``", "Req", "Number of lines written back to the main memory, including writebacks of dirty lines and uncached write or atomic requests", "0-31" - "``TCC_EA0_WRREQ[n]``", "Req", "Number of 32-byte and 64-byte transactions going over the ``TC_EA_wrreq`` interface (doesn't include probe commands)", "0-31" - "``TCC_EA0_WRREQ_64B[n]``", "Req", "Total number of 64-byte transactions (write or ``CMPSWAP``) going over the ``TC_EA_wrreq`` interface", "0-31" - "``TCC_EA0_WR_UNCACHED_32B[n]``", "Req", "Number of 32 or 64-byte write or atomic going over the ``TC_EA_wrreq`` interface due to uncached traffic", "0-31" - "``TCC_EA0_WRREQ_STALL[n]``", "Cycles", "Number of cycles a write request is stalled", "0-31" - "``TCC_EA0_WRREQ_IO_CREDIT_STALL[n]``", "Cycles", "Number of cycles an efficiency arbiter write request is stalled due to the interface running out of input-output (IO) credits", "0-31" - "``TCC_EA0_WRREQ_GMI_CREDIT_STALL[n]``", "Cycles", "Number of cycles an efficiency arbiter write request is stalled due to the interface running out of GMI credits", "0-31" - "``TCC_EA0_WRREQ_DRAM_CREDIT_STALL[n]``", "Cycles", "Number of cycles an efficiency arbiter write request is stalled due to the interface running out of DRAM credits", "0-31" - "``TCC_TOO_MANY_EA_WRREQS_STALL[n]``", "Cycles", "Number of cycles the L2 cache is unable to send an efficiency arbiter write request due to it reaching its maximum capacity of pending efficiency arbiter write requests", "0-31" - "``TCC_EA0_WRREQ_LEVEL[n]``", "Req", "The accumulated number of efficiency arbiter write requests in flight", "0-31" - "``TCC_EA0_ATOMIC[n]``", "Req", "Number of 32-byte or 64-byte atomic requests going over the ``TC_EA_wrreq`` interface", "0-31" - "``TCC_EA0_ATOMIC_LEVEL[n]``", "Req", "The accumulated number of efficiency arbiter atomic requests in flight", "0-31" - "``TCC_EA0_RDREQ[n]``", "Req", "Number of 32-byte or 64-byte read requests to efficiency arbiter", "0-31" - "``TCC_EA0_RDREQ_32B[n]``", "Req", "Number of 32-byte read requests to efficiency arbiter", "0-31" - "``TCC_EA0_RD_UNCACHED_32B[n]``", "Req", "Number of 32-byte efficiency arbiter reads due to uncached traffic. A 64-byte request is counted as 2", "0-31" - "``TCC_EA0_RDREQ_IO_CREDIT_STALL[n]``", "Cycles", "Number of cycles there is a stall due to the read request interface running out of IO credits", "0-31" - "``TCC_EA0_RDREQ_GMI_CREDIT_STALL[n]``", "Cycles", "Number of cycles there is a stall due to the read request interface running out of GMI credits", "0-31" - "``TCC_EA0_RDREQ_DRAM_CREDIT_STALL[n]``", "Cycles", "Number of cycles there is a stall due to the read request interface running out of DRAM credits", "0-31" - "``TCC_EA0_RDREQ_LEVEL[n]``", "Req", "The accumulated number of efficiency arbiter read requests in flight", "0-31" - "``TCC_EA0_RDREQ_DRAM[n]``", "Req", "Number of 32-byte or 64-byte efficiency arbiter read requests to High Bandwidth Memory (HBM)", "0-31" - "``TCC_EA0_WRREQ_DRAM[n]``", "Req", "Number of 32-byte or 64-byte efficiency arbiter write requests to HBM", "0-31" - "``TCC_TAG_STALL[n]``", "Cycles", "Number of cycles the normal request pipeline in the tag is stalled for any reason", "0-31" - "``TCC_NORMAL_WRITEBACK[n]``", "Req", "Number of writebacks due to requests that are not writeback requests", "0-31" - "``TCC_ALL_TC_OP_WB_WRITEBACK[n]``", "Req", "Number of writebacks due to all ``TC_OP`` writeback requests", "0-31" - "``TCC_NORMAL_EVICT[n]``", "Req", "Number of evictions due to requests that are not invalidate or probe requests", "0-31" - "``TCC_ALL_TC_OP_INV_EVICT[n]``", "Req", "Number of evictions due to all ``TC_OP`` invalidate requests", "0-31" - - .. tab-item:: MI200 hardware counter - - .. csv-table:: - :header: "Hardware counter", "Unit", "Definition", "Value range for ``n``" - - "``TCC_CYCLE[n]``", "Cycles", "Number of L2 cache free-running clocks", "0-31" - "``TCC_BUSY[n]``", "Cycles", "Number of L2 cache busy cycles", "0-31" - "``TCC_REQ[n]``", "Req", "Number of L2 cache requests of all types (measured at the tag block)", "0-31" - "``TCC_STREAMING_REQ[n]``", "Req", "Number of L2 cache streaming requests (measured at the tag block)", "0-31" - "``TCC_NC_REQ[n]``", "Req", "Number of non-coherently cached requests (measured at the tag block)", "0-31" - "``TCC_UC_REQ[n]``", "Req", "Number of uncached requests. This is measured at the tag block", "0-31" - "``TCC_CC_REQ[n]``", "Req", "Number of coherently cached requests. This is measured at the tag block", "0-31" - "``TCC_RW_REQ[n]``", "Req", "Number of coherently cached with write requests. This is measured at the tag block", "0-31" - "``TCC_PROBE[n]``", "Req", "Number of probe requests", "0-31" - "``TCC_PROBE_ALL[n]``", "Req", "Number of external probe requests with ``EA_TCC_preq_all == 1``", "0-31" - "``TCC_READ[n]``", "Req", "Number of L2 cache read requests (includes compressed reads but not metadata reads)", "0-31" - "``TCC_WRITE[n]``", "Req", "Number of L2 cache write requests", "0-31" - "``TCC_ATOMIC[n]``", "Req", "Number of L2 cache atomic requests of all types", "0-31" - "``TCC_HIT[n]``", "Req", "Number of L2 cache hits", "0-31" - "``TCC_MISS[n]``", "Req", "Number of L2 cache misses", "0-31" - "``TCC_WRITEBACK[n]``", "Req", "Number of lines written back to the main memory, including writebacks of dirty lines and uncached write or atomic requests", "0-31" - "``TCC_EA_WRREQ[n]``", "Req", "Number of 32-byte and 64-byte transactions going over the ``TC_EA_wrreq`` interface (doesn't include probe commands)", "0-31" - "``TCC_EA_WRREQ_64B[n]``", "Req", "Total number of 64-byte transactions (write or ``CMPSWAP``) going over the ``TC_EA_wrreq`` interface", "0-31" - "``TCC_EA_WR_UNCACHED_32B[n]``", "Req", "Number of 32 write or atomic going over the ``TC_EA_wrreq`` interface due to uncached traffic. A 64-byte request will be counted as 2", "0-31" - "``TCC_EA_WRREQ_STALL[n]``", "Cycles", "Number of cycles a write request is stalled", "0-31" - "``TCC_EA_WRREQ_IO_CREDIT_STALL[n]``", "Cycles", "Number of cycles an efficiency arbiter write request is stalled due to the interface running out of input-output (IO) credits", "0-31" - "``TCC_EA_WRREQ_GMI_CREDIT_STALL[n]``", "Cycles", "Number of cycles an efficiency arbiter write request is stalled due to the interface running out of GMI credits", "0-31" - "``TCC_EA_WRREQ_DRAM_CREDIT_STALL[n]``", "Cycles", "Number of cycles an efficiency arbiter write request is stalled due to the interface running out of DRAM credits", "0-31" - "``TCC_TOO_MANY_EA_WRREQS_STALL[n]``", "Cycles", "Number of cycles the L2 cache is unable to send an efficiency arbiter write request due to it reaching its maximum capacity of pending efficiency arbiter write requests", "0-31" - "``TCC_EA_WRREQ_LEVEL[n]``", "Req", "The accumulated number of efficiency arbiter write requests in flight", "0-31" - "``TCC_EA_ATOMIC[n]``", "Req", "Number of 32-byte or 64-byte atomic requests going over the ``TC_EA_wrreq`` interface", "0-31" - "``TCC_EA_ATOMIC_LEVEL[n]``", "Req", "The accumulated number of efficiency arbiter atomic requests in flight", "0-31" - "``TCC_EA_RDREQ[n]``", "Req", "Number of 32-byte or 64-byte read requests to efficiency arbiter", "0-31" - "``TCC_EA_RDREQ_32B[n]``", "Req", "Number of 32-byte read requests to efficiency arbiter", "0-31" - "``TCC_EA_RD_UNCACHED_32B[n]``", "Req", "Number of 32-byte efficiency arbiter reads due to uncached traffic. A 64-byte request is counted as 2", "0-31" - "``TCC_EA_RDREQ_IO_CREDIT_STALL[n]``", "Cycles", "Number of cycles there is a stall due to the read request interface running out of IO credits", "0-31" - "``TCC_EA_RDREQ_GMI_CREDIT_STALL[n]``", "Cycles", "Number of cycles there is a stall due to the read request interface running out of GMI credits", "0-31" - "``TCC_EA_RDREQ_DRAM_CREDIT_STALL[n]``", "Cycles", "Number of cycles there is a stall due to the read request interface running out of DRAM credits", "0-31" - "``TCC_EA_RDREQ_LEVEL[n]``", "Req", "The accumulated number of efficiency arbiter read requests in flight", "0-31" - "``TCC_EA_RDREQ_DRAM[n]``", "Req", "Number of 32-byte or 64-byte efficiency arbiter read requests to High Bandwidth Memory (HBM)", "0-31" - "``TCC_EA_WRREQ_DRAM[n]``", "Req", "Number of 32-byte or 64-byte efficiency arbiter write requests to HBM", "0-31" - "``TCC_TAG_STALL[n]``", "Cycles", "Number of cycles the normal request pipeline in the tag is stalled for any reason", "0-31" - "``TCC_NORMAL_WRITEBACK[n]``", "Req", "Number of writebacks due to requests that are not writeback requests", "0-31" - "``TCC_ALL_TC_OP_WB_WRITEBACK[n]``", "Req", "Number of writebacks due to all ``TC_OP`` writeback requests", "0-31" - "``TCC_NORMAL_EVICT[n]``", "Req", "Number of evictions due to requests that are not invalidate or probe requests", "0-31" - "``TCC_ALL_TC_OP_INV_EVICT[n]``", "Req", "Number of evictions due to all ``TC_OP`` invalidate requests", "0-31" - -Note the following: - -* ``TCC_REQ[n]`` may be more than the number of requests arriving at the texture cache per channel, - but it's a good indication of the total amount of work that needs to be performed. -* For ``TCC_EA0_WRREQ[n]``, atomics may travel over the same interface and are generally classified as - write requests. -* CC mtypes can produce uncached requests, and those are included in - ``TCC_EA0_WR_UNCACHED_32B[n]`` -* ``TCC_EA0_WRREQ_LEVEL[n]`` is primarily intended to measure average efficiency arbiter write latency. - - * Average write latency = ``TCC_PERF_SEL_EA0_WRREQ_LEVEL`` divided by ``TCC_PERF_SEL_EA0_WRREQ`` - -* ``TCC_EA0_ATOMIC_LEVEL[n]`` is primarily intended to measure average efficiency arbiter atomic - latency - - * Average atomic latency = ``TCC_PERF_SEL_EA0_WRREQ_ATOMIC_LEVEL`` divided by ``TCC_PERF_SEL_EA0_WRREQ_ATOMIC`` - -* ``TCC_EA0_RDREQ_LEVEL[n]`` is primarily intended to measure average efficiency arbiter read latency. - - * Average read latency = ``TCC_PERF_SEL_EA0_RDREQ_LEVEL`` divided by ``TCC_PERF_SEL_EA0_RDREQ`` - -* Stalls can occur regardless of the need for a read to be performed -* Normally, stalls are measured exactly at one point in the pipeline however in the case of - ``TCC_TAG_STALL[n]``, probes can stall the pipeline at a variety of places. There is no single point that - can accurately measure the total stalls - -MI300 and MI200 series derived metrics list -============================================================== - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``ALUStalledByLDS``", "Percentage of GPU time ALU units are stalled due to the LDS input queue being full or the output queue not being ready (value range: 0% (optimal) to 100%)" - "``FetchSize``", "Total kilobytes fetched from the video memory; measured with all extra fetches and any cache or memory effects taken into account" - "``FlatLDSInsts``", "Average number of flat instructions that read from or write to LDS, run per work item (affected by flow control)" - "``FlatVMemInsts``", "Average number of flat instructions that read from or write to the video memory, run per work item (affected by flow control). Includes flat instructions that read from or write to scratch" - "``GDSInsts``", "Average number of global data share read or write instructions run per work item (affected by flow control)" - "``GPUBusy``", "Percentage of time GPU is busy" - "``L2CacheHit``", "Percentage of fetch, write, atomic, and other instructions that hit the data in L2 cache (value range: 0% (no hit) to 100% (optimal))" - "``LDSBankConflict``", "Percentage of GPU time LDS is stalled by bank conflicts (value range: 0% (optimal) to 100%)" - "``LDSInsts``", "Average number of LDS read or write instructions run per work item (affected by flow control). Excludes flat instructions that read from or write to LDS." - "``MemUnitBusy``", "Percentage of GPU time the memory unit is active, which is measured with all extra fetches and writes and any cache or memory effects taken into account (value range: 0% to 100% (fetch-bound))" - "``MemUnitStalled``", "Percentage of GPU time the memory unit is stalled (value range: 0% (optimal) to 100%)" - "``MemWrites32B``", "Total number of effective 32B write transactions to the memory" - "``TCA_BUSY_sum``", "Total number of cycles texture cache arbiter has a pending request, over all texture cache arbiter instances" - "``TCA_CYCLE_sum``", "Total number of cycles over all texture cache arbiter instances" - "``SALUBusy``", "Percentage of GPU time scalar ALU instructions are processed (value range: 0% to 100% (optimal))" - "``SALUInsts``", "Average number of scalar ALU instructions run per work item (affected by flow control)" - "``SFetchInsts``", "Average number of scalar fetch instructions from the video memory run per work item (affected by flow control)" - "``VALUBusy``", "Percentage of GPU time vector ALU instructions are processed (value range: 0% to 100% (optimal))" - "``VALUInsts``", "Average number of vector ALU instructions run per work item (affected by flow control)" - "``VALUUtilization``", "Percentage of active vector ALU threads in a wave, where a lower number can mean either more thread divergence in a wave or that the work-group size is not a multiple of 64 (value range: 0%, 100% (optimal - no thread divergence))" - "``VFetchInsts``", "Average number of vector fetch instructions from the video memory run per work-item (affected by flow control); excludes flat instructions that fetch from video memory" - "``VWriteInsts``", "Average number of vector write instructions to the video memory run per work-item (affected by flow control); excludes flat instructions that write to video memory" - "``Wavefronts``", "Total wavefronts" - "``WRITE_REQ_32B``", "Total number of 32-byte effective memory writes" - "``WriteSize``", "Total kilobytes written to the video memory; measured with all extra fetches and any cache or memory effects taken into account" - "``WriteUnitStalled``", "Percentage of GPU time the write unit is stalled (value range: 0% (optimal) to 100%)" - -You can lower ``ALUStalledByLDS`` by reducing LDS bank conflicts or number of LDS accesses. -You can lower ``MemUnitStalled`` by reducing the number or size of fetches and writes. -``MemUnitBusy`` includes the stall time (``MemUnitStalled``). - -Hardware counters by and over all texture addressing unit instances ---------------------------------------------------------------------------------------------------------------- - -The following table shows the hardware counters *by* all texture addressing unit instances. - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TA_BUFFER_WAVEFRONTS_sum``", "Total number of buffer wavefronts processed" - "``TA_BUFFER_READ_WAVEFRONTS_sum``", "Total number of buffer read wavefronts processed" - "``TA_BUFFER_WRITE_WAVEFRONTS_sum``", "Total number of buffer write wavefronts processed" - "``TA_BUFFER_ATOMIC_WAVEFRONTS_sum``", "Total number of buffer atomic wavefronts processed" - "``TA_BUFFER_TOTAL_CYCLES_sum``", "Total number of buffer cycles (including read and write) issued to texture cache" - "``TA_BUFFER_COALESCED_READ_CYCLES_sum``", "Total number of coalesced buffer read cycles issued to texture cache" - "``TA_BUFFER_COALESCED_WRITE_CYCLES_sum``", "Total number of coalesced buffer write cycles issued to texture cache" - "``TA_FLAT_READ_WAVEFRONTS_sum``", "Sum of flat opcode reads processed" - "``TA_FLAT_WRITE_WAVEFRONTS_sum``", "Sum of flat opcode writes processed" - "``TA_FLAT_WAVEFRONTS_sum``", "Total number of flat opcode wavefronts processed" - "``TA_FLAT_ATOMIC_WAVEFRONTS_sum``", "Total number of flat opcode atomic wavefronts processed" - "``TA_TOTAL_WAVEFRONTS_sum``", "Total number of wavefronts processed" - -The following table shows the hardware counters *over* all texture addressing unit instances. - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TA_ADDR_STALLED_BY_TC_CYCLES_sum``", "Total number of cycles texture addressing unit address path is stalled by texture cache" - "``TA_ADDR_STALLED_BY_TD_CYCLES_sum``", "Total number of cycles texture addressing unit address path is stalled by texture data unit" - "``TA_BUSY_avr``", "Average number of busy cycles" - "``TA_BUSY_max``", "Maximum number of texture addressing unit busy cycles" - "``TA_BUSY_min``", "Minimum number of texture addressing unit busy cycles" - "``TA_DATA_STALLED_BY_TC_CYCLES_sum``", "Total number of cycles texture addressing unit data path is stalled by texture cache" - "``TA_TA_BUSY_sum``", "Total number of texture addressing unit busy cycles" - -Hardware counters over all texture cache per channel instances ---------------------------------------------------------------------------------------------------------------- - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TCC_ALL_TC_OP_WB_WRITEBACK_sum``", "Total number of writebacks due to all ``TC_OP`` writeback requests." - "``TCC_ALL_TC_OP_INV_EVICT_sum``", "Total number of evictions due to all ``TC_OP`` invalidate requests." - "``TCC_ATOMIC_sum``", "Total number of L2 cache atomic requests of all types." - "``TCC_BUSY_avr``", "Average number of L2 cache busy cycles." - "``TCC_BUSY_sum``", "Total number of L2 cache busy cycles." - "``TCC_CC_REQ_sum``", "Total number of coherently cached requests." - "``TCC_CYCLE_sum``", "Total number of L2 cache free running clocks." - "``TCC_EA0_WRREQ_sum``", "Total number of 32-byte and 64-byte transactions going over the ``TC_EA0_wrreq`` interface. Atomics may travel over the same interface and are generally classified as write requests. This does not include probe commands." - "``TCC_EA0_WRREQ_64B_sum``", "Total number of 64-byte transactions (write or `CMPSWAP`) going over the ``TC_EA0_wrreq`` interface." - "``TCC_EA0_WR_UNCACHED_32B_sum``", "Total Number of 32-byte write or atomic going over the ``TC_EA0_wrreq`` interface due to uncached traffic. Note that coherently cached mtypes can produce uncached requests, and those are included in this. A 64-byte request is counted as 2." - "``TCC_EA0_WRREQ_STALL_sum``", "Total Number of cycles a write request is stalled, over all instances." - "``TCC_EA0_WRREQ_IO_CREDIT_STALL_sum``", "Total number of cycles an efficiency arbiter write request is stalled due to the interface running out of IO credits, over all instances." - "``TCC_EA0_WRREQ_GMI_CREDIT_STALL_sum``", "Total number of cycles an efficiency arbiter write request is stalled due to the interface running out of GMI credits, over all instances." - "``TCC_EA0_WRREQ_DRAM_CREDIT_STALL_sum``", "Total number of cycles an efficiency arbiter write request is stalled due to the interface running out of DRAM credits, over all instances." - "``TCC_EA0_WRREQ_LEVEL_sum``", "Total number of efficiency arbiter write requests in flight." - "``TCC_EA0_RDREQ_LEVEL_sum``", "Total number of efficiency arbiter read requests in flight." - "``TCC_EA0_ATOMIC_sum``", "Total Number of 32-byte or 64-byte atomic requests going over the ``TC_EA0_wrreq`` interface." - "``TCC_EA0_ATOMIC_LEVEL_sum``", "Total number of efficiency arbiter atomic requests in flight." - "``TCC_EA0_RDREQ_sum``", "Total number of 32-byte or 64-byte read requests to efficiency arbiter." - "``TCC_EA0_RDREQ_32B_sum``", "Total number of 32-byte read requests to efficiency arbiter." - "``TCC_EA0_RD_UNCACHED_32B_sum``", "Total number of 32-byte efficiency arbiter reads due to uncached traffic." - "``TCC_EA0_RDREQ_IO_CREDIT_STALL_sum``", "Total number of cycles there is a stall due to the read request interface running out of IO credits." - "``TCC_EA0_RDREQ_GMI_CREDIT_STALL_sum``", "Total number of cycles there is a stall due to the read request interface running out of GMI credits." - "``TCC_EA0_RDREQ_DRAM_CREDIT_STALL_sum``", "Total number of cycles there is a stall due to the read request interface running out of DRAM credits." - "``TCC_EA0_RDREQ_DRAM_sum``", "Total number of 32-byte or 64-byte efficiency arbiter read requests to HBM." - "``TCC_EA0_WRREQ_DRAM_sum``", "Total number of 32-byte or 64-byte efficiency arbiter write requests to HBM." - "``TCC_HIT_sum``", "Total number of L2 cache hits." - "``TCC_MISS_sum``", "Total number of L2 cache misses." - "``TCC_NC_REQ_sum``", "Total number of non-coherently cached requests." - "``TCC_NORMAL_WRITEBACK_sum``", "Total number of writebacks due to requests that are not writeback requests." - "``TCC_NORMAL_EVICT_sum``", "Total number of evictions due to requests that are not invalidate or probe requests." - "``TCC_PROBE_sum``", "Total number of probe requests." - "``TCC_PROBE_ALL_sum``", "Total number of external probe requests with ``EA0_TCC_preq_all == 1``." - "``TCC_READ_sum``", "Total number of L2 cache read requests (including compressed reads but not metadata reads)." - "``TCC_REQ_sum``", "Total number of all types of L2 cache requests." - "``TCC_RW_REQ_sum``", "Total number of coherently cached with write requests." - "``TCC_STREAMING_REQ_sum``", "Total number of L2 cache streaming requests." - "``TCC_TAG_STALL_sum``", "Total number of cycles the normal request pipeline in the tag is stalled for any reason." - "``TCC_TOO_MANY_EA0_WRREQS_STALL_sum``", "Total number of cycles L2 cache is unable to send an efficiency arbiter write request due to it reaching its maximum capacity of pending efficiency arbiter write requests." - "``TCC_UC_REQ_sum``", "Total number of uncached requests." - "``TCC_WRITE_sum``", "Total number of L2 cache write requests." - "``TCC_WRITEBACK_sum``", "Total number of lines written back to the main memory including writebacks of dirty lines and uncached write or atomic requests." - "``TCC_WRREQ_STALL_max``", "Maximum number of cycles a write request is stalled." - -Hardware counters by, for, or over all texture cache per pipe instances ----------------------------------------------------------------------------------------------------------------- - -The following table shows the hardware counters *by* all texture cache per pipe instances. - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TCP_TA_TCP_STATE_READ_sum``", "Total number of state reads by ATCPPI" - "``TCP_TOTAL_CACHE_ACCESSES_sum``", "Total number of vector L1d accesses (including hits and misses)" - "``TCP_UTCL1_PERMISSION_MISS_sum``", "Total number of unified translation cache (L1) permission misses" - "``TCP_UTCL1_REQUEST_sum``", "Total number of address translation requests to unified translation cache (L1)" - "``TCP_UTCL1_TRANSLATION_MISS_sum``", "Total number of unified translation cache (L1) translation misses" - "``TCP_UTCL1_TRANSLATION_HIT_sum``", "Total number of unified translation cache (L1) translation hits" - -The following table shows the hardware counters *for* all texture cache per pipe instances. - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TCP_TCC_READ_REQ_LATENCY_sum``", "Total vector L1d to L2 request latency over all wavefronts for reads and atomics with return" - "``TCP_TCC_WRITE_REQ_LATENCY_sum``", "Total vector L1d to L2 request latency over all wavefronts for writes and atomics without return" - "``TCP_TCP_LATENCY_sum``", "Total wave access latency to vector L1d over all wavefronts" - -The following table shows the hardware counters *over* all texture cache per pipe instances. - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TCP_ATOMIC_TAGCONFLICT_STALL_CYCLES_sum``", "Total number of cycles tag RAM conflict stalls on an atomic" - "``TCP_GATE_EN1_sum``", "Total number of cycles vector L1d interface clocks are turned on" - "``TCP_GATE_EN2_sum``", "Total number of cycles vector L1d core clocks are turned on" - "``TCP_PENDING_STALL_CYCLES_sum``", "Total number of cycles vector L1d cache is stalled due to data pending from L2 Cache" - "``TCP_READ_TAGCONFLICT_STALL_CYCLES_sum``", "Total number of cycles tag RAM conflict stalls on a read" - "``TCP_TCC_ATOMIC_WITH_RET_REQ_sum``", "Total number of atomic requests to L2 cache with return" - "``TCP_TCC_ATOMIC_WITHOUT_RET_REQ_sum``", "Total number of atomic requests to L2 cache without return" - "``TCP_TCC_CC_READ_REQ_sum``", "Total number of coherently cached read requests to L2 cache" - "``TCP_TCC_CC_WRITE_REQ_sum``", "Total number of coherently cached write requests to L2 cache" - "``TCP_TCC_CC_ATOMIC_REQ_sum``", "Total number of coherently cached atomic requests to L2 cache" - "``TCP_TCC_NC_READ_REQ_sum``", "Total number of non-coherently cached read requests to L2 cache" - "``TCP_TCC_NC_WRITE_REQ_sum``", "Total number of non-coherently cached write requests to L2 cache" - "``TCP_TCC_NC_ATOMIC_REQ_sum``", "Total number of non-coherently cached atomic requests to L2 cache" - "``TCP_TCC_READ_REQ_sum``", "Total number of read requests to L2 cache" - "``TCP_TCC_RW_READ_REQ_sum``", "Total number of coherently cached with write read requests to L2 cache" - "``TCP_TCC_RW_WRITE_REQ_sum``", "Total number of coherently cached with write write requests to L2 cache" - "``TCP_TCC_RW_ATOMIC_REQ_sum``", "Total number of coherently cached with write atomic requests to L2 cache" - "``TCP_TCC_UC_READ_REQ_sum``", "Total number of uncached read requests to L2 cache" - "``TCP_TCC_UC_WRITE_REQ_sum``", "Total number of uncached write requests to L2 cache" - "``TCP_TCC_UC_ATOMIC_REQ_sum``", "Total number of uncached atomic requests to L2 cache" - "``TCP_TCC_WRITE_REQ_sum``", "Total number of write requests to L2 cache" - "``TCP_TCR_TCP_STALL_CYCLES_sum``", "Total number of cycles texture cache router stalls vector L1d" - "``TCP_TD_TCP_STALL_CYCLES_sum``", "Total number of cycles texture data unit stalls vector L1d" - "``TCP_TOTAL_ACCESSES_sum``", "Total number of vector L1d accesses" - "``TCP_TOTAL_READ_sum``", "Total number of vector L1d read accesses" - "``TCP_TOTAL_WRITE_sum``", "Total number of vector L1d write accesses" - "``TCP_TOTAL_ATOMIC_WITH_RET_sum``", "Total number of vector L1d atomic requests with return" - "``TCP_TOTAL_ATOMIC_WITHOUT_RET_sum``", "Total number of vector L1d atomic requests without return" - "``TCP_TOTAL_WRITEBACK_INVALIDATES_sum``", "Total number of vector L1d writebacks and invalidates" - "``TCP_VOLATILE_sum``", "Total number of L1 volatile pixels or buffers from texture addressing unit" - "``TCP_WRITE_TAGCONFLICT_STALL_CYCLES_sum``", "Total number of cycles tag RAM conflict stalls on a write" - -Hardware counter over all texture data unit instances --------------------------------------------------------- - -.. csv-table:: - :header: "Hardware counter", "Definition" - - "``TD_ATOMIC_WAVEFRONT_sum``", "Total number of atomic wavefront instructions" - "``TD_COALESCABLE_WAVEFRONT_sum``", "Total number of coalescable wavefronts according to texture addressing unit" - "``TD_LOAD_WAVEFRONT_sum``", "Total number of wavefront instructions (read, write, atomic)" - "``TD_SPI_STALL_sum``", "Total number of cycles texture data unit is stalled by shader processor input" - "``TD_STORE_WAVEFRONT_sum``", "Total number of write wavefront instructions" - "``TD_TC_STALL_sum``", "Total number of cycles texture data unit is stalled waiting for texture cache data" - "``TD_TD_BUSY_sum``", "Total number of texture data unit busy cycles while it is processing or waiting for data" diff --git a/docs/gpu-arch/mi300.md b/docs/gpu-arch/mi300.md deleted file mode 100644 index 8ac02393..00000000 --- a/docs/gpu-arch/mi300.md +++ /dev/null @@ -1,129 +0,0 @@ ---- -myst: - html_meta: - "description lang=en": "Learn about the AMD Instinct MI300 series architecture." - "keywords": "Instinct, MI300X, MI300A, microarchitecture, AMD, ROCm" ---- - -# AMD Instinct™ MI300 series microarchitecture - -The AMD Instinct MI300 series accelerators are based on the AMD CDNA 3 -architecture which was designed to deliver leadership performance for HPC, artificial intelligence (AI), and machine -learning (ML) workloads. The AMD Instinct MI300 series accelerators are well-suited for extreme scalability and compute performance, running -on everything from individual servers to the world’s largest exascale supercomputers. - -With the MI300 series, AMD is introducing the Accelerator Complex Die (XCD), which contains the -GPU computational elements of the processor along with the lower levels of the cache hierarchy. - -The following image depicts the structure of a single XCD in the AMD Instinct MI300 accelerator series. - -```{figure} ../images/gpu-arch/image007.png ---- -name: mi300-xcd -align: center ---- -XCD-level system architecture showing 40 Compute Units, each with 32 KB L1 cache, a Unified Compute System with 4 ACE Compute Accelerators, shared 4MB of L2 cache and an HWS Hardware Scheduler. -``` - -On the XCD, four Asynchronous Compute Engines (ACEs) send compute shader workgroups to the -Compute Units (CUs). The XCD has 40 CUs: 38 active CUs at the aggregate level and 2 disabled CUs for -yield management. The CUs all share a 4 MB L2 cache that serves to coalesce all memory traffic for the -die. With less than half of the CUs of the AMD Instinct MI200 Series compute die, the AMD CDNA™ 3 -XCD die is a smaller building block. However, it uses more advanced packaging and the processor -can include 6 or 8 XCDs for up to 304 CUs, roughly 40% more than MI250X. - -The MI300 Series integrate up to 8 vertically stacked XCDs, 8 stacks of -High-Bandwidth Memory 3 (HBM3) and 4 I/O dies (containing system -infrastructure) using the AMD Infinity Fabric™ technology as interconnect. - -The Matrix Cores inside the CDNA 3 CUs have significant improvements, emphasizing AI and machine -learning, enhancing throughput of existing data types while adding support for new data types. -CDNA 2 Matrix Cores support FP16 and BF16, while offering INT8 for inference. Compared to MI250X -accelerators, CDNA 3 Matrix Cores triple the performance for FP16 and BF16, while providing a -performance gain of 6.8 times for INT8. FP8 has a performance gain of 16 times compared to FP32, -while TF32 has a gain of 4 times compared to FP32. - -```{list-table} Peak-performance capabilities of the MI300X for different data types. -:header-rows: 1 -:name: mi300x-perf-table - -* - - Computation and Data Type - - FLOPS/CLOCK/CU - - Peak TFLOPS -* - - Matrix FP64 - - 256 - - 163.4 -* - - Vector FP64 - - 128 - - 81.7 -* - - Matrix FP32 - - 256 - - 163.4 -* - - Vector FP32 - - 256 - - 163.4 -* - - Vector TF32 - - 1024 - - 653.7 -* - - Matrix FP16 - - 2048 - - 1307.4 -* - - Matrix BF16 - - 2048 - - 1307.4 -* - - Matrix FP8 - - 4096 - - 2614.9 -* - - Matrix INT8 - - 4096 - - 2614.9 -``` - -The above table summarizes the aggregated peak performance of the AMD Instinct MI300X Open -Compute Platform (OCP) Open Accelerator Modules (OAMs) for different data types and command -processors. The middle column lists the peak performance (number of data elements processed in a -single instruction) of a single compute unit if a SIMD (or matrix) instruction is submitted in each clock -cycle. The third column lists the theoretical peak performance of the OAM. The theoretical aggregated -peak memory bandwidth of the GPU is 5.3 TB per second. - -The following image shows the block diagram of the APU (left) and the OAM package (right) both -connected via AMD Infinity Fabric™ network on-chip. - -```{figure} ../images/gpu-arch/image008.png ---- -name: mi300-arch -alt: -align: center ---- -MI300 series system architecture showing MI300A (left) with 6 XCDs and 3 CCDs, while the MI300X (right) has 8 XCDs. -``` - -## Node-level architecture - -```{figure} ../images/gpu-arch/mi300-node-level-arch.png ---- -name: mi300-node - -align: center ---- -MI300 series node-level architecture showing 8 fully interconnected MI300X OAM modules connected to (optional) PCIEe switches via retimers and HGX connectors. -``` - -The image above shows the node-level architecture of a system with AMD EPYC processors in a -dual-socket configuration and eight AMD Instinct MI300X accelerators. The MI300X OAMs attach to the -host system via PCIe Gen 5 x16 links (yellow lines). The GPUs are using seven high-bandwidth, -low-latency AMD Infinity Fabric™ links (red lines) to form a fully connected 8-GPU system. - - diff --git a/docs/gpu-arch/mi350.md b/docs/gpu-arch/mi350.md deleted file mode 100644 index 75736510..00000000 --- a/docs/gpu-arch/mi350.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -myst: - html_meta: - "description lang=en": "Learn about the AMD Instinct MI350 series architecture." - "keywords": "Instinct, MI350, microarchitecture, AMD, ROCm" ---- - -# AMD Instinct™ MI350 series microarchitecture - -**Coming Soon!** diff --git a/docs/images/gpu-arch/image001.png b/docs/images/gpu-arch/image001.png deleted file mode 100644 index befcd73c..00000000 Binary files a/docs/images/gpu-arch/image001.png and /dev/null differ diff --git a/docs/images/gpu-arch/image002.png b/docs/images/gpu-arch/image002.png deleted file mode 100644 index ecdadc32..00000000 Binary files a/docs/images/gpu-arch/image002.png and /dev/null differ diff --git a/docs/images/gpu-arch/image003.png b/docs/images/gpu-arch/image003.png deleted file mode 100644 index 611dcf78..00000000 Binary files a/docs/images/gpu-arch/image003.png and /dev/null differ diff --git a/docs/images/gpu-arch/image004.png b/docs/images/gpu-arch/image004.png deleted file mode 100644 index 0b802dfc..00000000 Binary files a/docs/images/gpu-arch/image004.png and /dev/null differ diff --git a/docs/images/gpu-arch/image005.png b/docs/images/gpu-arch/image005.png deleted file mode 100644 index f8a6f64b..00000000 Binary files a/docs/images/gpu-arch/image005.png and /dev/null differ diff --git a/docs/images/gpu-arch/image006.png b/docs/images/gpu-arch/image006.png deleted file mode 100644 index 88edc658..00000000 Binary files a/docs/images/gpu-arch/image006.png and /dev/null differ diff --git a/docs/images/gpu-arch/image007.png b/docs/images/gpu-arch/image007.png deleted file mode 100644 index 6dbcc17b..00000000 Binary files a/docs/images/gpu-arch/image007.png and /dev/null differ diff --git a/docs/images/gpu-arch/image008.png b/docs/images/gpu-arch/image008.png deleted file mode 100644 index 16ac7b3a..00000000 Binary files a/docs/images/gpu-arch/image008.png and /dev/null differ diff --git a/docs/images/gpu-arch/mi300-node-level-arch.png b/docs/images/gpu-arch/mi300-node-level-arch.png deleted file mode 100644 index c5256087..00000000 Binary files a/docs/images/gpu-arch/mi300-node-level-arch.png and /dev/null differ diff --git a/docs/index.md b/docs/index.md index e22589a3..c62ee76b 100644 --- a/docs/index.md +++ b/docs/index.md @@ -737,7 +737,7 @@ The AMD Instinct Documentation site provides comprehensive guides and technical :class-body: small-sd-card :class: small-sd-card +++ - +

Instinct Micro-architecture

Review hardware aspects of the AMD Instinct™ MI350, MI300, MI200 and MI100 series of GPU accelerators.

diff --git a/docs/resources/common-reference.md b/docs/resources/common-reference.md index b030ad3b..5dd609e1 100644 --- a/docs/resources/common-reference.md +++ b/docs/resources/common-reference.md @@ -22,7 +22,7 @@ :class-body: small-sd-card :class: small-sd-card +++ - +

Instinct Micro-architecture

Review hardware aspects of the AMD Instinct™ MI350, MI300, MI200 and MI100 series of GPU accelerators.

diff --git a/docs/sphinx/_toc.yml.in b/docs/sphinx/_toc.yml.in index 1e77ac93..1d342fc1 100644 --- a/docs/sphinx/_toc.yml.in +++ b/docs/sphinx/_toc.yml.in @@ -136,44 +136,8 @@ subtrees: - entries: - url: https://rocm.blogs.amd.com/software-tools-optimization/compute-memory-modes/README.html title: GPU Partitioning - - file: gpu-arch/gpu-arch.md + - url: https://rocm.docs.amd.com/en/latest/reference/gpu-arch/index.html title: Instinct Micro-architecture - subtrees: - - entries: - - file: gpu-arch/mi350.md - title: MI350 microarchitecture - subtrees: - - entries: - - url: https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-cdna4-instruction-set-architecture.pdf - title: AMD Instinct MI350/CDNA4 ISA - - url: https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-4-architecture-whitepaper.pdf - title: White paper - - file: gpu-arch/mi300.md - title: MI300 microarchitecture - subtrees: - - entries: - - url: https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/instruction-set-architectures/amd-instinct-mi300-cdna3-instruction-set-architecture.pdf - title: AMD Instinct MI300/CDNA3 ISA - - url: https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/white-papers/amd-cdna-3-white-paper.pdf - title: White paper - - file: gpu-arch/mi300-mi200-performance-counters.rst - title: MI300 and MI200 Performance counter - - file: gpu-arch/mi250.md - title: MI250 microarchitecture - subtrees: - - entries: - - url: https://www.amd.com/system/files/TechDocs/instinct-mi200-cdna2-instruction-set-architecture.pdf - title: AMD Instinct MI200/CDNA2 ISA - - url: https://www.amd.com/content/dam/amd/en/documents/instinct-business-docs/white-papers/amd-cdna2-white-paper.pdf - title: White paper - - file: gpu-arch/mi100.md - title: MI100 microarchitecture - subtrees: - - entries: - - url: https://www.amd.com/system/files/TechDocs/instinct-mi100-cdna1-shader-instruction-set-architecture%C2%A0.pdf - title: AMD Instinct MI100/CDNA1 ISA - - url: https://www.amd.com/content/dam/amd/en/documents/instinct-business-docs/white-papers/amd-cdna-white-paper.pdf - title: White paper - url: https://rocm.docs.amd.com/projects/amdsmi/en/latest/index.html title: AMD SMI API Doc - url: https://rocm.docs.amd.com/projects/HIP/en/latest/index.html