Skip to content

[GPU] Add BF16 support for quantize operations - #37230

Open
Prithviraj-R wants to merge 1 commit into
openvinotoolkit:masterfrom
Prithviraj-R:quantize_bf16
Open

[GPU] Add BF16 support for quantize operations#37230
Prithviraj-R wants to merge 1 commit into
openvinotoolkit:masterfrom
Prithviraj-R:quantize_bf16

Conversation

@Prithviraj-R

@Prithviraj-R Prithviraj-R commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

[GPU] Add BF16 support for quantize operations

Details:

  • Add BF16 data type support in quantize kernel implementations
  • Update quantize OpenCL kernels to handle BF16 data type
  • Add BF16 test cases for quantize operations

Implementation changes:

  • Updated all quantize OpenCL kernels (ref, scale_shift_opt, scale_shift_vload8_opt) to support BF16 input/output by using existing BF16 conversion utilities
  • Added BF16 format support in quantize.cpp OCL implementation
  • Extended quantize kernel selectors (ref, scale_shift_opt, scale_shift_vload8_opt) to accept BF16 data type
  • Added BF16 test case in quantize_gpu_test.cpp with bs_fs_yx_bsv16_fsv32 format

Changed files:

src/plugins/intel_gpu/src/graph/impls/ocl/quantize.cpp
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_ref.cl
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_scale_shift_opt.cl
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_scale_shift_vload8_opt.cl
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_ref.cpp
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_scale_shift_opt.cpp
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_scale_shift_vload8_opt.cpp
src/plugins/intel_gpu/tests/unit/test_cases/quantize_gpu_test.cpp

Reproduction step and snapshot (if applicable. Do not attach for customer model):

NA

Problematic graph:

NA

Checklist:

  • Is it a proper fix? (not a workaround)
  • Did you include test case for this fix, if necessary? Yes
  • Did you review existing test that can be extended to cover this scenario? Which test did you review? quantize_gpu_test.cpp

Tickets:

AI Assistance:

  • AI assistance used: no

@Prithviraj-R
Prithviraj-R requested review from a team as code owners August 4, 2026 12:09
@github-actions github-actions Bot added the category: GPU OpenVINO GPU plugin label Aug 4, 2026
@Prithviraj-R
Prithviraj-R force-pushed the quantize_bf16 branch 2 times, most recently from d6ac4e8 to f109068 Compare August 4, 2026 12:17
@p-durandin
p-durandin requested a lite review from Copilot August 4, 2026 13:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the Intel GPU quantize OCL implementation to support BF16 tensors by propagating BF16 enablement through kernel selectors, OCL kernel code paths (decode/compute/output handling), and adding a unit-test instantiation for BF16 inputs.

Changes:

  • Enable BF16 as an accepted input/output datatype in quantize kernel selectors (ref / scale_shift_opt / scale_shift_vload8_opt).
  • Update quantize OpenCL kernels to use compute-type decode paths suitable for BF16 (and related conversion macros).
  • Add BF16 handling and a BF16 test instantiation in quantize_gpu_test.cpp.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
src/plugins/intel_gpu/src/graph/impls/ocl/quantize.cpp Registers OCL quantize implementation map entries for BF16 (including yxfb).
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_ref.cl Switches quantize ref kernel to compute-type decode and saturating output conversion for non-FP outputs.
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_scale_shift_opt.cl Uses per-input compute types + decode macros for scale/shift/range tensors to support BF16.
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_scale_shift_vload8_opt.cl Updates vectorized vload8 kernel to compute via decoded compute vectors for BF16 support.
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_ref.cpp Enables BF16 in supported key for the ref quantize kernel.
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_scale_shift_opt.cpp Enables BF16 in supported key for scale_shift_opt kernel.
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_scale_shift_vload8_opt.cpp Enables BF16 in supported key for vload8 optimized kernel.
src/plugins/intel_gpu/tests/unit/test_cases/quantize_gpu_test.cpp Adds BF16 random fill/transfer/compare plumbing and a BF16 instantiation.

Comment thread src/plugins/intel_gpu/tests/unit/test_cases/quantize_gpu_test.cpp
@p-durandin

Copy link
Copy Markdown
Contributor

@Prithviraj-R please consider copilot remarks

#### Details:
- Add BF16 data type support in quantize kernel implementations
- Update quantize OpenCL kernels to handle BF16 data type
- Add BF16 test cases for quantize operations

#### Implementation changes:
- Updated all quantize OpenCL kernels (ref, scale_shift_opt, scale_shift_vload8_opt) to support BF16 input/output by using existing BF16 conversion utilities
- Added BF16 format support in quantize.cpp OCL implementation
- Extended quantize kernel selectors (ref, scale_shift_opt, scale_shift_vload8_opt) to accept BF16 data type
- Added BF16 test case in quantize_gpu_test.cpp with bs_fs_yx_bsv16_fsv32 format

#### Changed files:
src/plugins/intel_gpu/src/graph/impls/ocl/quantize.cpp
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_ref.cl
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_scale_shift_opt.cl
src/plugins/intel_gpu/src/kernel_selector/cl_kernels/quantize_gpu_scale_shift_vload8_opt.cl
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_ref.cpp
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_scale_shift_opt.cpp
src/plugins/intel_gpu/src/kernel_selector/kernels/quantize/quantize_kernel_scale_shift_vload8_opt.cpp
src/plugins/intel_gpu/tests/unit/test_cases/quantize_gpu_test.cpp

#### Reproduction step and snapshot (if applicable. Do not attach for customer model):
NA

#### Problematic graph:
NA

#### Checklist:
- [x] Is it a proper fix? (not a workaround)
- [x] Did you include test case for this fix, if necessary? Yes
- [x] Did you review existing test that can be extended to cover this scenario? Which test did you review? quantize_gpu_test.cpp

#### Tickets:
- CVS-189707

#### AI Assistance:
- AI assistance used: no
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: GPU OpenVINO GPU plugin

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants