Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce PerAxis Quantized constraint for StableHLO Quantized OPs #2007

Merged
merged 2 commits into from
Feb 27, 2024

Conversation

abhigunj
Copy link
Member

@abhigunj abhigunj commented Feb 14, 2024

StableHLO OPs supporting Quantization can be categorized into following three types
a. Only PerTensor Quantized Tensors
b. Only PerAxis Quantized Tensors (no OP in this category for now)
c. Both PerTensor and PerAxis Quantized Tensors
PerAxis constraint from the PR allow only PerTensorQuantized Tensor inputs to type (a) OPs and don't allow PerAxis Quantized Tensors.
Also,
Added negative test cases to validate this behavior
Added positive test cases for PerTensor , PerAxis Quantized Tensor support

Excluded OPs

  • Deprecated :BroadCast and Call
  • Not Specced :
     dynamic_iota, create_token, dynamic_broadcast_in_dim, cross-replica-sum, einsum, unary_einsum, dynamic_reshape, set_dimension_size, trace, return, torch_index_select, real_dynamic_slice, dynamic_pad, dynamic_gather, dynamic_conv, dynamic_reshape_shape, cstr_reshapable

@GleasonK
Copy link
Member

Delegating my review to @sdasgup3. I took a scan over the code / tests and from a high level everything LGTM.

Copy link
Member

@sdasgup3 sdasgup3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for the work. Here is some of my initial reviews which I want to publish soon. I need to check the test files in my follow up review.

Here is the remaining list of ops which are uncovered and the following support need to be added.

  • constant : is current using HLO_StaticShapeTensor which per this change has per-tensor output type. It should be both.
  • iota uses HLO_StaticShapeIntFpOrComplexTensor which is not updated. iota is expected to support per-tensor
  • infeed: Accepts both per-tensor and per-axis.
  • recv: should take both per-tensor and per-axis.
  • If, Case, While: The output supports both.
  • GetTupleElement / Tuple:  supports both.
  • broadcast: update the type coinstraints treating the in/out types similar to brodcast_in_dim
  • custom-call: both - fft: fix the result type as per spec.
  • transpose: both are supported. 

Nit:
 -  dynamic_iota, create_token, dynamic_broadcast_in_dim, cross-replica-sum, einsum, unary_einsum, dynamic_reshape, set_dimension_size, trace, return, torch_index_select, real_dynamic_slice, dynamic_pad, dynamic_gather, dynamic_conv, dynamic_reshape_shape, cstr_reshapable: Let's mention in the description that we are excluding the above stablehlo ops as they are not specced.

As follow up PRs:

  • collective_broadcast should also support per-tensor type like other distribution ops.  
    get_dimension_size: fix the spec
    Please free to open PRs for the above items.

stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/StablehloOps.td Show resolved Hide resolved
stablehlo/dialect/StablehloOps.td Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
stablehlo/tests/ops_stablehlo_quantized.mlir Outdated Show resolved Hide resolved
@abhigunj
Copy link
Member Author

Thanks again for the work. Here is some of my initial reviews which I want to publish soon. I need to check the test files in my follow up review.

Here is the remaining list of ops which are uncovered and the following support need to be added.

  • constant : is current using HLO_StaticShapeTensor which per this change has per-tensor output type. It should be both.
  • iota uses HLO_StaticShapeIntFpOrComplexTensor which is not updated. iota is expected to support per-tensor
  • infeed: Accepts both per-tensor and per-axis.
  • recv: should take both per-tensor and per-axis.
  • If, Case, While: The output supports both.
  • GetTupleElement / Tuple:  supports both.
  • broadcast: update the type coinstraints treating the in/out types similar to brodcast_in_dim
  • custom-call: both - fft: fix the result type as per spec.

Thanks for the thorough review, these OPs fall into result Quantized category, which I incorrectly ignored during audit. Updated the OP def. Updated the audit sheet.

  • transpose: both are supported.

Already taken care of?

Nit:  -  dynamic_iota, create_token, dynamic_broadcast_in_dim, cross-replica-sum, einsum, unary_einsum, dynamic_reshape, set_dimension_size, trace, return, torch_index_select, real_dynamic_slice, dynamic_pad, dynamic_gather, dynamic_conv, dynamic_reshape_shape, cstr_reshapable: Let's mention in the description that we are excluding the above stablehlo ops as they are not specced.

Done

As follow up PRs:

  • collective_broadcast should also support per-tensor type like other distribution ops.
  • get_dimension_size: fix the spec
    Please free to open PRs for the above items.

Yes, will create separate PR as it involves changes to the spec.

Copy link
Member

@sdasgup3 sdasgup3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm with some minor comments.

stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/Base.td Outdated Show resolved Hide resolved
stablehlo/dialect/StablehloOps.td Show resolved Hide resolved
stablehlo/dialect/Base.td Show resolved Hide resolved
Copy link
Member

@GleasonK GleasonK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll think more on naming, but that shouldn't block this from going in since tablegen variable names don't impact the generated code / impl.

stablehlo/dialect/Base.td Show resolved Hide resolved
stablehlo/dialect/Base.td Show resolved Hide resolved
stablehlo/dialect/StablehloOps.td Show resolved Hide resolved
@abhigunj abhigunj merged commit 19fd17d into openxla:main Feb 27, 2024
10 checks passed
abhigunj added a commit that referenced this pull request Feb 27, 2024
and make Quantized test compatible. This was missed during resolving
merged conflict
 for #2007
GleasonK added a commit that referenced this pull request Feb 27, 2024
Custom call op permits just about everything: Tuple, Tensor, Token,
Unranked, all quantization, etc. This change restores its ability to
operate on unranked tensors.

This was squashed in a merge conflict between #2045 and #2007. Adding a
testpoint to avoid this issue going forward.
@abhigunj abhigunj added the Migrate to MHLO PR that needs to be migrated to MLIR-HLO label Feb 28, 2024
abhigunj added a commit that referenced this pull request Mar 5, 2024
* made `isCompatibleElementTypeForHloTypeInference` stricter to return
error for {not Quantize, Quantize}, {per-axis Quantized, per-tensor
Quantized} cases
* `AddOp` VHLO Test failures : addressed test failures because {not
Quantize, Quantize} is not allowed
* CorrectedTraits for `CholeskyOp` and `ClampOp` to match it with the
spec

~~Note: This PR is based on in review PR
#2007

Follow up PR will add/update OP verifiers for OPs which need special
handling
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Migrate to MHLO PR that needs to be migrated to MLIR-HLO Quantization
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants