Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CUDA] Fix NumericLimits #22738

Merged
merged 2 commits into from
Nov 6, 2024
Merged

[CUDA] Fix NumericLimits #22738

merged 2 commits into from
Nov 6, 2024

Conversation

tianleiwu
Copy link
Contributor

@tianleiwu tianleiwu commented Nov 5, 2024 •
edited
Loading

Description

  • Fix NumericLimits<float> that used infinity as max, which is not consistent with std::numeric_limits<float>::max()
    In Windows, (float)(1e+300) is used for INFINITY, which causes compiler error in Visual Studio 2022 v17.12 Preview 5.
  • Rename NumericLimits<T>::Min to Lowest to be consistent with std::numeric_limits
  • Fix topk implementation: use NumericLimits<CudaT> instead of NumericLimits<T> in kernel. That could avoid defining a confusing defintion of NumericLimits<MLFloat16> that returns half instead of MLFloat16.
  • Use CUDART_MAX_NORMAL_FP16 if possible. It sets bits value directly, which is faster than converting float to half.

Note that NumericLimits does not support __nv_bfloat16 and _nv_fp8_e4m3 and __nv_fp8_e5m2 right now.

Motivation and Context

#22728

@tianleiwu tianleiwu marked this pull request as draft November 5, 2024 22:16
@tianleiwu tianleiwu marked this pull request as ready for review November 6, 2024 00:27
@tianleiwu tianleiwu requested review from yufenglee and snnn November 6, 2024 01:22
@tianleiwu tianleiwu merged commit d993ec3 into main Nov 6, 2024
91 checks passed
@tianleiwu tianleiwu deleted the tlwu/fix_numeric_limits branch November 6, 2024 17:53
ishwar-raut1 pushed a commit to ishwar-raut1/onnxruntime that referenced this pull request Nov 19, 2024
@tianleiwu
[CUDA] Fix NumericLimits (microsoft#22738)
3d6f2db 
### Description
* Fix `NumericLimits<float>` that used infinity as max, which is not
consistent with `std::numeric_limits<float>::max()`
In Windows, (float)(1e+300) is used for INFINITY, which causes compiler
error in Visual Studio 2022 v17.12 Preview 5.
* Rename `NumericLimits<T>::Min` to Lowest to be consistent with
std::numeric_limits
* Fix topk implementation: use `NumericLimits<CudaT>` instead of
`NumericLimits<T>` in kernel. That could avoid defining a confusing
defintion of `NumericLimits<MLFloat16>` that returns half instead of
MLFloat16.
* Use CUDART_MAX_NORMAL_FP16 if possible. It sets bits value directly,
which is faster than converting float to half.

Note that NumericLimits does not support __nv_bfloat16 and _nv_fp8_e4m3
and __nv_fp8_e5m2 right now.

### Motivation and Context
microsoft#22728
guschmue pushed a commit that referenced this pull request Dec 2, 2024
@tianleiwu @guschmue
[CUDA] Fix NumericLimits (#22738)
c5cf977 
### Description
* Fix `NumericLimits<float>` that used infinity as max, which is not
consistent with `std::numeric_limits<float>::max()`
In Windows, (float)(1e+300) is used for INFINITY, which causes compiler
error in Visual Studio 2022 v17.12 Preview 5.
* Rename `NumericLimits<T>::Min` to Lowest to be consistent with
std::numeric_limits
* Fix topk implementation: use `NumericLimits<CudaT>` instead of
`NumericLimits<T>` in kernel. That could avoid defining a confusing
defintion of `NumericLimits<MLFloat16>` that returns half instead of
MLFloat16.
* Use CUDART_MAX_NORMAL_FP16 if possible. It sets bits value directly,
which is faster than converting float to half.

Note that NumericLimits does not support __nv_bfloat16 and _nv_fp8_e4m3
and __nv_fp8_e5m2 right now.

### Motivation and Context
#22728
ankitm3k pushed a commit to intel/onnxruntime that referenced this pull request Dec 11, 2024
@tianleiwu @ankitm3k
[CUDA] Fix NumericLimits (microsoft#22738)
7881d16 
### Description
* Fix `NumericLimits<float>` that used infinity as max, which is not
consistent with `std::numeric_limits<float>::max()`
In Windows, (float)(1e+300) is used for INFINITY, which causes compiler
error in Visual Studio 2022 v17.12 Preview 5.
* Rename `NumericLimits<T>::Min` to Lowest to be consistent with
std::numeric_limits
* Fix topk implementation: use `NumericLimits<CudaT>` instead of
`NumericLimits<T>` in kernel. That could avoid defining a confusing
defintion of `NumericLimits<MLFloat16>` that returns half instead of
MLFloat16.
* Use CUDART_MAX_NORMAL_FP16 if possible. It sets bits value directly,
which is faster than converting float to half.

Note that NumericLimits does not support __nv_bfloat16 and _nv_fp8_e4m3
and __nv_fp8_e5m2 right now.

### Motivation and Context
microsoft#22728
ankitm3k pushed a commit to intel/onnxruntime that referenced this pull request Dec 11, 2024
@tianleiwu @ankitm3k
[CUDA] Fix NumericLimits (microsoft#22738)
12e77df 
### Description
* Fix `NumericLimits<float>` that used infinity as max, which is not
consistent with `std::numeric_limits<float>::max()`
In Windows, (float)(1e+300) is used for INFINITY, which causes compiler
error in Visual Studio 2022 v17.12 Preview 5.
* Rename `NumericLimits<T>::Min` to Lowest to be consistent with
std::numeric_limits
* Fix topk implementation: use `NumericLimits<CudaT>` instead of
`NumericLimits<T>` in kernel. That could avoid defining a confusing
defintion of `NumericLimits<MLFloat16>` that returns half instead of
MLFloat16.
* Use CUDART_MAX_NORMAL_FP16 if possible. It sets bits value directly,
which is faster than converting float to half.

Note that NumericLimits does not support __nv_bfloat16 and _nv_fp8_e4m3
and __nv_fp8_e5m2 right now.

### Motivation and Context
microsoft#22728
adrianlizarraga pushed a commit that referenced this pull request Feb 4, 2025
@tianleiwu @adrianlizarraga
[CUDA] Fix NumericLimits (#22738)
3848811 
### Description
* Fix `NumericLimits<float>` that used infinity as max, which is not
consistent with `std::numeric_limits<float>::max()`
In Windows, (float)(1e+300) is used for INFINITY, which causes compiler
error in Visual Studio 2022 v17.12 Preview 5.
* Rename `NumericLimits<T>::Min` to Lowest to be consistent with
std::numeric_limits
* Fix topk implementation: use `NumericLimits<CudaT>` instead of
`NumericLimits<T>` in kernel. That could avoid defining a confusing
defintion of `NumericLimits<MLFloat16>` that returns half instead of
MLFloat16.
* Use CUDART_MAX_NORMAL_FP16 if possible. It sets bits value directly,
which is faster than converting float to half.

Note that NumericLimits does not support __nv_bfloat16 and _nv_fp8_e4m3
and __nv_fp8_e5m2 right now.

### Motivation and Context
#22728
@adrianlizarraga adrianlizarraga mentioned this pull request Feb 4, 2025
Merged
jywu-msft pushed a commit that referenced this pull request Feb 10, 2025
@adrianlizarraga @snnn
@tianleiwu @jchen351 @mszhanyi @carzh
[ORT 1.20.2 Release] Cherry pick 1st round (#23574)
8608bf0 
### Description
- Update ORT version to 1.20.2
- Cherry-pick commits:
  - #23243
  - #22738
  - #22868
  - #23281
  - #22543
  - #22566
  - #23308
  - #23017
  - [Main feature] #23368



### Motivation and Context
<!-- - Why is this change required? What problem does it solve?
- If it fixes an open issue, please link to the issue here. -->

---------

Co-authored-by: Changming Sun <chasun@microsoft.com>
Co-authored-by: Tianlei Wu <tlwu@microsoft.com>
Co-authored-by: Jian Chen <cjian@microsoft.com>
Co-authored-by: Yi Zhang <zhanyi@microsoft.com>
Co-authored-by: Caroline Zhu <wolfivyaura@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@snnn snnn snnn approved these changes

@yufenglee yufenglee Awaiting requested review from yufenglee

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@tianleiwu @snnn