Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Dygraph] Fix Perf of FusedFeedForward and FusedAttention with AllReduce #46780

Merged
merged 2 commits into from
Oct 9, 2022
Merged

[Dygraph] Fix Perf of FusedFeedForward and FusedAttention with AllReduce #46780

merged 2 commits into from
Oct 9, 2022

Conversation

haohongxiang
Copy link
Contributor

@haohongxiang haohongxiang commented Oct 8, 2022 •
edited
Loading

PR types

Bug fixes

PR changes

OPs

Describe

[Dygraph] Fix Perf of FusedFeedForward and FusedAttention with AllReduce

CE case：PaddleNLP_gpt3_bs16_fp16_DP1-MP8-PP4_N4C32 + fuse策略
baseline：88367.7（老动态图）
PR前：74893（新动态图）
PR后：88915（+18.7%，与老动态图持平）

@paddle-bot
Copy link

paddle-bot bot commented Oct 8, 2022

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@FeixLiu FeixLiu merged commit 078e8c7 into PaddlePaddle:develop Oct 9, 2022
haohongxiang added a commit to haohongxiang/Paddle that referenced this pull request Oct 17, 2022
@haohongxiang
[Dygraph] Fix Perf of FusedFeedForward and FusedAttention with AllRed…
b3eb592 
…uce (PaddlePaddle#46780)
@haohongxiang haohongxiang mentioned this pull request Oct 18, 2022
Merged
fuyinno4 pushed a commit that referenced this pull request Oct 18, 2022
@haohongxiang
[cherry-pick] Fix perf issues of mp/pp/fuse in eager mode (#47071)
b84edd9 
* [Dygraph] Fix performance of pp+mp by using send/recv_calc_stream instead of send/recv (#46116)

* [Dygraph] Fix Perf of FusedFeedForward and FusedAttention with AllReduce (#46780)

* update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@LiYuRio LiYuRio LiYuRio approved these changes

@FeixLiu FeixLiu FeixLiu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@haohongxiang @FeixLiu @LiYuRio