Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Dy2St]Get grad names when call append backward to fix high order gradient #53250

Merged
merged 15 commits into from
Apr 27, 2023
Merged

[Dy2St]Get grad names when call append backward to fix high order gradient #53250

merged 15 commits into from
Apr 27, 2023

Conversation

0x45f
Copy link
Contributor

@0x45f 0x45f commented Apr 23, 2023 •
edited
Loading

PR types

Bug fixes

PR changes

Others

Description

PCard-66972

1 背景

高阶情况下动转静的input、param、out对应的grad var name不再是简单的x@GRAD，他们的name可能的形式可能有x@GRAD@GRAD、grad/grad/x@GRAD、x@GRAD_0等。所以之前通过 x.name + '@Grad' 进行拼接的方式无法得到正确的grad var name，所以需要对动转静获取grad var name的模块进行升级以支持高阶的情况

2 本PR的修改

之前是通过遍历program中的var，正则匹配出grad var name，但是这种方式在一些情况下是错误的且难以维护。
在本PR中对获取grad var name的方式进行了升级，通过append_backward返回的grad_info_map来拿到对应的grad var，从而拿到grad var name，只要静态图下append_backward的逻辑是正确的那么就一定可以拿到正确的grad var name。

关于calc_gradient的修改，PR中只是将calc_gradient的主要逻辑抽离出来，形成一个calc_gradient_helper函数，提供给动转静模块使用

@paddle-bot
Copy link

paddle-bot bot commented Apr 23, 2023

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot
Copy link

paddle-bot bot commented Apr 23, 2023

❌ The PR is not created using PR's template. You can refer to this Demo.
Please use PR's template, it helps save our maintainers' time so that more developers get helped.

@0x45f 0x45f changed the title Add inputs to backend in dy2st [Dy2St]Get grad names when call append backward to fix high order gradient Apr 24, 2023
cxxly
cxxly previously approved these changes Apr 26, 2023
View reviewed changes
Copy link
Contributor

@cxxly cxxly left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for backward.py

Charles-hit
Charles-hit previously approved these changes Apr 26, 2023
View reviewed changes
Copy link
Contributor

@Charles-hit Charles-hit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm for test_dropout_op

Aurelius84
Aurelius84 reviewed Apr 27, 2023
View reviewed changes
python/paddle/fluid/backward.py Outdated Show resolved Hide resolved
python/paddle/fluid/backward.py Outdated Show resolved Hide resolved
python/paddle/fluid/tests/unittests/test_dropout_op.py Show resolved Hide resolved
@0x45f 0x45f dismissed stale reviews from Charles-hit and cxxly via bb49207 April 27, 2023 02:33
Aurelius84
Aurelius84 approved these changes Apr 27, 2023
View reviewed changes
Copy link
Contributor

@Aurelius84 Aurelius84 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@0x45f 0x45f mentioned this pull request Apr 27, 2023
Closed
XiaoguangHu01
XiaoguangHu01 approved these changes Apr 27, 2023
View reviewed changes
Copy link
Contributor

@XiaoguangHu01 XiaoguangHu01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

zhiqiu
zhiqiu approved these changes Apr 27, 2023
View reviewed changes
Copy link
Contributor

@zhiqiu zhiqiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM for atol

@0x45f 0x45f merged commit 2d17df9 into PaddlePaddle:develop Apr 27, 2023
@0x45f 0x45f deleted the add-inputs-for-backend branch April 27, 2023 09:17
0x45f added a commit to 0x45f/Paddle that referenced this pull request Apr 27, 2023
@0x45f
[Dy2St]Get grad names when call append backward to fix high order gra…
723ceef 
…dient (PaddlePaddle#53250)

[Dy2St]Get grad names when call append backward to fix high order gradient (PaddlePaddle#53250)
Aurelius84 pushed a commit to Aurelius84/Paddle that referenced this pull request May 4, 2023
@0x45f @Aurelius84
[Dy2St]Get grad names when call append backward to fix high order gra…
e87a3a3 
…dient (PaddlePaddle#53250)

[Dy2St]Get grad names when call append backward to fix high order gradient (PaddlePaddle#53250)
@Aurelius84 Aurelius84 mentioned this pull request May 4, 2023
Merged
XiaoguangHu01 pushed a commit that referenced this pull request May 5, 2023
@Aurelius84 @0x45f
[Dy2St]Get grad names when call append backward to fix high order gra…
584d610 
…dient (#53250) (#53493)

[Dy2St]Get grad names when call append backward to fix high order gradient (#53250)

Co-authored-by: WangZhen <23097963+0x45f@users.noreply.github.com>
@cxxly cxxly mentioned this pull request May 11, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@XiaoguangHu01 XiaoguangHu01 XiaoguangHu01 approved these changes

@wanghuancoder wanghuancoder wanghuancoder approved these changes

@chenwhql chenwhql chenwhql approved these changes

@Aurelius84 Aurelius84 Aurelius84 approved these changes

@qingqing01 qingqing01 qingqing01 approved these changes

@zhiqiu zhiqiu zhiqiu approved these changes

@Charles-hit Charles-hit Charles-hit left review comments

@cxxly cxxly cxxly left review comments

@cyber-pioneer cyber-pioneer Awaiting requested review from cyber-pioneer

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
9 participants
@0x45f @zhiqiu @qingqing01 @cxxly @Aurelius84 @chenwhql @wanghuancoder @XiaoguangHu01 @Charles-hit