[Auto Parallel Performance] Support BF16 Training

[JZ-LIANG]

PR types

Function optimization

PR changes

Others

Describe

Support BF16 Training for Auto Parallel. Generalizing the amp-o2 pass to support both "fp16" and "bf16".

Revise and Unify AMP User API: organizing amp config into a two-level selection:
first select the amp-level: O1, O2, O3
second select the amp-dtype: float16 or bfloat16

Support Multiple-Subblock inference like in Bloom model.

example:
AMP-O2 with float16

    strategy = auto.Strategy()
    strategy.auto_mode = "semi"
    amp = strategy.amp
    amp.enable = True
    amp.level = "o2"
    amp.custom_black_list = [
        'c_softmax_with_cross_entropy',
        'elementwise_div',
        'reduce_sum',
    ]

AMP-O2 with bfloat16

    strategy = auto.Strategy()
    strategy.auto_mode = "semi"
    amp = strategy.amp
    amp.enable = True
    amp.dtype = "bfloat16"
    amp.level = "o2"
    amp.custom_black_list = [
        'c_softmax_with_cross_entropy',
        'elementwise_div',
        'reduce_sum',
    ]

[paddle-bot]

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[aoyulong]

LGTM. For now, this interface of bf16 pass is OK. In the future, this may be changed based on the feedback.