support Llama3.1 8B 128K generation on single GPU 80GB

[GuoxiaWang]

PR types

Function optimization

PR changes

Models

Description

support Llama3.1 8B 128K generation on single GPU 80GB

[paddle-bot]

Thanks for your contribution!

[codecov]

Codecov Report

Attention: Patch coverage is 35.89744% with 25 lines in your changes missing coverage. Please review.

Project coverage is 55.50%. Comparing base (5fd6dd2) to head (739f454).
Report is 226 commits behind head on develop.

Files with missing lines	Patch %	Lines
paddlenlp/transformers/llama/modeling.py	30.55%	25 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8811      +/-   ##
===========================================
+ Coverage    55.47%   55.50%   +0.03%     
===========================================
  Files          631      631              
  Lines        98468    98542      +74     
===========================================
+ Hits         54621    54699      +78     
+ Misses       43847    43843       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[wawltor]

LGTM