Add RingFlashAttention for context parallel

[zhangyuqin1998]

PR types

New features

PR changes

Models

Description

为fleet的context parallel增加ring flash attention的支持

[paddle-bot]

Thanks for your contribution!

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

Attention: Patch coverage is 14.28571% with 246 lines in your changes are missing coverage. Please review.

Project coverage is 54.17%. Comparing base (b36b6a0) to head (4e88520).

Files	Patch %	Lines
paddlenlp/transformers/ring_flash_attention.py	16.14%	135 Missing ⚠️
paddlenlp/transformers/context_parallel_utils.py	6.66%	56 Missing ⚠️
paddlenlp/trainer/training_args.py	4.76%	20 Missing ⚠️
paddlenlp/transformers/llama/modeling.py	25.00%	18 Missing ⚠️
paddlenlp/transformers/llama/fusion_ops.py	15.38%	11 Missing ⚠️
paddlenlp/trainer/trainer.py	14.28%	6 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #8383      +/-   ##
===========================================
- Coverage    54.29%   54.17%   -0.13%     
===========================================
  Files          617      619       +2     
  Lines        96339    96618     +279     
===========================================
+ Hits         52310    52339      +29     
- Misses       44029    44279     +250     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ForFishes]

TODO：batch_isend_irecv异步流下，无法wait，需要修复。对性能有影响。