AssertionError: 2 is not divisible by 4

[Harryjun]

我遇到了个这个错误
qwen2-1.5b 学生模型 MPSIZE 4
qwen2-14b 教师模型 MPSIZE 4
八卡A100

ng_qwen2.py", line 642, in __init__
[rank4]:     self.self_attn = QWEN2_ATTENTION_CLASSES[config._attn_implementation](config, layer_idx)
[rank4]:   File "/mnt/bn/lq1-nas-menglingjun/query_analysis_v1_train/research-HEAD/q2/miniLLM/LMOps-main/minillm/transformers/src/transformers/models/qwen2/modeling_qwen2.py", line 311, in __init__
[rank4]:     self.num_key_value_heads = divide(self.num_key_value_heads, self.world_size) # n_kv_p
[rank4]:   File "/mnt/bn/lq1-nas-menglingjun/query_analysis_v1_train/research-HEAD/q2/miniLLM/LMOps-main/minillm/transformers/src/transformers/mpu/utils.py", line 18, in divide
[rank4]:     ensure_divisibility(numerator, denominator)
[rank4]:   File "/mnt/bn/lq1-nas-menglingjun/query_analysis_v1_train/research-HEAD/q2/miniLLM/LMOps-main/minillm/transformers/src/transformers/mpu/utils.py", line 11, in ensure_divisibility
[rank4]:     assert numerator % denominator == 0, '{} is not divisible by {}'.format(
[rank4]: AssertionError: 2 is not divisible by 4