DocOwl1.5-Omni 推理失败

[FanWan]

采用官方库安装环境，推理报错。

Some weights of MPLUGDocOwlLlamaForCausalLM were not initialized from the model checkpoint at /record/llm_models/DocOwl1.5-Omni and are newly initialized: ['model.layers.9.self_attn.rotary_emb.inv_freq', 'model.layers.12.self_attn.rotary_emb.inv_freq', 'model.layers.16.self_attn.rotary_emb.inv_freq', 'model.layers.14.self_attn.rotary_emb.inv_freq', 'model.layers.24.self_attn.rotary_emb.inv_freq', 'model.layers.21.self_attn.rotary_emb.inv_freq', 'model.layers.11.self_attn.rotary_emb.inv_freq', 'model.layers.10.self_attn.rotary_emb.inv_freq', 'model.layers.4.self_attn.rotary_emb.inv_freq', 'model.layers.7.self_attn.rotary_emb.inv_freq', 'model.layers.5.self_attn.rotary_emb.inv_freq', 'model.layers.23.self_attn.rotary_emb.inv_freq', 'model.layers.22.self_attn.rotary_emb.inv_freq', 'model.layers.17.self_attn.rotary_emb.inv_freq', 'model.layers.6.self_attn.rotary_emb.inv_freq', 'model.layers.2.self_attn.rotary_emb.inv_freq', 'model.layers.3.self_attn.rotary_emb.inv_freq', 'model.layers.31.self_attn.rotary_emb.inv_freq', 'model.layers.1.self_attn.rotary_emb.inv_freq', 'model.layers.29.self_attn.rotary_emb.inv_freq', 'model.layers.20.self_attn.rotary_emb.inv_freq', 'model.layers.26.self_attn.rotary_emb.inv_freq', 'model.layers.13.self_attn.rotary_emb.inv_freq', 'model.layers.18.self_attn.rotary_emb.inv_freq', 'model.layers.15.self_attn.rotary_emb.inv_freq', 'model.layers.27.self_attn.rotary_emb.inv_freq', 'model.layers.28.self_attn.rotary_emb.inv_freq', 'model.layers.30.self_attn.rotary_emb.inv_freq', 'model.layers.8.self_attn.rotary_emb.inv_freq', 'model.layers.25.self_attn.rotary_emb.inv_freq', 'model.layers.19.self_attn.rotary_emb.inv_freq']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
load model from /record/llm_models/DocOwl1.5-Omni
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Setting pad_token_id to eos_token_id:2 for open-end generation.
Traceback (most recent call last):
File "/share/wanfan01/open_llm/mPLUG-DocOwl/DocOwl1.5/docowl_infer.py", line 85, in
answer = docowl.inference(image, query)
File "/share/wanfan01/open_llm/mPLUG-DocOwl/DocOwl1.5/docowl_infer.py", line 48, in inference
output_ids = self.model.generate(
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
return func(*args, **kwargs)
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/transformers/generation/utils.py", line 1515, in generate
return self.greedy_search(
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/transformers/generation/utils.py", line 2332, in greedy_search
outputs = self(
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/accelerate/hooks.py", line 169, in new_forward
output = module._old_forward(*args, **kwargs)
File "/share/wanfan01/open_llm/mPLUG-DocOwl/DocOwl1.5/mplug_docowl/model/modeling_mplug_docowl.py", line 246, in forward
outputs = self.model(
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/share/wanfan01/open_llm/mPLUG-DocOwl/DocOwl1.5/mplug_docowl/model/modeling_llama2.py", line 347, in model_forward
layer_outputs = decoder_layer(
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/accelerate/hooks.py", line 169, in new_forward
output = module._old_forward(*args, **kwargs)
File "/share/wanfan01/open_llm/mPLUG-DocOwl/DocOwl1.5/mplug_docowl/model/modeling_llama2.py", line 224, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/work/anaconda3/envs/docowl/lib/python3.10/site-packages/accelerate/hooks.py", line 169, in new_forward
output = module._old_forward(*args, **kwargs)
File "/share/wanfan01/open_llm/mPLUG-DocOwl/DocOwl1.5/mplug_docowl/model/modeling_llama2.py", line 139, in forward
key_states = repeat_kv(key_states, self.num_key_value_groups)
NameError: name 'repeat_kv' is not defined

[CourageKeenTriumph]

请问解决了吗