IndexError: index out of range in self on training_reward_model

[SeekPoint]

(gh_llama-trl) ub2004@ub2004-B85M-A0:~/llm_dev/llama-trl$ python training_reward_model.py --model_name '/data-ssd-1t/hf_model/llama-7b-hf' --dataset_name './data/comparison_data.json' --output_dir './checkpoints/training_reward_model/'

/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/optimization.py:407: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set no_deprecation_warning=True to disable this warning
warnings.warn(
wandb: (1) Create a W&B account
wandb: (2) Use an existing W&B account
wandb: (3) Don't visualize my results
wandb: Enter your choice: 3
wandb: You chose "Don't visualize my results"
wandb: Tracking run with wandb version 0.15.2
wandb: W&B syncing is set to offline in this directory.
wandb: Run wandb online or set WANDB_MODE=online to enable cloud syncing.
0%| | 0/11166 [00:00<?, ?it/s]You're using a LlamaTokenizerFast tokenizer. Please note that with a fast tokenizer, using the __call__ method is faster than using a method to encode the text followed by a call to the pad method to get a padded encoding.
/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/tokenization_utils_base.py:2382: UserWarning: max_length is ignored when padding=True and there is no truncation strategy. To pad to max length, use padding='max_length'.
warnings.warn(
Traceback (most recent call last):
File "/home/ub2004/llm_dev/llama-trl/training_reward_model.py", line 307, in
trainer.train(script_args.resume_from_checkpoint)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/trainer.py", line 1664, in train
return inner_training_loop(
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/trainer.py", line 1940, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/trainer.py", line 2735, in training_step
loss = self.compute_loss(model, inputs)
File "/home/ub2004/llm_dev/llama-trl/training_reward_model.py", line 198, in compute_loss
rewards_j = model(input_ids=inputs["input_ids_j"], attention_mask=inputs["attention_mask_j"])[0]
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/peft/peft_model.py", line 566, in forward
return self.base_model(
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 821, in forward
transformer_outputs = self.model(
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/transformers/models/llama/modeling_llama.py", line 531, in forward
inputs_embeds = self.embed_tokens(input_ids)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1194, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 160, in forward
return F.embedding(
File "/home/ub2004/anaconda3/envs/gh_llama-trl/lib/python3.10/site-packages/torch/nn/functional.py", line 2210, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
wandb: Waiting for W&B process to finish... (failed 1).
wandb: You can sync this run to the cloud by running:
wandb: wandb sync /home/ub2004/llm_dev/llama-trl/wandb/offline-run-20230612_091914-tynnd9g5
wandb: Find logs at: ./wandb/offline-run-20230612_091914-tynnd9g5/logs
(gh_llama-trl) ub2004@ub2004-B85M-A0:~/llm_dev/llama-trl$