tuning_lm_with_rl.py does not appear to have a file named config.json

[judyhappy]

Hi Jason,

I followed the steps
Step 1 - Supervised Fine-tuning, generate "/checkpoints/supervised_llama/" including folders:

checkpoint-2000
checkpoint-3000
checkpoint-4000
final_checkpoint

Step 2 Training Reward Model, generate "/checkpoints/training_reward_model/" including folders:

llama-7b-hf_peft_gpt-4-llm_rm_0_2e-05
peft_last_checkpoint

Step 3 Tuning LM with PPO.

accelerate launch --multi_gpu --num_machines 1  --num_processes 2     tuning_lm_with_rl.py     --log_with wandb     --model_name ./checkpoints/supervised_llama/     --reward_model_name ./checkpoints/training_reward_model/     --adafactor False     --tokenizer_name ./data/model/     --save_freq 100     --output_max_length 128     --batch_size 8     --gradient_accumulation_steps 8     --batched_gen True     --ppo_epochs 4     --learning_rate 1.4e-5     --early_stopping True     --output_dir './checkpoints/tuning_llama_rl/'

But there is an Error:

CUDA SETUP: Loading binary /usr/local/lib/python3.8/dist-packages/bitsandbytes/libbitsandbytes_cuda117_nocublaslt.so...
Traceback (most recent call last):
  File "tuning_lm_with_rl.py", line 159, in <module>
    tokenizer = AutoTokenizer.from_pretrained(script_args.model_name)
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/tokenization_auto.py", line 657, in from_pretrained
    config = AutoConfig.from_pretrained(
  File "/usr/local/lib/python3.8/dist-packages/transformers/models/auto/configuration_auto.py", line 916, in from_pretrained
    config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py", line 573, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/transformers/configuration_utils.py", line 628, in _get_config_dict
    resolved_config_file = cached_file(
  File "/usr/local/lib/python3.8/dist-packages/transformers/utils/hub.py", line 380, in cached_file
    raise EnvironmentError(
OSError: ./checkpoints/supervised_llama does not appear to have a file named config.json. Checkout 'https://huggingface.co/./checkpoints/supervised_llama/None' for available files.

There is no config.json under supervised_llama or training_reward_model.

[judyhappy]

Step 1 and Step 2 don't generate any config.json. So which config.json should be used for step 3?

[43zxj]

Step 1 and Step 2 don't generate any config.json. So which config.json should be used for step 3?

Hello, have you solved this problem yet? Could you tell me which config.json should be used for step 3? Thank you!!!!