After training with DPOTrainer of trl, and saving, loading error when using AutoPeftModelForCausalLM

[MS-YUN]

After training with DPOTrainer of trl, and saving, loading error when using AutoPeftModelForCausalLM

After training with DPOTrainer of trl, I saved it locally as below and then loaded it with AutoPeftModelForCausalLM, and an error came out. When I load a checkpoint stored locally with SFTTranier in the same way, I don't get an error. I put token (no problem) and something else. but I keep getting the same error. I ask for the help. Thank you.

dpo_trainer.model.save_pretrained("./model/dpo_results/final_checkpoint")
tokenizer.save_pretrained("./model/dpo_results/final_checkpoint")

('./model/dpo_results\final_checkpoint\tokenizer_config.json', './model/dpo_results\final_checkpoint\special_tokens_map.json', './model/dpo_results\final_checkpoint\vocab.json', './model/dpo_results\final_checkpoint\merges.txt', './model/dpo_results\final_checkpoint\added_tokens.json', './model/dpo_results\final_checkpoint\tokenizer.json')

os.listdir("./model/dpo_results/final_checkpoint")

['README.md', 'adapter_model.bin', 'adapter_config.json', 'tokenizer_config.json', 'special_tokens_map.json', 'vocab.json', 'merges.txt', 'tokenizer.json']

import torch
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer, AutoModelForCausalLM

DEV = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
adapter_path = "./model/dpo_results/final_checkpoint"
model = AutoPeftModelForCausalLM.from_pretrained(
adapter_path,
torch_dtype=torch.bfloat16,
load_in_4bit=True,
token = HUGGINGFACEHUB_API_TOKEN,
)
tokenizer = AutoTokenizer.from_pretrained(adapter_path)

HTTPError Traceback (most recent call last)
File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_errors.py:261, in hf_raise_for_status(response, endpoint_name)
260 try:
--> 261 response.raise_for_status()
262 except HTTPError as e:

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\requests\models.py:1021, in Response.raise_for_status(self)
1020 if http_error_msg:
-> 1021 raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/None/resolve/main/config.json

The above exception was the direct cause of the following exception:

RepositoryNotFoundError Traceback (most recent call last)
File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\transformers\utils\hub.py:428, in cached_file(path_or_repo_id, filename, cache_dir, force_download, resume_download, proxies, token, revision, local_files_only, subfolder, repo_type, user_agent, _raise_exceptions_for_missing_entries, _raise_exceptions_for_connection_errors, _commit_hash, **deprecated_kwargs)
426 try:
427 # Load from URL or cache if already cached
--> 428 resolved_file = hf_hub_download(
429 path_or_repo_id,
430 filename,
431 subfolder=None if len(subfolder) == 0 else subfolder,
432 repo_type=repo_type,
433 revision=revision,
434 cache_dir=cache_dir,
435 user_agent=user_agent,
436 force_download=force_download,
437 proxies=proxies,
438 resume_download=resume_download,
439 token=token,
440 local_files_only=local_files_only,
441 )
442 except GatedRepoError as e:

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_validators.py:118, in validate_hf_hub_args.._inner_fn(*args, **kwargs)
116 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs)
--> 118 return fn(*args, **kwargs)

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\file_download.py:1195, in hf_hub_download(repo_id, filename, subfolder, repo_type, revision, library_name, library_version, cache_dir, local_dir, local_dir_use_symlinks, user_agent, force_download, force_filename, proxies, etag_timeout, resume_download, token, local_files_only, legacy_cache_layout)
1194 try:
-> 1195 metadata = get_hf_file_metadata(
1196 url=url,
1197 token=token,
1198 proxies=proxies,
1199 timeout=etag_timeout,
1200 )
1201 except EntryNotFoundError as http_error:
1202 # Cache the non-existence of the file and raise

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_validators.py:118, in validate_hf_hub_args.._inner_fn(*args, **kwargs)
116 kwargs = smoothly_deprecate_use_auth_token(fn_name=fn.name, has_token=has_token, kwargs=kwargs)
--> 118 return fn(*args, **kwargs)

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\file_download.py:1541, in get_hf_file_metadata(url, token, proxies, timeout)
1532 r = _request_wrapper(
1533 method="HEAD",
1534 url=url,
(...)
1539 timeout=timeout,
1540 )
-> 1541 hf_raise_for_status(r)
1543 # Return

File c:\Users\nl202\miniconda3\envs\py31006th\lib\site-packages\huggingface_hub\utils_errors.py:293, in hf_raise_for_status(response, endpoint_name)
285 message = (
286 f"{response.status_code} Client Error."
287 + "\n\n"
(...)
291 " make sure you are authenticated."
292 )
--> 293 raise RepositoryNotFoundError(message, response) from e
295 elif response.status_code == 400:

RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-6508609b-66056fe260c0c09f5c752227;a56272fe-4e1b-43fc-b435-e0b47d9222f0)

Repository Not Found for url: https://huggingface.co/None/resolve/main/config.json.
Please make sure you specified the correct repo_id and repo_type.
If you are trying to access a private or gated repo, make sure you are authenticated.

The above exception was the direct cause of the following exception:

OSError Traceback (most recent call last)
e:\gdstrm\내 드라이브\Colab Notebooks\dl\dl_nlp\RLHF_3_DPO\2_미디엄블로그\work_미디엄블로그.ipynb Cell 84 line 7
5 DEV = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
6 adapter_path = "./model/dpo_results/final_checkpoint"
----> 7 model = AutoPeftModelForCausalLM.from_pretrained(
8 adapter_path,
9 torch_dtype=torch.bfloat16,
10 load_in_4bit=True,
11 token = HUGGINGFACEHUB_API_TOKEN,
12 )
13 tokenizer = AutoTokenizer.from_pretrained(adapter_path)

.............

OSError: None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo either by logging in with huggingface-cli login or by passing token=<your_token>

Please help me if you know how to resolve the error Thank you.

※ PS, I used "facebook/opt-350m" as base_model. and I followed this guide faithfully (https://github.com/mzbac/llama2-fine-tune)`

[mzbac]

The code looks correct to me. Maybe double-check your adapter_config.json to see if it has the correct base_model_name_or_path.