Non-existent target modules used for PEFT in the provided SFT Colab notebook .
#3
by
Veb-BLK
- opened
from peft import LoraConfig, get_peft_model
target_modules = [
"q_proj", "v_proj", "fc1", "fc2", "linear",
"gate_proj", "up_proj", "down_proj",
]
peft_config = LoraConfig(
lora_alpha=16,
lora_dropout=0.05,
r=8,
bias="none",
target_modules=target_modules,
task_type="CAUSAL_LM",
)
model = get_peft_model(model, peft_config)
print(model)
I can't find the keys related to "linear", "gate_proj", "up_proj", "down_proj".
I'm new to fine-tuning. What should I use instead?