How to Tokenize this Model With My Custom Dataset
#2
by
Nobleson
- opened
I am a newbiew in LLM and i found this model to perfectly matched my use-case, all i need is to finetune it.
The first thing i have done is, checking the dataset that has been used to train this model, build my custom dataset on it.
Where i have problem is the way the dataset is struction: Instruction , Intent, Tag and Response, unlike other examples i have come accross on the internet with just two fields: text and label.
Please can one share a script that will help me preprocessed such dataset that is used to train this model?
My goal is to train finetune retail chatbot model for my business need.
I appreciate your response.