Dataset format

[rajivpoddar]

Is there a particular dataset format required for finetuning codellama? I have the dataset in the OpenAI suggested format which is basically a jsonl with each entry having messages: [{role: 'system', content: '<system prompt>'}, {role: 'user', content: '<user prompt>'}, {role: 'assistant', content: '<assistant reply>'}]} object. Will this format work?

[mzbac]

You have to map the dataset in the formatting_prompts_func yourself.

[chartgod]

def formatting_prompts_func(example):
output_texts = []
for i in range(len(example['prompt'])):
text = f"An AI tool that corrects and rephrase user text grammar errors delimited by triple backticks to standard English.\n### Input: {example['prompt'][i]}\n ### Output: {example['completion'][i]}"
output_texts.append(text)
return output_texts

To create a learning format from this source code, [INST] < <>{text} [/INST]

text = """
A: , B: ""

Can I use it like this?