How to incorporate additional data in fine tuning LLM

In LLM training, it's essential to define the input and output clearly.

With the Alpaca format, the input prompt is the combination of "instruction" and "input", and the output is the "output" field.

For the passionMan/diabetes_instruct_v7 dataset, the input remains the same, but you can choose different fields for the output:

• Use "output" if you're building a simple Q&A chatbot.
• Use "rationale" + "explanation" if you want the model to show its reasoning.

You can use the Hugging Face datasets library to convert passionMan/diabetes_instruct_v7 into Alpaca-style JSONL format for fine-tuning.

from datasets import load_dataset
import json


dataset = load_dataset("passionMan/diabetes_instruct_v7", split="train")

alpaca_data = []
for item in dataset:
    output = f"{item['rationale'].strip()} {item['explanation'].strip()}"
    alpaca_data.append({
        "instruction": item["instruction"],
        "input": item["input"],
        "output": output
    })

with open("diabetes_alpaca.json", "w", encoding="utf-8") as f:
    for entry in alpaca_data:
        f.write(json.dumps(entry, ensure_ascii=False) + "\n")