Lab 3: Gemma4 Finetuning

Difficulty: Advanced · Estimated time: ~5 hours

Objective

1. Prepare a custom instruction dataset
2. Finetune Gemma4 with QLoRA using Unsloth on Google Colab
3. Push the adapter to HuggingFace Hub
4. Create a HuggingFace Spaces demo
5. Evaluate with LM Eval Harness

Step 1 — Prepare dataset

python

from datasets import Dataset
import json

# Load your instruction dataset
with open("instructions.json") as f:
    data = json.load(f)

dataset = Dataset.from_list(data)
dataset = dataset.train_test_split(test_size=0.1)
dataset.push_to_hub("your-username/tds-instructions")

Format: {"instruction": "...", "input": "...", "output": "..."}

Step 2 — Finetune with Unsloth

Run on Google Colab (T4 GPU):

python

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="google/gemma-2b",
    max_seq_length=2048,
    load_in_4bit=True,
)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    lora_alpha=16,
    lora_dropout=0,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)

from trl import SFTTrainer
from transformers import TrainingArguments

trainer = SFTTrainer(
    model=model,
    tokenizer=tokenizer,
    train_dataset=dataset["train"],
    args=TrainingArguments(
        per_device_train_batch_size=2,
        gradient_accumulation_steps=4,
        warmup_steps=10,
        max_steps=100,
        learning_rate=2e-4,
        output_dir="outputs",
    ),
)
trainer.train()

Step 3 — Push to HuggingFace Hub

python

model.push_to_hub_gguf("your-username/gemma-tds-finetuned", tokenizer)

Submission

GitHub repo + HuggingFace Hub link with: finetuning notebook, adapter weights, Spaces demo URL

Grading rubric

Criterion	Points
Dataset properly formatted and uploaded	20
QLoRA finetuning completes successfully	25
Adapter pushed to HuggingFace Hub	15
Spaces demo responds to queries	20
LM Eval Harness results documented	20
Total	100