I Fine-Tuned Microsoft Phi-2 to Read Teacher Evaluations. It Started at 34% Accuracy.

The first time I ran the raw model on a batch of student comments about their teachers, it called 93% of them “neutral.” A kid wrote “Mr. Harrison made me actually look forward to calculus, and I hate math” — neutral. Another wrote “Ms. Rivera spent the whole semester reading off slides she didn’t make and never answered a single question” — also neutral.

Turns out a 2.7-billion-parameter language model, fresh off its general-purpose training, has no idea what teacher feedback sounds like. It defaults to the middle of the road every time. I needed to teach it.

The goal looked simple: take Microsoft Phi-2 — one of the strongest small language models you can run on a single GPU — and fine-tune it (train it further on a specific dataset) to classify student evaluations as positive, neutral, or negative. The data formatting alone took a weekend.

The numbers first

After fine-tuning:

• Overall accuracy: 34.9% → 87.2%
• Positive sentiment accuracy: 7.3% → 97.0%
• Negative sentiment accuracy: 11.0% → 84.7%
• Training loss: dropped 38% (from 1.41 to 0.87)

The positive jump surprised me. The base model was guessing — 7.3% is worse than random across three categories. After fine-tuning, it caught 97% of the genuinely positive comments. Negative sentiment was harder because student evaluations contain real sarcasm, and the model still misses some of it. I will take 84.7% over 11%.

The setup

The run used ordinary Python packages and a standard pip install:

pip install accelerate peft einops datasets bitsandbytes trl transformers datasets

The data: short comments were a bad training signal

The dataset was a tab-separated file of teacher evaluations — student comments paired with sentiment labels. I loaded it with pandas and immediately hit my first wrong turn: short comments.

A comment like “Good teacher” tells you exactly nothing useful. It is positive, but there is no signal for the model to learn from: no useful word patterns, structure, or context. I filtered out anything under 200 characters. That cleaned up the training data more than any later hyperparameter change.

import pandas as pd
from sklearn.model_selection import train_test_split

# Load data
filename = "./data/teacher_performance/ReadyToTrain_data_2col_with_subjectivity_final.tsv"
training_data_df = pd.read_csv(
    filename,
    sep='\t',
    encoding="utf-8",
    encoding_errors="replace"
)

# Filter comments longer than 200 characters
training_data_df = training_data_df[
    training_data_df['StudentComments'].str.len() > 200
]

The splits also had to stay balanced. Splitting the data without stratifying by sentiment makes the model see far more of one class and skew hard toward it. I split each sentiment category separately with scikit-learn’s train_test_split and a fixed random seed so the result is reproducible:

# Split data for each sentiment
X_train = []
X_test = []

for sentiment in ['positive', 'neutral', 'negative']:
    train, test = train_test_split(
        training_data_df[training_data_df['Sentiment'] == sentiment],
        random_state=42
    )
    X_train.append(train)
    X_test.append(test)

Fitting Phi-2 on one GPU

Phi-2 has 2.7 billion parameters. At full 16-bit precision (the standard floating-point format most models ship in), that’s about 5.4 GB just for the weights — before you add memory for activations, gradients, and optimizer states during training. A consumer GPU with 24 GB of VRAM would buckle under a full fine-tune.

I used 4-bit quantization: each weight goes from 16 bits to 4 bits in Normal Float 4 (NF4), a format designed for the bell-curve distribution most neural-network weights follow. That cuts VRAM usage by roughly 60% with almost no quality loss. bitsandbytes configures it through Hugging Face’s BitsAndBytesConfig:

from transformers import (
    AutoModelForCausalLM,
    AutoTokenizer,
    BitsAndBytesConfig
)

# Quantization configuration
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_use_double_quant=False,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16
)

# Load model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-2",
    trust_remote_code=True,
    device_map="auto",
    quantization_config=bnb_config
)

device_map="auto" tells the loader to spread layers across available GPUs if you have more than one. On a single card it just puts everything there. trust_remote_code=True is required for Phi-2 because it uses a custom model architecture that hasn’t been merged into the main Transformers library yet.

Training without melting the GPU

Even with 4-bit quantization, updating every weight in a 2.7B-parameter model during training is expensive — both in memory and in risk of the model forgetting what it already knew (catastrophic forgetting, where new training overwrites general knowledge the model needs).

LoRA — Low-Rank Adaptation — handles both problems. It freezes the original weights and adds pairs of small trainable matrices to selected attention layers. Only those matrices change. At the end, they merge back into the base weights, and the model runs at full speed with zero overhead.

I targeted four projection layers inside attention: q_proj, k_proj, v_proj (the query, key, and value projections that let the model decide which parts of the input to pay attention to) and dense (the output projection). The rank r=16 controls the adapter dimensions. A higher rank gives more capacity and more parameters; 16 is a solid default for a task like this.

from peft import LoraConfig

lora_config = LoraConfig(
    r=16,
    lora_alpha=16,
    target_modules=[
        "q_proj",
        "k_proj",
        "v_proj",
        "dense"
    ],
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM"
)

I ran 50 epochs (full passes through the training data) because the learning rate was low and the adapter matrices were small — they need more passes to converge. A batch size of 4 with gradient accumulation across 8 steps made the effective batch 32 samples before each weight update. fp16=True kept the forward and backward passes in half-precision to save memory.

training_arguments = TrainingArguments(
    output_dir="./runs/Sentiment-Analysis-Phi2-fine-tuned",
    num_train_epochs=50,
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    learning_rate=2e-4,
    weight_decay=0.001,
    fp16=True
)

Running the training

Hugging Face’s SFTTrainer (Supervised Fine-Tuning Trainer) puts the model, dataset, LoRA config, and tokenizer into one training loop. .train() handles the forward passes, loss calculation, backpropagation, optimizer steps, and checkpointing:

sft_trainer = SFTTrainer(
    model=base_model,
    train_dataset=train_data,
    eval_dataset=eval_data,
    peft_config=lora_config,
    tokenizer=tokenizer,
    args=training_arguments
)

sft_trainer.train()

The training loss started at 1.41 and reached 0.87 after 50 epochs, a 38% reduction. Loss measures how far predictions are from the correct answer; lower is better. The curve fell without spiking back up, which meant the model was learning without overfitting.

from peft import AutoPeftModelForCausalLM
import torch

new_model = AutoPeftModelForCausalLM.from_pretrained(
    adapter_dir,                  # directory SFTTrainer wrote to
    torch_dtype=torch.bfloat16,
    device_map={"": 0},
)
merged = new_model.merge_and_unload()
merged.save_pretrained("merged_model", safe_serialization=True)
tokenizer.save_pretrained("merged_model")

Where It Still Stumbles

84.7% on negative sentiment is good but not great. The remaining 15% are mostly borderline cases — students being politely critical, sarcastic, or mixing a compliment with a complaint in the same sentence. A human reader catches the tone shift; the model sometimes registers only the positive words and misses the knife twist at the end.

The next experiments would be a higher LoRA rank (r=32 or r=64) for more capacity, a few hundred hand-labeled misclassifications for a second pass, and the subjectivity score column from the original dataset. I suspect highly subjective comments are the cases the model misses.

The full training notebook is on GitHub. Questions or ideas for the next iteration: [email protected].