Confusion Matrix, Precision and Recall Explained with a Threshold
Learn TP, FP, FN and TN, precision, recall, accuracy and F1 by moving a classification threshold on real model scores, then solve an exam-style numerical.
Lesson summary
A threshold makes the decisions; the confusion matrix counts them; precision and recall judge them.
- A threshold turns a probability into a yes or no.
- Every prediction lands in one of TP, FP, FN or TN.
- Precision = TP/(TP+FP). Recall = TP/(TP+FN).
- Raising the threshold raises precision and lowers recall.
- Pick the cutoff from the cost of each mistake.
The model returns .76 and the cutoff is .80. What does the code do?
No refund. .76 is below .80, so the code says no even though the model leans yes.
The code refunded a customer who did not ask. Which cell is that?
False positive. The code said yes (positive), and it was wrong (false).
TP = 5, FP = 2, FN = 1. What is recall?
5/(5+1) = 5/6 ≈ .83.
You raise the cutoff. Which usually goes up: precision or recall?
Precision. Fewer, surer refunds means fewer false positives but more missed customers.
Stage 1 of 7
Why did the store refund the wrong customer?
Priya asks whether her refund from last month went through. The store’s code reads her message and sends her another ₹1,200. She never asked for a second refund. Was the model wrong, or was the code?
Here are the words you need. A threshold, or cutoff, turns a probability into yes or no. A confusion matrix counts four kinds of right and wrong answers. Precision and recall are two scores built from those counts.
Our example is a real kind of model: TypeSafe’s Jev reads each message and answers one question with a probability. The store, messages and numbers are invented for practice. Let us start with the step between a probability and a decision.
Stage 2 of 7
How does a probability become a yes or no?
Remember Priya’s message. The store asks Jev one question about it: does this message ask for a refund? Jev does not answer yes or no. It answers .76.
That number is the probability that the answer is yes. Near 1 means a strong yes and near 0 a strong no. For this kind of yes or no question, called a Noul, the number is the whole answer, with no separate confidence score.
Code turns the number into an action with one comparison: refund when p is at least 0.5. Since .76 is at least .5, Priya gets a refund. The model never saw that 0.5. The store chose it.
Today ten messages arrived. Rank them by p and draw the cutoff line at .50. Seven sit on or above the line and get a refund. Three sit below and get none.
Drag the Cutoff slider. The line moves and the number of refunds changes, but not one probability changes. A threshold changes the decisions, never the model’s answers.
Every classifier works like this: a spam filter, a disease test, a fraud check. A score goes in, and a cutoff turns it into yes or no. Next question: which of those seven refunds were right?
Stage 3 of 7
What are TP, FP, FN and TN?
We have seven refunds at cutoff .50, but were they right? A support agent read all ten messages and marked who really asked for a refund. A tick means asked and a cross means did not.
Messages 1, 2, 3, 5 and 7 got a refund and really asked. These are true positives, or TP. Positive means the code said yes. True means that was right. TP = 5.
Messages 4 and 6 got a refund but never asked. Message 4 is Priya’s. These are false positives: the code said yes and was wrong. FP = 2, and each one is ₹1,200 sent by mistake.
Message 8 really asked but scored .38, below the line, so no refund. That is a false negative: the code said no and was wrong. FN = 1, one customer left waiting.
Messages 9 and 10 did not ask and got nothing. Those are true negatives, TN = 2. The four cells always add up to the total: 5 + 2 + 1 + 2 = 10.
An exam trick for the names: read the second word first. Positive or negative is what the code predicted. True or false says whether that prediction was right. Next, we turn these four counts into two scores.
Stage 4 of 7
What is the difference between precision and recall?
Here is the confusion matrix we just filled: TP 5, FP 2, FN 1, TN 2. The store wants two answers from it. How trustworthy are our refunds, and how many real requests did we catch?
Precision looks down the refund column. The code sent 7 refunds and 5 of them were real. Precision = TP/(TP+FP) = 5/7 = .71. When the code says yes, it is right 71% of the time.
Recall looks across the asked row. Six customers really asked, and 5 of them got a refund. Recall = TP/(TP+FN) = 5/6 = .83. We caught 83% of the real requests.
Both scores put TP on top. Only the bottom changes. Precision uses the Predicted column, so it adds FP. Recall uses the Real row, so it adds FN. P for predicted, R for real.
Precision divides by the column, recall divides by the row.
Exams also ask two more. Accuracy = (TP+TN)/total = 7/10 = .70. F1 combines precision and recall: 2TP/(2TP+FP+FN) = 10/13 = .77, the same as 2PR/(P+R).
All of these came from one cutoff, .50. The model’s probabilities stay fixed, so the only thing that can change these scores is where the line sits. Next, we move it.
Stage 5 of 7
Why does raising the threshold lower recall?
At cutoff .50 we had precision .71 and recall .83. In money terms: two wrong refunds and one customer who asked and got nothing. Can a different cutoff fix both?
Raise the line to .80. Only messages 1, 2 and 3 get refunds, and all three really asked. Precision jumps to 1.0. But recall falls to 3/6 = .50: three real requests now sit below the line.
Lower the line to .30. Now all six customers who asked get their refund, so recall is 1.0. But messages 4 and 6 are refunded again, and precision drops to 6/8 = .75.
Each mistake has a cost. A false positive sends ₹1,200 that staff must recover. A false negative leaves a customer who asked and heard nothing. Raising the cutoff trades the first kind of mistake for the second.
Drag the Cutoff slider slowly from .10 to .90. Watch precision climb while recall falls, and watch the rupees lost shrink while the waiting customers grow.
So the store picks the cutoff from what each mistake costs it. TypeSafe’s docs give the same advice: raise the cutoff when a false yes is expensive, such as a refund. Pick it separately for each question, and test it before you rely on it. Now it is your turn.
Stage 6 of 7
Can you find the cutoff and prove it?
Here is tomorrow’s batch, messages 11 to 20, already marked by the support agent. The store’s rule: precision at least .80, so no more than one in five refunds is a mistake, and recall at least .70.
Set the cutoff with minus and plus, then press Sort. Then prove it the way an exam would: pick the right fraction for precision, then for recall. A wrong formula or a missed target loses.
Stage 7 of 7
Can you solve it on paper?
We started with a probability. Jev said .76, and a cutoff the store chose turned it into a refund. The model scores; the code decides.
Then the support agent’s ticks let us sort every decision into four cells: TP, FP, FN and TN. Priya’s wrong refund was a false positive.
From those counts came precision, 5/7, down the refund column, and recall, 5/6, across the asked row. Same top, different bottom.
Moving the cutoff traded one for the other. Higher cutoff, higher precision, lower recall. The right cutoff depends on which mistake costs more.
Now an exam question in a new setting. A spam filter scores eight emails, and the cutoff is 0.5. Find TP, FP, FN and TN, then precision and recall. Work it out on paper before you go on.
Emails 1, 2, 3 and 4 are flagged. Three are spam, so TP = 3 and FP = 1. Emails 5 and 8 are spam but missed, so FN = 2, and TN = 2. Precision = 3/4 = .75 and recall = 3/5 = .60. Same method, any classifier.