1. Home
  2. Blog
  3. Artificial Intelligence

Artificial Intelligence

Naive Bayes Classifier Explained: Build a Spam Filter in Python

A naive Bayes classifier predicts a label by combining how common each class is with how likely each word or feature is within that class. Learn Bayes rule, the naive independence assumption, Laplace smoothing, why we add log probabilities, and Gaussian naive Bayes, by building a small spam filter from scratch and running it.

9 min read
Naive Bayes classifier explained: build a spam filter in Python

Key takeaways

  • A naive Bayes classifier picks the class with the highest prior probability times the likelihood of the observed features.
  • “Naive” means it assumes features are independent given the class, which is rarely true but often works well.
  • Without smoothing, one unseen word made both class scores exactly 0 in our test; Laplace smoothing fixes it.
  • Multiplying many small probabilities underflowed to 0.0; summing logarithms gave a usable -1842.07 instead.

Before deep learning, the first widely successful email spam filters were built on a surprisingly simple idea: count how often words appear in spam and in normal mail, then use probability to score new messages. That idea is the naive Bayes classifier. It trains in a single pass, works with tiny datasets, explains its decisions and remains a strong baseline for text classification.

We built one from scratch in Python 3.13.6, then a Gaussian version with NumPy 2.5.5; every number below comes from those runs. The ten training messages are made-up examples, so treat the probabilities as illustrations, not benchmarks. For a different classic algorithm, compare k nearest neighbors.

What is a naive Bayes classifier?

Naive Bayes is a family of supervised learning algorithms that apply Bayes’ theorem to classification. Given features (words in an email, measurements of a flower) it estimates the probability of each class and predicts the most probable one. Training only requires counting, so it is extremely fast, and the model is a small table of probabilities you can inspect.

Bayes rule in one line

P(class | features) = P(features | class) x P(class) / P(features)
  • P(class) is the prior: how common the class is overall.
  • P(features | class) is the likelihood: how typical these features are for that class.
  • P(features) is the same for every class, so for choosing the winner we can ignore it and compare likelihood x prior.

The naive independence assumption

Estimating the probability of an entire email word sequence is impossible with small data. The naive assumption says: given the class, each word is independent of the others. Then the likelihood becomes a simple product:

P(w1, w2, ..., wn | spam) = P(w1 | spam) x P(w2 | spam) x ... x P(wn | spam)

Words are obviously not independent (“free” and “prize” go together), so the probabilities the model outputs are often too extreme. But the ranking of classes is frequently still right, which is all a classifier needs.

Our training data

spam: "win a free prize now", "free money claim now",
      "limited offer win cash", "claim your free gift card"
ham:  "meeting moved to monday", "can you review my pull request",
      "lunch on monday?", "the build is green now",
      "please review the test plan", "call me after the meeting"

Four spam and six normal (“ham”) messages. Lowercased word tokens gave a vocabulary of 35 words: 18 word occurrences in spam and 28 in ham.

Training: priors and word counts

priors: {'spam': 0.4, 'ham': 0.6}
vocab size: 35   words per class: {'spam': 18, 'ham': 28}

The priors are just class frequencies: 4 of 10 messages are spam. The likelihoods come from counting each word per class. That is the entire training step, a single pass over the data, which is why naive Bayes can train on millions of documents in seconds.

Laplace smoothing: handling unseen words

P(free | spam) = (3 + 1) / (18 + 35) = 0.0755
P(free | ham)  = (0 + 1) / (28 + 35) = 0.0159

“free” appeared 3 times in spam and never in ham. Without adjustment, P(free | ham) would be 0, and any message containing “free” could never be classed as ham, no matter what else it said. Laplace (add-one) smoothing adds 1 to every count and the vocabulary size to the denominator, so no word ever has zero probability. We tested the unsmoothed version on “free review”:

no smoothing 'free review': {'spam': 0.0, 'ham': 0.0}

Both scores collapsed to exactly zero, because “review” never appeared in spam and “free” never appeared in ham. The model could not decide at all. The smoothing parameter, alpha, does not have to be 1; smaller values are common and can be tuned.

Why we add logarithms instead of multiplying

0.01 ** 400        # 0.0  (underflow)
400 * log(0.01)    # -1842.0680743952364

A long email multiplies hundreds of small probabilities, and the product quickly becomes smaller than the tiniest number a float can hold, so it rounds to 0. Taking logs turns the product into a sum of manageable negative numbers, and the class with the largest log score is still the winner. Our classifier works entirely in log space and converts back to probabilities only at the end, using the same subtract-the-maximum trick as the softmax function.

def logp(text, c, alpha=1.0):
    s = math.log(prior[c])
    for w in tok(text):
        s += math.log((counts[c][w] + alpha) / (total[c] + alpha * len(vocab)))
    return s

Predictions on new messages

'free prize claim'          -> spam  {'spam': 0.964, 'ham': 0.036}
'review the meeting notes'  -> ham   {'spam': 0.036, 'ham': 0.964}
'win monday lunch'          -> ham   {'spam': 0.359, 'ham': 0.641}
'free review'               -> spam  {'spam': 0.557, 'ham': 0.443}

Reading the results

  • “free prize claim” uses three spam-only words, so it is confidently spam.
  • “review the meeting notes” uses ham words; “notes” was never seen, but smoothing let the other words decide.
  • “win monday lunch” mixes a spam word with two ham words and leans ham, helped by the higher ham prior.
  • “free review” is a near tie, 0.557 to 0.443: one strong spam word against one ham word. With more training data, such borderline cases become rarer, and in production you would usually require a high spam probability before filtering.

Because the model is just word probabilities, you can explain every decision by listing which words pushed it towards each class, something deep learning models struggle to offer. Better text features, such as TF-IDF weights or word pairs, usually improve accuracy further.

Tip: the probabilities from naive Bayes are often overconfident. Use them to rank, and choose your decision threshold on validation data.

Gaussian naive Bayes for numeric features

Words are counts, but many features are measurements. Gaussian naive Bayes assumes each feature follows a normal distribution within each class, and learns its mean and standard deviation. We generated 400 synthetic points in two classes with two numeric features each, trained on 300 and tested on 100:

mu = [Xtr[ytr == c].mean(0) for c in (0, 1)]
sd = [Xtr[ytr == c].std(0) for c in (0, 1)]
def g(x, m, s):
    return (-0.5 * ((x - m) / s) ** 2 - np.log(s * np.sqrt(2 * np.pi))).sum(1)

# Gaussian NB test accuracy: 0.98

98 of 100 test points were classified correctly. The whole model is four means, four standard deviations and two priors. Notice that the score is again a sum of log densities, one per feature, which is the naive assumption at work.

Naive Bayes variants

Variant Features Typical use
Multinomial Counts, such as word frequencies Text classification, spam, topics
Bernoulli Binary present or absent Short texts, yes or no features
Gaussian Continuous measurements Sensor data, simple numeric datasets
Complement Counts, imbalanced classes Text with rare classes

Our spam filter was multinomial. Libraries such as scikit-learn provide all four; we did not run scikit-learn for this article.

Strengths and weaknesses

Strengths Weaknesses
Trains in one pass, very fast Independence assumption is rarely true
Works with small datasets Probabilities are often overconfident
Handles many classes and features Cannot learn feature interactions
Easy to explain and update Usually beaten by modern models on large data

For text, modern systems often use embeddings and large language models, but naive Bayes remains a valuable first baseline: if a complex model cannot beat it, something is wrong.

Final thoughts

The naive Bayes classifier turns counting into prediction: priors from class frequencies, likelihoods from feature counts, Bayes rule to combine them. Add Laplace smoothing so unseen words do not zero out a class, work in log space to avoid underflow, and choose the variant that matches your features. Our small spam filter showed each of those pieces, from confident predictions to a borderline tie, in a few dozen lines of Python.

Frequently asked questions

What is a naive Bayes classifier?

It is a probabilistic classifier that applies Bayes rule and assumes features are independent given the class. It predicts the class with the highest prior times likelihood.

Why is naive Bayes called naive?

Because it assumes every feature is independent of the others once the class is known. That assumption is usually false, but the classifier often still ranks classes correctly.

What is Laplace smoothing in naive Bayes?

It adds a small count, usually 1, to every feature in every class so that a word never seen in a class does not get zero probability and wipe out that class score.

Why use log probabilities in naive Bayes?

Multiplying many small probabilities underflows to zero in floating point. Adding their logarithms keeps the numbers in a safe range and gives the same winning class.

When should I use Gaussian naive Bayes?

Use it when features are continuous measurements that are roughly normally distributed within each class. Use multinomial naive Bayes for word counts and Bernoulli for binary features.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Random Forest in Python: Bagging, Feature Sampling and OOB From Scratch

Random forest in Python from scratch: bootstrap sampling, feature sampling, number of trees, out-of-bag evaluation and impurity vs permutation importance.

Read article →
Software Development

MIME Types Explained: Content-Type, nosniff and a Common Types List

MIME types explained and tested in a browser: Content-Type, nosniff, scripts and stylesheets with wrong types, a common MIME types…

Read article →
DevOps

Git Clean Explained: Remove Untracked Files Without Regret

Git clean explained with dry runs: -n, -f, -d, -x, -X, exclusions, paths, nested repositories and interactive mode, plus a…

Read article →
Test Automation

Pytest approx: Comparing Floats With rel, abs and Collections

Pytest approx explained: default tolerances, the near-zero trap, rel and abs, lists and dicts, NaN, NumPy, and how it differs…

Read article →
Software Testing

Accessibility Testing: WCAG Checklist, Automated Tools and Manual Checks

Accessibility testing explained: WCAG basics, an axe-core scan of a broken and a fixed page, what automation missed, contrast checks…

Read article →
Databases & SQL

SQL UNPIVOT: Turn Columns Into Rows, 4 Ways Tested

SQL UNPIVOT explained: turn columns into rows with UNION ALL, CROSS JOIN with CASE, JSON functions or native UNPIVOT, with…

Read article →
Software Development

Intl NumberFormat in JavaScript: Currency, Compact, Percent and Units

Intl.NumberFormat in JavaScript: locales, currency, compact notation, percent, units, rounding, formatToParts, the non-breaking space trap and formatter reuse.

Read article →
Software Development

Python Abstract Class: ABC, abstractmethod and When to Use Them

Python abstract class explained: ABC and abstractmethod, abstract properties, shared logic with super, register, collections.abc and ABC vs Protocol, tested.

Read article →
Software Development

Python Subprocess Explained: run, Capture Output, Timeouts and Safety

Python subprocess explained with tested examples: run, capture_output, check, timeouts, input, environment, shell injection, encoding on Windows and Popen streaming.

Read article →
Databases & SQL

SQL EXPLAIN Plan: How to Read Query Plans and Fix Slow Queries

SQL EXPLAIN plan explained: read scans, index searches, covering indexes and temporary sorts, with before and after timings for five…

Read article →
Artificial Intelligence

Decision Tree in Python: Gini, Entropy and Overfitting From Scratch

Decision tree in Python from scratch: Gini impurity, entropy, information gain, best splits, readable rules, and how depth and leaf…

Read article →
Software Development

URL Query Parameters: Encoding, URLSearchParams and Common Traps

URL query parameters explained: reading and building them with URLSearchParams and urllib, percent encoding, plus signs, repeated keys and nine…

Read article →
DevOps

Git Blame Explained: Find Who Changed a Line, and Why

Git blame explained with a real repository: ignore whitespace, line ranges and functions, ignore-revs files for formatting commits, moved code…

Read article →
Test Automation

Pytest Benchmark: Measure Speed and Catch Performance Regressions

Pytest benchmark tutorial: the benchmark fixture, reading results, parametrized and pedantic benchmarks, saving baselines and failing the build on regressions.

Read article →
Software Testing

Defect Density: Formula, Worked Examples and How to Use It

Defect density explained: the formula, a five-module worked example, per KLOC vs per function point, severity weighting, defect removal efficiency…

Read article →
Databases & SQL

Case Insensitive Search in SQL: LOWER, LIKE, COLLATE and Indexes

Case insensitive search in SQL: LOWER, LIKE, COLLATE NOCASE and expression indexes compared with query plans, plus accented letters and…

Read article →
Databases & SQL

ON DELETE CASCADE in SQL: CASCADE vs SET NULL vs RESTRICT

ON DELETE CASCADE in SQL tested: CASCADE vs SET NULL vs RESTRICT, multi-level cascades, the SQLite foreign key pragma, row…

Read article →
Software Development

JavaScript BigInt Explained: Big Numbers, Division and JSON

JavaScript BigInt explained: the 2^53 limit, creating and converting BigInt, division, mixing types, JSON stringify and parse without losing digits,…

Read article →
Artificial Intelligence

Logistic Regression in Python: Sigmoid, Log Loss and Gradient Descent From Scratch

Logistic regression in Python from scratch: sigmoid, log loss, gradient descent, odds ratios, thresholds, sigmoid overflow and separable data, tested…

Read article →
DevOps

Git Worktree Explained: Work on Two Branches at Once

Git worktree explained: add, list, remove, prune, lock and detached worktrees, a hotfix workflow, and the errors you will hit,…

Read article →
Test Automation

Pytest ini Explained: addopts, markers, testpaths and Precedence

Pytest ini explained: testpaths, addopts, strict markers, xfail_strict, filterwarnings, minversion, config file precedence and command-line overrides, tested.

Read article →
Software Testing

Test Estimation Techniques: WBS, Three Point PERT and Monte Carlo

Test estimation techniques worked through: WBS, three point PERT, total uncertainty, a Monte Carlo check, percentage rules and Wideband Delphi.

Read article →
Databases & SQL

Gaps and Islands in SQL: Find Streaks and Missing Ranges

Gaps and islands in SQL: find login streaks and missing ranges with ROW_NUMBER, DENSE_RANK, LAG and LEAD, plus the duplicate…

Read article →
Databases & SQL

INSERT INTO SELECT in SQL: Copy Rows Between Tables Safely

INSERT INTO SELECT in SQL: copy rows between tables with chosen columns, joins and summaries, avoid duplicates on reruns, upsert,…

Read article →
Artificial Intelligence

Linear Regression in Python: Least Squares From Scratch, With R Squared

Linear regression in Python from scratch: closed form, NumPy least squares and gradient descent, R squared, RMSE, outliers and curved…

Read article →
DevOps

Git Remote Explained: Add, Set-URL, Rename, Remove and Push URLs

Git remote explained: add, set-url, separate push URLs, rename, remove and prune, plus how to avoid leaking tokens in remote…

Read article →
Test Automation

Pytest Mock: mocker.patch, spy, side_effect and autospec Explained

Pytest mock explained: mocker.patch, return_value, side_effect, spy, autospec and PropertyMock, with tested examples for APIs, clocks and retries.

Read article →
Software Testing

System Testing Explained: Types, Process and a Real End-to-End Example

System testing explained: types, entry and exit criteria, the process, and a real end-to-end test suite run against a small…

Read article →
Artificial Intelligence

Naive Bayes Classifier Explained: Build a Spam Filter in Python

Naive Bayes classifier explained by building a spam filter in Python: Bayes rule, the naive assumption, Laplace smoothing, log probabilities…

Read article →
DevOps

Git Branch Explained: Create, Switch, Rename, Delete and Recover

Git branch explained with tested commands: list, create, switch, rename, delete, merged checks, recovering deleted branches, remote tracking and naming.

Read article →
Test Automation

Pytest raises: Testing Exceptions, match and excinfo

Pytest raises explained with tested examples: match regexes, excinfo, custom and chained exceptions, parametrized error cases, pytest.warns and common mistakes.

Read article →
Software Testing

Non Functional Testing: Types, Metrics and a Real Load Test

Non functional testing explained: 10 key types, measurable requirements, latency percentiles and what a small real load test revealed about…

Read article →
Artificial Intelligence

K Means Clustering Explained: How It Works, With Python Code

K means clustering explained: the algorithm, a from-scratch NumPy version, choosing k with the elbow and silhouette methods, k-means++, scaling…

Read article →
DevOps

Git Log Explained: 15 Useful Commands and Pretty Formats

Git log explained with 15 tested commands: oneline, graph, filters by author, date, message and code, file history, stats, pretty…

Read article →
Test Automation

Pytest conftest.py Explained: Shared Fixtures, Hooks and Overrides

Pytest conftest.py explained with a tested suite: shared fixtures, scope and teardown, folder overrides, autouse, custom options, markers and useful…

Read article →
Software Testing

Usability Testing: Methods, Metrics, SUS Score and Checklist

Usability testing explained: methods, planning, writing tasks, running sessions, metrics, SUS scoring, how many users you need and a practical…

Read article →