1. Home
  2. Blog
  3. Artificial Intelligence

Artificial Intelligence

K Nearest Neighbors (KNN) Explained: How It Works, With Python Code

K nearest neighbors classifies a new point by looking at the k most similar examples it has already seen and taking a vote. Learn the algorithm step by step, code it from scratch in NumPy, see how the choice of k and feature scaling change accuracy, and where KNN powers modern vector search.

9 min read
K nearest neighbors explained: how KNN works, with Python code

Key takeaways

  • K nearest neighbors stores the training data and predicts by majority vote (or average) of the k closest examples.
  • Our from-scratch NumPy version scored 1.000 test accuracy with k=1 and k=3 and 0.978 with k=5 to 45, but fell to 0.222 when k equalled the whole training set.
  • Adding one unscaled noisy feature dropped accuracy from 1.000 (standardized) to 0.444. Always scale features.
  • Brute-force KNN cost about 19 ms per query on 100,000 points; large systems use approximate nearest neighbour indexes.

Some machine learning algorithms learn weights through hours of training. K nearest neighbors learns nothing up front: it remembers the examples and, when asked about a new point, finds the most similar ones and lets them vote. That simplicity makes KNN the perfect first algorithm to understand, a strong baseline for small datasets, and the idea behind the vector search used in modern AI retrieval.

Every number in this article comes from code we ran with Python 3.13.6 and NumPy 2.5.5. The dataset is synthetic, generated with a fixed seed so you can reproduce it. We did not run scikit-learn for this article.

What is the k nearest neighbors algorithm?

KNN is a supervised learning algorithm for classification and regression. To label a new point, it measures the distance to every training point, picks the k closest, and returns the most common label among them (classification) or their average value (regression). Because it does all its work at prediction time, it is called a lazy or instance-based learner.

The algorithm in four steps

  1. Choose k, the number of neighbours to consult.
  2. Compute the distance from the new point to every training point.
  3. Sort the distances and keep the k smallest.
  4. Vote: return the most common label, or the mean value for regression.

There is no training step beyond storing the data, which is both KNN’s charm and, for large datasets, its main cost.

Our dataset: limes, lemons and oranges

We generated 180 fruits, 60 of each kind, with two features: weight in grams and diameter in centimetres. Limes average about 60 g and 4.5 cm, lemons 110 g and 6.0 cm, oranges 180 g and 7.5 cm, with some natural spread so the groups overlap a little. After shuffling, 135 fruits became the training set and 45 the test set.

KNN from scratch in NumPy

import numpy as np, collections

def predict(Xtr, ytr, x, k):
    d = np.sqrt(((Xtr - x) ** 2).sum(axis=1))      # Euclidean distance to every point
    nn = np.argsort(d)[:k]                          # indexes of the k nearest
    label = collections.Counter(ytr[nn]).most_common(1)[0][0]
    return label, nn, d[nn]

def accuracy(Xtr, ytr, Xte, yte, k):
    return np.mean([predict(Xtr, ytr, x, k)[0] == t for x, t in zip(Xte, yte)])

That is the whole algorithm: one line of vectorised distance maths, one sort and one vote. NumPy computes all 135 distances at once without a Python loop.

Looking at the neighbours

q = np.array([120.0, 6.1])          # a 120 g fruit, 6.1 cm across
label, nn, dist = predict(Xtr, ytr, q, k=5)
# query [120.   6.1] -> lemon
#    neighbour [120.   5.7] lemon  dist 0.37
#    neighbour [122.8  6.4] lemon  dist 2.77
#    neighbour [123.5  7. ] orange dist 3.65
#    neighbour [124.5  5.4] lemon  dist 4.56
#    neighbour [114.8  6. ] lemon  dist 5.21

Four lemons and one orange, so the vote says lemon. Seeing the neighbours is one of KNN’s best features: the prediction comes with its own explanation. Notice something odd, though: the distances are almost entirely about weight. A 3 g difference counts far more than a 1.3 cm difference in diameter, simply because grams are bigger numbers. We come back to that in the scaling section.

Choosing k

k Test accuracy
1 1.000
3 1.000
5 0.978
7 0.978
15 0.978
45 0.978
135 (all points) 0.222

On a test set of 45, one mistake is a drop of 0.022, so k from 1 to 45 performed almost identically on this easy, well-separated data. The extreme case is the lesson: with k equal to the whole training set, every prediction is simply the most common class overall, and accuracy collapsed to 0.222. In general:

  • Small k follows the data closely and is sensitive to noise and mislabelled points (overfitting).
  • Large k smooths the decision boundary and eventually ignores local structure (underfitting).
  • Pick k with cross-validation on your data. An odd k avoids ties in two-class problems.

Why feature scaling matters

To show the scaling problem clearly, we added a third, useless feature in large units: random values around 5,000 with a spread of 1,500, like a price in paise that has nothing to do with the type of fruit.

Features (k=5) Test accuracy
Weight and diameter, raw 0.978
Weight, diameter and noisy feature, raw 0.444
Same three features, standardized 1.000
Weight and diameter, standardized 0.978

With raw values, the noise feature’s large numbers dominated every distance, so the “nearest” neighbours were just fruits with similar random prices, and accuracy fell to 0.444, barely better than guessing among three classes. Standardizing each feature (subtract the training mean, divide by the training standard deviation) put all three on the same scale and restored accuracy. The small difference between 1.000 and 0.978 is a single test fruit and not meaningful on a 45-item test set. Always compute the scaling statistics on the training data only, then apply them to test data, to avoid leaking information.

Tip: KNN has no way to learn that a feature is irrelevant. Scale features, and drop or down-weight ones that do not help.

Distance metrics

Metric Formula idea Good for
Euclidean Straight-line distance Continuous, scaled features
Manhattan Sum of absolute differences High dimensions, grid-like data
Cosine Angle between vectors Text and embeddings, where direction matters more than length
Hamming / Jaccard Count or share of differing items Binary features and sets

For text embeddings, cosine similarity is the usual choice; see our cosine similarity guide. For sets of tags, Jaccard similarity fits better.

Ties and distance weighting

collections.Counter([0, 1]).most_common(1)   # [(0, 1)]

With k=2 and one vote each, most_common simply returned the label it saw first, an arbitrary tie-break. Use an odd k for two classes, or break ties by the nearest neighbour. A popular refinement is distance weighting, where each vote counts as 1 divided by the distance, so very close neighbours outweigh distant ones. That also reduces sensitivity to the exact choice of k.

KNN for regression

def knn_reg(x, k=5):
    nn = np.argsort(np.abs(w[:, 0] - x))[:k]
    return dia[nn].mean()

knn_reg(150)   # 7.5 cm predicted diameter for a 150 g fruit

For numeric targets, KNN averages the neighbours’ values instead of voting. Here we predicted diameter from weight alone. KNN regression produces step-like predictions and cannot extrapolate beyond the range of the training data, so in our run a 400 g fruit was predicted at just 7.74 cm, the average diameter of the five heaviest oranges, rather than anything larger.

How fast is KNN?

Brute-force KNN compares each query with every stored point. We timed 100 queries against 100,000 random points with 16 features, using np.argpartition to find the 5 nearest without a full sort: about 19 ms per query. That is fine for thousands of predictions on a small dataset, but it grows linearly with the data. Tree structures such as KD-trees and ball trees help in low dimensions; for millions of high-dimensional vectors, systems use approximate nearest neighbour (ANN) indexes such as HNSW, trading a tiny amount of accuracy for huge speed-ups. We did not benchmark those here.

KNN, embeddings and vector search

Modern AI applications use KNN constantly, just under another name. A retrieval-augmented generation system turns documents into embeddings, stores them in a vector database, and for each question finds the k nearest document vectors by cosine similarity. That is KNN search on embeddings, usually with an ANN index. Read more in what are embeddings, our vector database guide and what is RAG.

Pros and cons of KNN

Pros Cons
Simple to understand and implement Slow predictions on large datasets
No training time Stores the entire dataset in memory
Explains predictions by showing neighbours Very sensitive to feature scale and irrelevant features
Handles many classes naturally Struggles in very high dimensions without good embeddings
Adapts instantly when new data is added Needs a sensible distance metric

Final thoughts

K nearest neighbors is machine learning stripped to its essence: similar things tend to have similar labels. Store the examples, measure distance, let the k closest vote. Our from-scratch version reached perfect or near-perfect accuracy on clean data, collapsed when k was absurdly large, and was wrecked by a single unscaled feature until we standardized it. Scale your features, choose k by validation, and use approximate indexes at scale. For turning scores into probabilities, see the softmax function.

Frequently asked questions

How does the k nearest neighbors algorithm work?

It stores the training data. To predict a new point, it measures the distance to every stored point, takes the k closest, and returns their most common label for classification or their average for regression.

How do I choose the value of k in KNN?

Try several values with cross-validation and pick the one with the best validation score. Small k is sensitive to noise, large k blurs class boundaries, and an odd k avoids ties with two classes.

Why do I need to scale features for KNN?

KNN uses distances, so features with large numeric ranges dominate. In our test an unscaled noisy feature cut accuracy to 0.444, and standardizing restored it.

Is KNN supervised or unsupervised?

KNN classification and regression are supervised because they need labelled examples. The related nearest neighbour search used in vector databases needs no labels and simply finds similar items.

Is KNN good for large datasets?

Plain brute-force KNN gets slow because every prediction compares against all stored points. Large systems use KD-trees, ball trees or approximate nearest neighbour indexes such as HNSW.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

K Means Clustering Explained: How It Works, With Python Code

K means clustering explained: the algorithm, a from-scratch NumPy version, choosing k with the elbow and silhouette methods, k-means++, scaling…

Read article →
Software Development

API Versioning: URL vs Header vs Media Type, With Deprecation

API versioning explained: breaking changes, URL vs header vs media type vs query versioning, default versions, deprecation and sunset headers,…

Read article →
DevOps

Git Log Explained: 15 Useful Commands and Pretty Formats

Git log explained with 15 tested commands: oneline, graph, filters by author, date, message and code, file history, stats, pretty…

Read article →
Test Automation

Pytest conftest.py Explained: Shared Fixtures, Hooks and Overrides

Pytest conftest.py explained with a tested suite: shared fixtures, scope and teardown, folder overrides, autouse, custom options, markers and useful…

Read article →
Software Testing

Usability Testing: Methods, Metrics, SUS Score and Checklist

Usability testing explained: methods, planning, writing tasks, running sessions, metrics, SUS scoring, how many users you need and a practical…

Read article →
Databases & SQL

SQL UPDATE Statement: Safe Updates, UPDATE With JOIN and RETURNING

The SQL UPDATE statement explained with tested examples: WHERE guards, CASE updates, UPDATE with JOIN, the subquery NULL trap, RETURNING…

Read article →
Databases & SQL

SQL COUNT: COUNT(*) vs COUNT(column) vs COUNT(DISTINCT) Explained

SQL COUNT explained: COUNT(*) vs COUNT(column) vs COUNT(DISTINCT), NULLs and empty strings, conditional counts, HAVING, the LEFT JOIN trap and…

Read article →
Software Development

JavaScript Generators Explained: yield, next() and 10 Examples

JavaScript generators explained with 10 tested examples: yield, next, infinite and lazy sequences, yield*, return and throw, iterables and async…

Read article →
Software Development

Python Recursion Explained: 9 Examples, Memoization and the Recursion Limit

Python recursion explained with 9 tested examples: base cases, a traced call stack, memoization with lru_cache, the recursion limit, trees…

Read article →
Software Development

Python CSV: Read and Write CSV Files (With 7 Gotchas Fixed)

Read and write CSV files with the Python csv module: reader, writer, DictReader, DictWriter, quoting, delimiters, encodings and 7 real…

Read article →
Artificial Intelligence

K Nearest Neighbors (KNN) Explained: How It Works, With Python Code

K nearest neighbors explained: the KNN algorithm, a from-scratch NumPy version, choosing k, feature scaling, distance metrics, regression and vector…

Read article →
Software Development

HTTP Cookies Explained: Secure, HttpOnly, SameSite and Scope

HTTP cookies explained: Set-Cookie, Path, Domain, expiry, Secure, HttpOnly, SameSite, prefixes, deleting cookies and a secure session recipe, tested locally.

Read article →
DevOps

Git Rebase Interactive: Squash, Fixup, Reword and Autosquash

Git rebase interactive explained step by step: pick, reword, squash, fixup, edit, drop, autosquash, conflicts and undoing a rebase with…

Read article →
Software Testing

Alpha vs Beta Testing: Key Differences, Exit Criteria and Checklist

Alpha vs beta testing compared: who tests, what each phase finds, entry and exit criteria, how to run a beta,…

Read article →
Test Automation

Pytest Monkeypatch: 10 Tested Examples for Env Vars, Time and APIs

Pytest monkeypatch explained with 10 passing tests: environment variables, functions, config, time, sleeps, paths, blocking the network and the wrong-target…

Read article →
Databases & SQL

SQL Auto Increment: MySQL, PostgreSQL, SQL Server and SQLite

SQL auto increment in MySQL, PostgreSQL, SQL Server and SQLite: syntax, reading the new ID, why gaps appear, ID reuse,…

Read article →
Databases & SQL

SQL BETWEEN: Inclusive Ranges and the Date Trap Explained

SQL BETWEEN explained with tested queries: inclusive ranges, reversed bounds, NULLs, the last-day date trap, half-open ranges and index-friendly date…

Read article →
Software Development

JavaScript Template Literals: 10 Examples Including Tagged Templates

JavaScript template literals explained with 10 tested examples: interpolation, multiline strings, escaping, tagged templates for safe HTML and SQL, and…

Read article →
Artificial Intelligence

Softmax Function Explained: Formula, Python Code and Stable Version

The softmax function explained: formula, worked example, NumPy code, the stable version that avoids NaN, temperature, cross-entropy and softmax vs…

Read article →
DevOps

Git Diff Explained: Staged Changes, Branches and Commits

Git diff explained: read the output, compare unstaged and staged changes, commits and branches, two dots vs three dots, and…

Read article →
Test Automation

Pytest Xdist: Run Tests in Parallel and Cut Test Time

Run pytest tests in parallel with pytest-xdist: install, the -n option, distribution modes, real timings and how to fix shared-state…

Read article →
Software Testing

Verification vs Validation in Software Testing: Example and Table

Verification vs validation in software testing explained with a comparison table and a tested coupon example where every spec test…

Read article →
Databases & SQL

SQL CAST Explained: Convert Data Types With Tested Examples

SQL CAST explained with tested examples: text to number, truncation, integer division, numbers stored as text, dates, bad input and…

Read article →
Databases & SQL

SQL DISTINCT: How to Remove Duplicate Rows (and Its Traps)

SQL DISTINCT explained with tested queries: one and many columns, NULLs, COUNT DISTINCT, DISTINCT vs GROUP BY, the SUM DISTINCT…

Read article →
Artificial Intelligence

Levenshtein Distance: Formula, Worked Table and Python Code

Levenshtein distance explained: the edit operations, the dynamic programming table for kitten and sitting, Python code with two-row memory, normalised…

Read article →
DevOps

Gitignore: Patterns, Negation and Fixing Tracked Files

A tested gitignore guide: pattern rules, leading and trailing slashes, negation with !, the ignored parent folder trap, git check-ignore…

Read article →
Software Testing

Acceptance Criteria: Given When Then Examples and Checklist

How to write acceptance criteria testers can run: rule-based and Given When Then formats, a worked discount code example turned…

Read article →
Software Testing

State Transition Testing: Diagram, Table and Example

State transition testing explained with a login lockout example: states, events, the diagram, the state table, valid and invalid transitions,…

Read article →
Test Automation

Python Unittest: Assert Methods, setUp and Mocks With Examples

A practical Python unittest tutorial: TestCase, assert methods, setUp and tearDown, subTest, skips, expected failures and mock, with a test…

Read article →
Artificial Intelligence

Jaccard Similarity: Formula, Python, MinHash

Jaccard similarity measures overlap between two sets as intersection over union. Work the formula, compute it in Python for words…

Read article →
DevOps

Git Hooks: Pre-commit and Commit-msg Examples

Git hooks run scripts at key moments such as commit and push. See tested pre-commit, commit-msg and prepare-commit-msg hooks, sharing…

Read article →
Test Automation

API Testing in Python: Pytest + Requests Suite

Build an API testing suite in Python with pytest and requests: fixtures, status codes, headers, JSON schema validation, CRUD flows,…

Read article →
Software Testing

Test Design Techniques: 9 Methods + Examples

Test design techniques help you write fewer tests that catch more bugs. Learn 9 black-box, white-box and experience-based methods with…

Read article →
Artificial Intelligence

TF-IDF Explained: Formula and Worked Example

TF-IDF scores a word by how often it appears in a document and how rare it is across all documents….

Read article →
DevOps

Git Tag: Create, Push and Delete Release Tags

Git tag explained with tested commands: annotated vs lightweight tags, listing and sorting, pushing with follow-tags, deleting local and remote…

Read article →
Test Automation

Pytest Coverage: pytest-cov and Branch Gaps

Measure pytest coverage with pytest-cov: term-missing reports, branch coverage, HTML reports, pyproject settings and a fail-under gate for CI, with…

Read article →