1. Home
  2. Blog
  3. Software Development

Software Development

Python Collections Module Explained: Counter, defaultdict, deque and More, Tested

The Python collections module gives you ready-made containers for counting, grouping, queues and lightweight records. Learn when to use Counter, defaultdict, deque, namedtuple, OrderedDict and ChainMap, with output we ran on Python 3.13.

9 min read
Python collections module explained: Counter, defaultdict, deque and more

Key takeaways

  • Python collections is a standard library module: no install, just from collections import ....
  • Counter counts things, defaultdict groups things, and deque is the right tool for queues.
  • Draining 100,000 items from the front took about 11 seconds with list.pop(0) and 3 milliseconds with deque.popleft() in our run.
  • namedtuple gives readable, immutable records; ChainMap layers settings without copying them.

Most Python code is built on lists and dictionaries, and that works until you write the same counting loop for the fifth time or watch a queue slow to a crawl. The Python collections module exists for exactly those moments. It ships with Python and adds a handful of specialised containers that are faster, shorter and clearer than the hand-written versions.

Every example below was run on Python 3.13.6 on a Windows laptop, and the outputs are copied from that run. Timings will differ on your machine, but the size of the gaps should not. If you need a refresher on the basics first, read our Python dictionary guide and our comparison of list vs tuple vs set.

What is the collections module?

The collections module is part of the Python standard library. It provides container data types that extend the built-in dict, list and tuple. The ones you will use most are:

  • Counter: a dictionary that counts hashable items.
  • defaultdict: a dictionary that creates a default value for missing keys.
  • deque: a double-ended queue with fast appends and pops at both ends.
  • namedtuple: a tuple whose fields have names.
  • OrderedDict: a dictionary with extra ordering tools.
  • ChainMap: a view that searches several dictionaries in order.

Because Counter, defaultdict and OrderedDict are dictionary subclasses, anything that accepts a dict also accepts them. That makes them easy to drop into existing code.

Counter: count anything in one line

Counting words, status codes or test results is a classic job. Instead of a loop with if key in d, pass any iterable to Counter:

from collections import Counter

words = "the cat sat on the mat the cat ran".split()
c = Counter(words)
print(c)
print(c.most_common(2))
print(c["dog"])
print(c.total())
Counter({'the': 3, 'cat': 2, 'sat': 1, 'on': 1, 'mat': 1, 'ran': 1})
[('the', 3), ('cat', 2)]
0
9

Two details are worth noticing. Looking up a missing key returns 0 instead of raising KeyError, and most_common(n) gives you a sorted top-N list without any extra code. The total() method, added in Python 3.10, returns the sum of all counts.

Counters also support arithmetic, which is handy when you compare two inventories or two test runs:

a = Counter(apples=3, pears=1)
b = Counter(apples=1, kiwis=2)
print(a + b)   # Counter({'apples': 4, 'kiwis': 2, 'pears': 1})
print(a - b)   # Counter({'apples': 2, 'pears': 1})
print(a & b)   # Counter({'apples': 1})
print(a | b)   # Counter({'apples': 3, 'kiwis': 2, 'pears': 1})

Subtraction drops any count that falls to zero or below, so a - b has no kiwis. Intersection keeps the minimum of each count and union keeps the maximum. A practical use: count failing test names across two nightly runs and subtract to see which failures are new.

defaultdict: group without checking keys

Grouping records by a key normally needs a check for whether the key exists. A defaultdict takes a factory function, such as list or int, and calls it the first time a missing key is used:

from collections import defaultdict

orders = [("asha", "books"), ("ravi", "pens"), ("asha", "pens"), ("li", "books")]
g = defaultdict(list)
for cust, item in orders:
    g[cust].append(item)
print(dict(g))
{'asha': ['books', 'pens'], 'ravi': ['pens'], 'li': ['books']}

You can get the same result with dict.setdefault(cust, []).append(item), and in our run the two dictionaries compared equal. The defaultdict version reads better, and it does not build a throwaway empty list on every call the way setdefault does.

Tip: a plain lookup on a defaultdict creates the key. In our test, reading dd["x"] on an empty defaultdict(int) returned 0 and left {'x': 0} behind. Use "x" in dd or dd.get("x") when you only want to check.

deque: a real queue with both ends open

A deque (pronounced “deck”) supports fast append, appendleft, pop and popleft. It also has two features lists lack: a maximum length and rotation.

from collections import deque

last3 = deque(maxlen=3)
for n in range(1, 7):
    last3.append(n)
print(last3)          # deque([4, 5, 6], maxlen=3)

r = deque([1, 2, 3, 4, 5])
r.rotate(2)
print(r)              # deque([4, 5, 1, 2, 3])

With maxlen set, adding to a full deque silently drops an item from the other end. That makes it perfect for a “last N log lines” buffer, a moving average or recent search history. rotate is useful for round-robin scheduling, for example cycling through test workers.

list vs deque: the timing that matters

Lists are fast at the end but slow at the front, because removing the first item shifts every other item left. We drained a container from the front with each approach and timed it with timeit:

Items list.pop(0) deque.popleft()
20,000 0.408 s 0.0006 s
50,000 2.566 s 0.0014 s
100,000 11.362 s 0.0030 s

Doubling the items roughly quadrupled the list time, which is the signature of quadratic work, while the deque time grew in a straight line. At 100,000 items the deque was several thousand times faster. If your code processes jobs, messages or breadth-first search nodes in arrival order, use a deque.

The trade-off: deques are slower than lists for random access in the middle, such as d[50000], and they do not support slicing. Use a list when you index often and a deque when you push and pop at the ends.

namedtuple: readable records

A tuple like (3, 4) works, but p[0] says nothing about what the value means. namedtuple builds a small class with named fields:

from collections import namedtuple

Point = namedtuple("Point", "x y")
p = Point(3, 4)
print(p, p.x, p._asdict(), p._replace(x=10))
p.x = 5
Point(x=3, y=4) 3 {'x': 3, 'y': 4} Point(x=10, y=4)
AttributeError: can't set attribute

Named tuples are immutable, use as little memory as plain tuples, and still unpack like tuples. _replace returns a new copy with one field changed. If you need mutable fields, defaults with validation or methods, a dataclass is usually the better fit; see our Python dataclasses guide. The typed alternative typing.NamedTuple gives the same object with type hints.

OrderedDict: still useful after Python 3.7

Since Python 3.7, normal dictionaries keep insertion order, so many people assume OrderedDict is obsolete. It is not, for two reasons we confirmed:

from collections import OrderedDict

od = OrderedDict(a=1, b=2, c=3)
od.move_to_end("a")
print(list(od))                                         # ['b', 'c', 'a']
print(dict(a=1, b=2) == dict(b=2, a=1))                 # True
print(OrderedDict(a=1, b=2) == OrderedDict(b=2, a=1))   # False

First, move_to_end and popitem(last=False) make it a simple base for a least-recently-used cache. Second, equality between two OrderedDicts is order-sensitive, while plain dicts ignore order. That matters when the order itself is part of the data, such as column order in a report.

ChainMap: layered settings

Configuration often comes from several places: command-line flags override environment variables, which override defaults. ChainMap searches a list of dictionaries in order without merging them:

from collections import ChainMap

defaults = {"theme": "light", "lang": "en", "debug": False}
env = {"debug": True}
cli = {"lang": "fr"}
cfg = ChainMap(cli, env, defaults)
print(cfg["theme"], cfg["lang"], cfg["debug"])   # light fr True

Each lookup checks cli first, then env, then defaults. Because nothing is copied, a change to any of the source dictionaries shows up immediately. Writes go only to the first mapping, which keeps your defaults safe. This pairs well with the patterns in our environment variables guide.

Python collections cheat sheet

Need Use Instead of
Count items, top N Counter Manual dict loop and sort
Group values by key defaultdict(list) if key not in d checks
Queue or sliding window deque, maxlen list.pop(0)
Small immutable record namedtuple Bare tuple with index access
Reorder or order-aware equality OrderedDict Rebuilding a dict
Layered config lookups ChainMap Copying and merging dicts

The module also contains UserDict, UserList and UserString, which are wrapper classes meant for subclassing when you want to customise behaviour safely. Most everyday code never needs them.

Common mistakes with collections

  • Creating keys by accident. Reading a missing key from a defaultdict inserts it. Later code that loops over the keys then sees empty entries.
  • Expecting negative counts. Counter subtraction with - drops zero and negative results. Use c.subtract(other) if you need to keep negatives.
  • Using a deque for random access. Indexing near the middle is slower than with a list, and slicing raises an error.
  • Treating namedtuple as mutable. Assignment fails. Use _replace or switch to a dataclass.
  • Forgetting maxlen drops data. A full bounded deque discards items silently, which is great for buffers but wrong for job queues that must not lose work.

When the data comes from a stream you process lazily, combine these containers with generators and the tools in our Python itertools guide. For example, Counter accepts a generator directly, so you can count lines in a large log without loading it into memory.

Final thoughts

The Python collections module is small, but it removes a surprising amount of boilerplate. Reach for Counter when you count, defaultdict when you group, deque whenever items leave from the front, and namedtuple or ChainMap when readability matters. Our timing test showed the stakes clearly: the right container turned an 11-second loop into a few milliseconds.

Frequently asked questions

Do I need to install the Python collections module?

No. It is part of the standard library, so you only need an import such as from collections import Counter.

What is the difference between dict and defaultdict?

A dict raises KeyError for a missing key. A defaultdict calls its factory function, such as list or int, to create a default value and stores it under that key.

When should I use deque instead of list?

Use deque when you add or remove items at the front, as in queues and sliding windows. In our test, draining 100,000 items from the front took about 11 seconds with a list and 3 milliseconds with a deque.

Is OrderedDict still needed in Python 3.7 and later?

Less often, because dicts keep insertion order. It is still useful for move_to_end, popitem from either end and order-sensitive equality checks.

Should I use namedtuple or a dataclass?

Use namedtuple for small, immutable records that should behave like tuples. Use a dataclass when you need mutable fields, defaults, methods or validation.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
Software Development

HTTP Methods Explained: GET, POST, PUT, PATCH, DELETE, Tested on a Live Server

HTTP methods explained with a small server we ran: GET, POST, PUT, PATCH, DELETE, HEAD and OPTIONS, safe vs idempotent,…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Software Development

JavaScript Sort Array: Numbers, Objects, Strings and Why 10 Comes Before 5

How to sort an array in JavaScript correctly: numbers with a compare function, objects by one or more fields, strings…

Read article →
Software Development

Python String Methods: Cheat Sheet, Real Examples and the strip() Trap

Python string methods with tested examples: split, join, strip, replace, find, startswith, is-methods, casefold, partition and translate, plus the strip()…

Read article →
Software Development

Python zip() Explained: Loop Two Lists at Once, Unzip, strict=True and Traps

Python zip() pairs items from several lists in one loop. Tested examples for dicts, unzipping with *, strict=True, zip_longest, sliding…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
Software Development

UTF-8 Encoding Explained: How Text Becomes Bytes and How to Fix Garbled Text

UTF-8 encoding explained with byte-level examples we ran in Python: how characters become 1 to 4 bytes, why café turns…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Databases & SQL

Delete Duplicate Rows in SQL: 5 Safe Methods, Tested Step by Step

How to delete duplicate rows in SQL safely: find them, preview them, then remove them with ROW_NUMBER, MIN(id), EXISTS, rowid…

Read article →
Databases & SQL

SQL RANK vs DENSE_RANK vs ROW_NUMBER: Differences with Tested Examples

SQL RANK vs DENSE_RANK vs ROW_NUMBER in one table: how each handles ties, top-N per group, the second-highest salary trap…

Read article →
Software Development

JavaScript Optional Chaining (?.) Explained: 8 Examples, ?? Defaults and Traps

JavaScript optional chaining stops “Cannot read properties of undefined” errors. 8 tested examples for properties, arrays and methods, ?? vs…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Databases & SQL

SQL COALESCE Explained: Defaults, NULLIF, IFNULL vs ISNULL and Traps, Tested

SQL COALESCE explained with queries we ran in SQLite 3.50: first non-null values, empty strings, NULL in maths, NULLIF for…

Read article →
Databases & SQL

SQL Self Join Explained: Employee-Manager, Pairs and Row Comparisons, Tested

SQL self join explained with queries we ran in SQLite 3.50: employee and manager, earning more than the manager, unique…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →
Artificial Intelligence

Prompt Injection Explained: How AI Apps Get Tricked and How to Defend Them

Prompt injection explained for builders: direct and indirect attacks, why keyword filters fail (our test blocked 1 of 4), and…

Read article →
DevOps

DNS Records Explained: A, AAAA, CNAME, MX, TXT and TTL With Real Lookups

DNS records explained with lookups we ran: how resolution works, A and AAAA, CNAME rules, MX and null MX, TXT…

Read article →
DevOps

Feature Flags Explained: Safe Releases, Gradual Rollouts and Flag Hygiene

Feature flags explained with a rollout simulation we ran: flag types, percentage rollouts with stable hashing, kill switches, allow lists,…

Read article →
Test Automation

Test Data Generation With Python Faker: Realistic, Repeatable and Safe Data

Test data generation with Python Faker, run and measured: seeding for repeatable data, locales, unique values, domain-shaped records, edge cases…

Read article →
Software Testing

Shift-Left Testing Explained: Find Bugs Earlier Without Slowing the Team

Shift-left testing explained: what it means, why early defects are cheaper, practices from requirement reviews to CI checks, the tester's…

Read article →
Artificial Intelligence

LLM Structured Output Explained: Getting Reliable JSON From AI Models

LLM structured output explained with validation we ran in Pydantic: JSON mode and schemas, constrained decoding, the ways model output…

Read article →
Test Automation

API Contract Testing Explained: Catch Breaking Changes Before They Ship

API contract testing explained with a schema check we ran: what a contract is, consumer-driven contracts, Pact, OpenAPI, which changes…

Read article →
Software Testing

Risk-Based Testing Explained: Score Risks, Build a Matrix and Test What Matters

Risk-based testing explained step by step: identifying risks, scoring likelihood and impact, building a risk matrix, mapping risk to test…

Read article →