1. Home
  2. Blog
  3. Software Development

Software Development

Python Set Explained: Union, Intersection and Real Examples

A Python set stores unique, hashable items and answers “is this in here?” almost instantly. Learn how to create sets, combine them with union, intersection and difference, use frozenset, and avoid the traps, with output from real runs.

8 min read
Python set operations: union, intersection, difference and examples

Key takeaways

  • A Python set is an unordered collection of unique, hashable items. Duplicates disappear automatically.
  • Use | for union, & for intersection, - for difference and ^ for symmetric difference.
  • Membership tests are fast. In our test, checking one value 200 times took 342.93 ms in a 100,000 item list and 0.023 ms in a set.
  • {} is an empty dict, not a set. Write set() instead.

You have two lists of user IDs and need the people who appear in both. Or you have a list full of repeated tags and want each tag once. You could write nested loops, but a Python set does both jobs in one line, and does them faster. Sets are one of the most useful built-in types in Python, yet many developers only reach for them to remove duplicates.

This guide covers everything you need for daily work: creating sets, the four set operations, comparisons, adding and removing items, performance, and the mistakes that trip people up. Every output below comes from Python 3.13.6 runs on our machine.

What is a Python set?

A set is a collection with three rules:

  • Unique items. Each value appears at most once. Adding a value that is already there does nothing.
  • No order. A set has no index, so s[0] raises a TypeError. Printed order is not something you should rely on.
  • Hashable items only. Numbers, strings and tuples of those are fine. Lists and dicts are not, because they can change.

Sets are mutable: you can add and remove items after creation. If you need an unchangeable version, Python gives you frozenset, covered below. For a side by side view of the three main collection types, see our guide on list vs tuple vs set.

Creating sets

There are three common ways to build a set: a literal with curly braces, the set() constructor on any iterable, and a set comprehension.

a = {"alice", "bob", "carol", "dan"}
b = set(["carol", "dan", "erin"])
letters = set("banana")       # {'a', 'b', 'n'} in some order
empty = set()

print(type({}), type(empty))
# <class 'dict'> <class 'set'>

That last line is the first trap. Empty curly braces create an empty dictionary, because dicts came first in Python. An empty set always needs set().

The four set operations

Set operations are where sets shine. Each one has an operator form and a method form. We used these two sets of team members:

a = {"alice", "bob", "carol", "dan"}
b = {"carol", "dan", "erin"}

print(sorted(a | b))   # union
print(sorted(a & b))   # intersection
print(sorted(a - b))   # difference
print(sorted(b - a))
print(sorted(a ^ b))   # symmetric difference

Output from our run:

['alice', 'bob', 'carol', 'dan', 'erin']
['carol', 'dan']
['alice', 'bob']
['erin']
['alice', 'bob', 'erin']

We wrapped each result in sorted() only to make the output stable for this article. The table below sums up what each operation returns.

Operation Operator Method Returns
Union a | b a.union(b) Everything in either set
Intersection a & b a.intersection(b) Only items in both
Difference a - b a.difference(b) Items in a but not in b
Symmetric difference a ^ b a.symmetric_difference(b) Items in exactly one set

Operators vs methods

There is one practical difference. Operators need both sides to be sets, while methods accept any iterable. So a.union(["x", "y"]) works, but a | ["x", "y"] raises a TypeError. Methods also take several arguments at once, as in a.union(b, c, d).

In-place versions

Each operation has an in-place form that changes the left set instead of building a new one: |= or update(), &= or intersection_update(), -= or difference_update(), and ^= or symmetric_difference_update(). Use them when the set is large and you no longer need the original.

Subset, superset and disjoint checks

Sets also answer relationship questions. <= asks “is every item of the left side also in the right side?”, and >= asks the reverse. isdisjoint() is true when two sets share nothing.

print({"carol"} <= a, a.isdisjoint({"zed"}))
# True True

A real use: checking that a user holds every permission an action needs, with required <= user_permissions. It reads like the rule it enforces. The strict forms < and > also require the sets to differ.

Adding and removing items

Use add() for one item and update() for many. For removal there are two methods that look alike but behave differently on a missing item:

s = {1, 2}
s.add(3)
s.discard(9)          # missing item: silently ignored
try:
    s.remove(9)       # missing item: raises
except KeyError as e:
    print("KeyError", e)
# KeyError 9

Pick remove() when a missing item means a bug you want to hear about, and discard() when missing is fine. There is also pop(), which removes and returns an arbitrary item, and clear(), which empties the set. Because sets have no order, never use pop() expecting the “first” or “last” item.

Removing duplicates from a list

The classic use. Convert to a set and back:

nums = [3, 1, 3, 2, 1, 3]
print(set(nums))                  # {1, 2, 3}
print(list(dict.fromkeys(nums)))  # [3, 1, 2]

Notice the difference. set() removes duplicates but does not promise to keep the original order. If order matters, dict.fromkeys() keeps the first time each value appeared, because dicts keep insertion order. For lists of dicts, which are not hashable, you need a key, such as a set of IDs you have already seen.

Why set lookups are so fast

A set is built on a hash table. To check x in s, Python hashes x and jumps straight to where it would be stored. A list has to compare items one by one until it finds a match. We timed 200 lookups of the last value in a 100,000 item collection:

import timeit
data = list(range(100_000))
st = set(data)
timeit.timeit(lambda: 99_999 in data, number=200)
timeit.timeit(lambda: 99_999 in st, number=200)
list in 342.93 ms
set in 0.023 ms

That is roughly 15,000 times faster for this case. Exact numbers will vary by machine, but the shape will not: list lookup time grows with the list, while set lookup stays roughly constant on average. If you check membership inside a loop, converting the list to a set once is often the single biggest speedup available.

Tip: building the set costs time too. For one or two lookups, a list is fine. The set pays off when you check membership many times against the same data.

What can go in a set?

Only hashable objects. A hashable object has a hash value that never changes during its lifetime, which is what lets the hash table find it again. Immutable built-ins such as int, float, str, bool and tuples of hashable items qualify. Lists, dicts and sets do not:

try:
    {[1, 2]}
except TypeError as e:
    print("TypeError:", e)
# TypeError: unhashable type: 'list'

The fix is usually to convert the inner item. A list becomes a tuple, and a set becomes a frozenset. Also watch for values that compare equal: {1, 1.0, True} holds a single item, because 1 == 1.0 == True and they share the same hash.

frozenset: the immutable set

A frozenset has all the read operations of a set but no add() or remove(). Since it cannot change, it is hashable, so you can use it as a dictionary key or put it inside another set:

fs = frozenset({1, 2})
print({fs: "ok"})
# {frozenset({1, 2}): 'ok'}

This is handy for caching results keyed by a group of options where order does not matter, such as a set of selected filters. Two frozensets with the same items are equal and hash the same, whatever order you built them in.

Set comprehensions

A set comprehension looks like a list comprehension with curly braces. Duplicates collapse automatically:

print({x * x for x in range(-3, 4)})
# {0, 9, 4, 1}

Seven inputs produced four results, because -3 and 3 both square to 9, and so on. Notice the printed order is not sorted. That is normal for sets: never write code or tests that depend on it. For more on the syntax, see our list comprehension guide.

Real use cases

  • Finding new or removed items. new_ids - old_ids gives records added since the last sync, and old_ids - new_ids gives records deleted.
  • Test assertions. When order does not matter, assert set(result) == {"a", "b"} avoids flaky tests caused by ordering. Note it also hides duplicates, so use sorted() if counts matter.
  • Permission checks. required <= granted, as shown earlier.
  • Visited tracking. Graph searches and crawlers keep a seen set so they never process a node or URL twice.
  • Validating input. unknown = set(payload) - ALLOWED_FIELDS flags unexpected keys in an API request in one line.
  • Comparing word lists. Set overlap is the basis of measures such as Jaccard similarity, used for near duplicate detection.

Common mistakes

  1. Writing {} for an empty set. You get a dict. Use set().
  2. Expecting order. Sets are unordered. Sort before display or comparison of sequences.
  3. Indexing a set. s[0] fails. Convert to a list, or iterate.
  4. Calling set("hello") to make a set of one word. You get the letters {'h', 'e', 'l', 'o'}. Write {"hello"} instead.
  5. Putting lists inside. Convert them to tuples first.
  6. Using remove() where the item might be missing. Prefer discard() or check first.
  7. Losing duplicates you needed. If counts matter, use collections.Counter, which our collections guide covers.

Final thoughts

The Python set is small but powerful. It removes duplicates, answers membership questions in roughly constant time, and turns comparisons between groups into readable one-liners. Remember the four operators, use set() for an empty set, keep only hashable items inside, and switch to frozenset when you need a set as a key. Next time you write a loop inside a loop to compare two lists, try a set first.

FAQ

What is a set in Python?

A set is a built-in collection of unique, hashable items with no order. Adding a duplicate does nothing, and membership checks are fast because sets use a hash table.

How do I create an empty set in Python?

Use set(). Empty curly braces create an empty dictionary, not a set.

What is the difference between union and intersection?

Union returns every item found in either set. Intersection returns only the items found in both sets.

Does a Python set keep insertion order?

No. Sets are unordered, so never rely on the printed order. Use dict.fromkeys on a list if you need unique items in their original order.

When should I use frozenset instead of set?

Use frozenset when the collection must not change or when you need it as a dictionary key or inside another set, since only immutable sets are hashable.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Jaccard Similarity: Formula, Python, MinHash

Jaccard similarity measures overlap between two sets as intersection over union. Work the formula, compute it in Python for words…

Read article →
Software Development

Unix Timestamp Explained: Seconds vs ms

A Unix timestamp counts seconds since 1 January 1970 UTC. Learn seconds vs milliseconds, converting in Python, JavaScript and SQL,…

Read article →
DevOps

Git Hooks: Pre-commit and Commit-msg Examples

Git hooks run scripts at key moments such as commit and push. See tested pre-commit, commit-msg and prepare-commit-msg hooks, sharing…

Read article →
Test Automation

API Testing in Python: Pytest + Requests Suite

Build an API testing suite in Python with pytest and requests: fixtures, status codes, headers, JSON schema validation, CRUD flows,…

Read article →
Software Testing

Test Design Techniques: 9 Methods + Examples

Test design techniques help you write fewer tests that catch more bugs. Learn 9 black-box, white-box and experience-based methods with…

Read article →
Databases & SQL

SQL Temporary Table: Create, Use and Drop It

SQL temporary tables store interim results for one session. Learn syntax for SQL Server, PostgreSQL, MySQL, Oracle and SQLite, session…

Read article →
Databases & SQL

SQL CROSS JOIN: Every Combination + Examples

SQL CROSS JOIN returns every combination of rows from two tables. See the row math, product variants, calendar grids that…

Read article →
Software Development

JavaScript Date: Format, Parse, Timezone Bugs

JavaScript Date explained with tested Node.js code: zero-based months, the date-only UTC parsing trap, toISOString, getTimezoneOffset, Intl formatting, date math…

Read article →
Software Development

Python Requests: GET, POST, Timeouts, Retries

Python requests explained with code tested against a local API: GET with params, POST json vs data, headers and auth,…

Read article →
Software Development

Python Deepcopy vs Copy: Fix Nested Lists

Python deepcopy vs shallow copy explained with tested code: aliasing, nested lists, the [[0]*3]*3 trap, memo and cycles, custom __deepcopy__,…

Read article →
Artificial Intelligence

TF-IDF Explained: Formula and Worked Example

TF-IDF scores a word by how often it appears in a document and how rare it is across all documents….

Read article →
Software Development

Base64 Encoding Explained, With Python Code

Base64 encoding turns bytes into 64 safe text characters. See the bit-level steps, padding, URL-safe base64, the 33% size cost,…

Read article →
DevOps

Git Tag: Create, Push and Delete Release Tags

Git tag explained with tested commands: annotated vs lightweight tags, listing and sorting, pushing with follow-tags, deleting local and remote…

Read article →
Test Automation

Pytest Coverage: pytest-cov and Branch Gaps

Measure pytest coverage with pytest-cov: term-missing reports, branch coverage, HTML reports, pyproject settings and a fail-under gate for CI, with…

Read article →
Software Testing

Error Guessing in Testing: Checklist + Demo

Error guessing uses tester experience to predict where code breaks. Get a ready checklist of risky inputs, a step by…

Read article →
Databases & SQL

SQL LAG and LEAD: Compare Rows, 5 Examples

SQL LAG and LEAD read values from the previous or next row without a self join. Five tested examples: change,…

Read article →
Databases & SQL

SQL STRING_AGG: Combine Rows Into One String

SQL STRING_AGG joins values from many rows into one string per group. See syntax for PostgreSQL, SQL Server, MySQL GROUP_CONCAT,…

Read article →
Software Development

JavaScript Classes: Private Fields and Extends

JavaScript classes explained with tested Node.js code: constructors, methods, #private fields, getters, static members, extends and super, plus the errors…

Read article →
Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →