1. Home
  2. Blog
  3. Software Development

Software Development

Python Deepcopy vs Shallow Copy: Fix Shared Nested Lists for Good

You copied a list, changed the copy, and the original changed too. Learn the difference between assignment, shallow copy and Python deepcopy, why nested lists and dicts get shared, the grid trap, cycles, custom __deepcopy__ and what deepcopy costs, with tested output.

9 min read
Python deepcopy vs shallow copy explained

Key takeaways

  • b = a copies nothing. Both names point to the same object.
  • A shallow copy (copy.copy, .copy(), [:]) makes a new outer container but shares the objects inside it.
  • Python deepcopy (copy.deepcopy) copies the whole tree, so nested lists and dicts are independent.
  • deepcopy is slower. In our test it took 13.1 ms for a 2,000 row structure, against 0.0007 ms for a shallow copy.

This bug catches almost every Python developer at least once. You build a default configuration, copy it for a test or a new user, tweak the copy, and later discover the default has changed as well. Nothing is wrong with Python. The copy simply was not as deep as you thought. Understanding Python deepcopy versus a shallow copy fixes this class of bug for good.

Every result below comes from Python 3.13.6 runs on our machine. We use the identity operator is throughout, because it answers the real question: “are these the same object?”

First: names point to objects

In Python, a variable is a name attached to an object. Assignment never copies data. It attaches another name to the same object. Lists, dicts and sets are mutable, so a change made through one name is visible through every other name for that object. That is the root of all copy confusion.

Three options: alias, shallow copy, deep copy

Operation New outer object? New inner objects?
b = a No No
copy.copy(a) or a.copy() Yes No, shared
copy.deepcopy(a) Yes Yes, recursively

The demo: one config, three copies

import copy

cfg = {"name": "prod", "hosts": ["a", "b"], "limits": {"cpu": 2}}
alias   = cfg
shallow = copy.copy(cfg)
deep    = copy.deepcopy(cfg)

shallow["hosts"].append("c")   # change a nested list via the shallow copy
shallow["name"] = "staging"    # replace a top-level value via the shallow copy
deep["limits"]["cpu"] = 8      # change a nested dict via the deep copy

print(cfg)
{'name': 'prod', 'hosts': ['a', 'b', 'c'], 'limits': {'cpu': 2}}

Read that output carefully, because it holds the whole lesson:

  • 'c' leaked into the original. The shallow copy shares the hosts list, so appending through it changed the one list both dicts hold.
  • name is still 'prod'. Assigning a new value to a key replaces the reference in the shallow copy’s own outer dict, which is separate.
  • cpu is still 2. The deep copy has its own limits dict.

The identity checks confirm it:

print(alias is cfg, shallow is cfg, shallow["hosts"] is cfg["hosts"], deep["hosts"] is cfg["hosts"])
# True False True False

So the rule is: a shallow copy is safe for replacing values, but not for mutating nested objects in place. If you will mutate nested data, you need a deep copy.

Ways to make a shallow copy

All of these produce the same kind of shallow copy of a list:

nested = [[1, 2], [3, 4]]
copies = [nested[:], nested.copy(), list(nested), copy.copy(nested)]
print([c[0] is nested[0] for c in copies])
print(copy.deepcopy(nested)[0] is nested[0])
# [True, True, True, True]
# False

Every shallow method shared the first inner list. Only deepcopy produced a new one. The same applies to dicts: dict(d), d.copy(), {**d} and copy.copy(d) are all shallow. The unpacking syntax looks like a fresh object, but nested values are still shared. For flat lists of numbers or strings, shallow is all you need.

The grid trap: [[0] * 3] * 3

A close cousin of the copy bug happens when you build nested lists:

grid = [[0] * 3] * 3
grid[0][0] = 1
print(grid)
# [[1, 0, 0], [1, 0, 0], [1, 0, 0]]

grid = [[0] * 3 for _ in range(3)]
grid[0][0] = 1
print(grid)
# [[1, 0, 0], [0, 0, 0], [0, 0, 0]]

Multiplying the outer list repeats a reference to one inner list three times, so changing one row changes “all” of them. The list comprehension creates a new inner list on each pass. This is the standard way to build a 2D grid, board or matrix without NumPy.

Tuples and immutable objects

Copying immutable objects is mostly pointless, and Python knows it. copy.copy of a tuple returns the same tuple. deepcopy is smarter still: it returns the same tuple only when everything inside is immutable.

t = (1, 2, [3])
print(copy.copy(t) is t, copy.deepcopy(t) is t)
# True False

Because this tuple contains a list, deepcopy built a new tuple holding a new list. Strings, numbers and frozensets are never duplicated at all, which saves memory. A tuple is only “fully frozen” if its contents are, as our list vs tuple vs set guide explains.

Cycles and shared references

A naive recursive copy would loop forever on a structure that contains itself. deepcopy keeps a memo dictionary of objects it has already copied, so it handles cycles and preserves sharing:

a = [1]
a.append(a)               # a list that contains itself
b = copy.deepcopy(a)
print(b[1] is b, b is not a)
# True True

shared = [0]
d = {"x": shared, "y": shared}
dd = copy.deepcopy(d)
print(dd["x"] is dd["y"])
# True

The copied list points to itself, not to the original. And when two keys shared one list in the original, they still share one (new) list in the copy. deepcopy reproduces the shape of your object graph, not just its values.

Customising deepcopy for your classes

By default, deepcopy copies every attribute of your objects. Sometimes that is wrong. A database connection, file handle or socket should be shared, not cloned. Define __deepcopy__ to control it:

class Conn:
    def __init__(self):
        self.cache = {}
        self.socket = "<live socket>"

    def __deepcopy__(self, memo):
        new = Conn.__new__(Conn)
        new.cache = copy.deepcopy(self.cache, memo)  # independent data
        new.socket = self.socket                     # shared resource
        return new

c = Conn(); c.cache["k"] = [1]
c2 = copy.deepcopy(c)
print(c2.cache is c.cache, c2.socket is c.socket)
# False True

Always pass memo along when you deep copy attributes, so cycle handling keeps working. __copy__ customises shallow copies the same way. Our classes and objects guide covers these special methods in context.

What deepcopy cannot copy

Some objects represent live operating system resources and refuse to be copied:

import threading
copy.deepcopy({"lock": threading.Lock()})
# TypeError: cannot pickle '_thread.lock' object

The error mentions pickle because deepcopy falls back on the same protocol that pickling uses. Locks, open files, sockets and many C extension objects behave this way. The fix is a custom __deepcopy__ that shares or recreates the resource, as above, or keeping such resources out of the data you copy.

How slow is deepcopy?

deepcopy walks every object, checks the memo and copies recursively, so it costs real time on big structures. We timed 20 runs on a dict with 2,000 rows, each holding a list and a nested dict:

deepcopy 13.1 ms, json 5.1 ms, copy 0.0007 ms

A shallow copy only copies one outer dict, so it is almost free. Round-tripping through json.dumps and json.loads was about 2.5 times faster than deepcopy here, but only works for JSON types: tuples become lists, sets fail, and datetimes need converting. Use it only for plain data, as our Python JSON guide explains. In hot loops, the best fix is often design: avoid copying by building new objects instead of mutating shared ones.

Dataclasses and dataclasses.replace

dataclasses.replace() is a convenient way to make a modified copy, but it is shallow:

from dataclasses import dataclass, field, replace

@dataclass
class Order:
    id: int
    items: list = field(default_factory=list)

o = Order(1, ["pen"])
r = replace(o, id=2)
r.items.append("ink")
print(o)
# Order(id=1, items=['pen', 'ink'])

The new order shares the items list with the original. Use copy.deepcopy(o) when the fields hold mutable data you will change, or use tuples and frozen=True dataclasses so nothing can be mutated in place. Our dataclasses guide covers frozen classes.

Copies in tests

Shared fixtures are a common source of flaky tests. If several tests receive the same dict from a module-level constant and one of them mutates it, later tests see the change and fail depending on run order. Two fixes:

  • Return a fresh object from a pytest fixture with function scope, so each test builds its own.
  • Deep copy a shared template at the start of the fixture: return copy.deepcopy(BASE_CONFIG).

Mutable default arguments, such as def f(items=[]), cause the same kind of leak between calls. Use None as the default and create the list inside the function.

Which one should you use?

  1. Flat data of numbers and strings: shallow copy.
  2. Nested data you will mutate: deepcopy.
  3. Nested data you will only read or replace at the top level: shallow copy is fine.
  4. Large or hot-path data: avoid copying, or use immutable structures.
  5. Objects holding resources: define __deepcopy__.

Debugging tip: when an object changes unexpectedly, print id(x) or test a is b for the suspect parts. Matching identities tell you immediately which objects are shared.

Final thoughts

Most copy bugs come from one misunderstanding: assuming a copy is deep when it is shallow. Remember that assignment copies nothing, that .copy(), slicing and copy.copy share nested objects, and that Python deepcopy copies the whole tree while respecting cycles and shared references. Reach for deepcopy when you will mutate nested data, accept its cost, and customise it for objects that hold resources.

FAQ

What is the difference between copy and deepcopy in Python?

copy makes a new outer object but shares the objects inside it. deepcopy also copies every nested object, so changes to the copy never affect the original.

Why did changing my copied list change the original?

Either you assigned the list instead of copying it, or you made a shallow copy and changed a nested list that both copies share. Use copy.deepcopy for nested data.

Is list slicing a deep copy?

No. Slicing with an empty range creates a shallow copy. The new list holds references to the same inner objects as the original.

Does deepcopy handle circular references?

Yes. deepcopy keeps a memo of objects already copied, so self references and shared objects are reproduced correctly without infinite recursion.

Why does deepcopy raise cannot pickle?

Some objects such as locks, files and sockets cannot be copied. Define a custom deepcopy method that shares or recreates them, or keep them out of copied data.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

Jaccard Similarity: Formula, Python, MinHash

Jaccard similarity measures overlap between two sets as intersection over union. Work the formula, compute it in Python for words…

Read article →
Software Development

Unix Timestamp Explained: Seconds vs ms

A Unix timestamp counts seconds since 1 January 1970 UTC. Learn seconds vs milliseconds, converting in Python, JavaScript and SQL,…

Read article →
DevOps

Git Hooks: Pre-commit and Commit-msg Examples

Git hooks run scripts at key moments such as commit and push. See tested pre-commit, commit-msg and prepare-commit-msg hooks, sharing…

Read article →
Test Automation

API Testing in Python: Pytest + Requests Suite

Build an API testing suite in Python with pytest and requests: fixtures, status codes, headers, JSON schema validation, CRUD flows,…

Read article →
Software Testing

Test Design Techniques: 9 Methods + Examples

Test design techniques help you write fewer tests that catch more bugs. Learn 9 black-box, white-box and experience-based methods with…

Read article →
Databases & SQL

SQL Temporary Table: Create, Use and Drop It

SQL temporary tables store interim results for one session. Learn syntax for SQL Server, PostgreSQL, MySQL, Oracle and SQLite, session…

Read article →
Databases & SQL

SQL CROSS JOIN: Every Combination + Examples

SQL CROSS JOIN returns every combination of rows from two tables. See the row math, product variants, calendar grids that…

Read article →
Software Development

JavaScript Date: Format, Parse, Timezone Bugs

JavaScript Date explained with tested Node.js code: zero-based months, the date-only UTC parsing trap, toISOString, getTimezoneOffset, Intl formatting, date math…

Read article →
Software Development

Python Requests: GET, POST, Timeouts, Retries

Python requests explained with code tested against a local API: GET with params, POST json vs data, headers and auth,…

Read article →
Software Development

Python Deepcopy vs Copy: Fix Nested Lists

Python deepcopy vs shallow copy explained with tested code: aliasing, nested lists, the [[0]*3]*3 trap, memo and cycles, custom __deepcopy__,…

Read article →
Artificial Intelligence

TF-IDF Explained: Formula and Worked Example

TF-IDF scores a word by how often it appears in a document and how rare it is across all documents….

Read article →
Software Development

Base64 Encoding Explained, With Python Code

Base64 encoding turns bytes into 64 safe text characters. See the bit-level steps, padding, URL-safe base64, the 33% size cost,…

Read article →
DevOps

Git Tag: Create, Push and Delete Release Tags

Git tag explained with tested commands: annotated vs lightweight tags, listing and sorting, pushing with follow-tags, deleting local and remote…

Read article →
Test Automation

Pytest Coverage: pytest-cov and Branch Gaps

Measure pytest coverage with pytest-cov: term-missing reports, branch coverage, HTML reports, pyproject settings and a fail-under gate for CI, with…

Read article →
Software Testing

Error Guessing in Testing: Checklist + Demo

Error guessing uses tester experience to predict where code breaks. Get a ready checklist of risky inputs, a step by…

Read article →
Databases & SQL

SQL LAG and LEAD: Compare Rows, 5 Examples

SQL LAG and LEAD read values from the previous or next row without a self join. Five tested examples: change,…

Read article →
Databases & SQL

SQL STRING_AGG: Combine Rows Into One String

SQL STRING_AGG joins values from many rows into one string per group. See syntax for PostgreSQL, SQL Server, MySQL GROUP_CONCAT,…

Read article →
Software Development

JavaScript Classes: Private Fields and Extends

JavaScript classes explained with tested Node.js code: constructors, methods, #private fields, getters, static members, extends and super, plus the errors…

Read article →
Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →