Python File Handling: Read, Write, CSV, JSON and pathlib With Examples

Logs, reports, configuration, exports: sooner or later, every script touches the file system. Python file handling is simple to start with, but a few details decide whether your code is reliable or quietly destroys data.

This guide covers text files, CSV, JSON, binary data and paths, with a focus on safe habits. We ran these examples on Python 3.13 in a temporary folder, and the output shown comes from that run.

Open files with open() and with

The built-in open() function returns a file object. Use it inside a with block:

with open("notes.txt", "w", encoding="utf-8") as file:
    file.write("first line\n")
    file.write("second line\n")

print(file.closed)

# True

When the block ends, Python closes the file, even if an error occurred inside it. Without with, you have to call close() yourself, and an exception can skip that line. Using a file after it has closed raises an error:

file.read()

# ValueError: I/O operation on closed file.

File modes

The second argument to open() says what you plan to do:

Add b for binary data, as in "rb" or "wb". Text is the default, and it can also be written explicitly as t.

Write and append

write() does not add a line break, so include \n yourself. Appending keeps what is already there:

with open("notes.txt", "a", encoding="utf-8") as file:
    file.write("third line\n")

Compare that with mode "w", which replaced our three lines with one, with no warning at all:

with open("notes.txt", "w", encoding="utf-8") as file:
    file.write("replaced\n")

# notes.txt now contains only: replaced

When you must not overwrite something, use "x". It raised FileExistsError in our test, since notes.txt already existed.

Read a file

There are several ways to read, and the right choice depends on the file’s size:

with open("notes.txt", encoding="utf-8") as file:
    print(repr(file.read()))

# 'first line\nsecond line\n'

with open("notes.txt", encoding="utf-8") as file:
    for line in file:
        print(line.strip())

# first line
# second line
# third line

Notice the \n characters kept in the raw string. strip() removes them. A file object also remembers its position, so reading continues where it left off:

with open("notes.txt", encoding="utf-8") as file:
    print(file.readline().strip())
    print(file.readlines())

# first line
# ['second line\n', 'third line\n']

Looping over the file is the key technique for big logs and exports. It reads a line at a time, so a multi-gigabyte file does not have to fit in memory.

Handle file errors

Files may be missing, locked or unreadable, so plan for that:

try:
    with open("missing.txt", encoding="utf-8") as file:
        data = file.read()
except FileNotFoundError as error:
    print(error)

# [Errno 2] No such file or directory: 'missing.txt'

Try opening the file and catching the error, rather than checking first with exists(), since the file could vanish between the check and the open. Our guide to Python exception handling covers this approach.

CSV files

Do not split CSV lines by comma yourself, because quoted values can contain commas. The standard csv module handles those rules. DictWriter and DictReader are the most readable options:

import csv

with open("orders.csv", "w", newline="", encoding="utf-8") as file:
    writer = csv.DictWriter(file, fieldnames=["id", "customer", "total"])
    writer.writeheader()
    writer.writerow({"id": 1, "customer": "Asha", "total": 1200})
    writer.writerow({"id": 2, "customer": "Ben", "total": 300})

with open("orders.csv", newline="", encoding="utf-8") as file:
    rows = list(csv.DictReader(file))

print(rows[0])
print(sum(int(r["total"]) for r in rows))

# {'id': '1', 'customer': 'Asha', 'total': '1200'}
# 1500

Two details matter. Pass newline="" when opening CSV files, as the documentation recommends, to prevent blank rows on Windows. And every value comes back as a string, even numbers, which is why we needed int() in the sum. For heavy data work, a library such as pandas can be a better fit.

JSON files

JSON is the usual format for configuration and API data. It maps neatly onto Python dictionaries and lists:

import json

config = {"debug": True, "retries": 3, "hosts": ["a", "b"]}

with open("config.json", "w", encoding="utf-8") as file:
    json.dump(config, file, indent=2)

with open("config.json", encoding="utf-8") as file:
    print(json.load(file)["retries"])

# 3

Remember the naming: dump and load work with files, while dumps and loads work with strings. Types such as dates cannot be stored directly, so you will need to convert them to strings first.

Work with paths using pathlib

Gluing paths together with strings breaks across operating systems. The standard pathlib module handles separators for you and offers handy shortcuts:

from pathlib import Path

path = Path("data") / "reports" / "summary.txt"
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text("all good\n", encoding="utf-8")

print(path.read_text(encoding="utf-8").strip())   # all good
print(path.suffix, path.stem, path.exists())      # .txt summary True
print(sorted(p.name for p in Path(".").glob("*.csv")))
# ['orders.csv']

That last one solves a common headache. Relative paths resolve from where you ran the script, not from where the script lives, so building paths from Path(__file__).parent avoids many “file not found” surprises.

Binary files

Images, PDFs and ZIP files are not text. Open them with b, and you read and write bytes, with no encoding involved:

with open("pixel.bin", "wb") as file:
    file.write(bytes([255, 0, 128]))

with open("pixel.bin", "rb") as file:
    print(file.read())

# b'\xff\x00\x80'

Safer writes

If a program crashes halfway through writing an important file, you can be left with a corrupted result. A common fix is to write to a temporary file first, and then swap it into place:

temp = Path("settings.tmp")
temp.write_text("new", encoding="utf-8")
temp.replace("settings.txt")

print(Path("settings.txt").read_text(), temp.exists())

# new False

replace() overwrites the destination. On the same file system, the rename is generally atomic, so readers see either the old file or the new one rather than a half-written mix.

A practical example: summarize a log

This short script puts several ideas together. It reads a log one line at a time, so its size does not matter, counts each level using Counter, and writes a summary to a new file:

from collections import Counter

counts = Counter()
with open("app.log", encoding="utf-8") as file:
    for line in file:
        counts[line.split()[0]] += 1

with open("summary.txt", "w", encoding="utf-8") as out:
    for level, total in counts.most_common():
        out.write(f"{level}: {total}\n")

# summary.txt
# ERROR: 2
# INFO: 2
# WARN: 1

Our sample log had five lines. A blank line would break line.split()[0], so real scripts should skip empty lines first.

Common mistakes

• Using "w" when you meant "a". Your existing data is gone immediately on open, even if you never write anything.
• Omitting the encoding. The default varies by system, so a script that works on Linux can fail on Windows with accented text or emoji.
• Calling read() on huge files. This can exhaust memory, so loop over lines instead.
• Forgetting int() on CSV values. "10" + "20" gives "1020".
• Leaving out newline="" for CSV. You may get unexpected blank rows on Windows.
• Not testing file code. Use pytest’s tmp_path fixture so that tests never touch real files. See our pytest tutorial.

Final thoughts

A few habits cover almost everything here: use with, set the encoding, think twice about "w", and let csv, json and pathlib do the hard parts. To practise, write a small script that reads a CSV, totals one column and saves a JSON summary to a new folder.

Frequently asked questions

Keep learning