Python
Topic 13 · Writing real programs · ⏱ about 3 h

Files: Reading & Writing Text, CSV & JSON

Keep data after your program ends: open, read and write text files safely with with, build file paths with pathlib, and read and write the two formats you will meet most, CSV tables and JSON records.

Show:
In this topic
Beginner

Why files?

Variables disappear when a program ends. To keep data, or to work with data someone else made, you use files. Almost every data project starts by reading a file (a spreadsheet exported as CSV, a JSON download from a website, a folder of text) and ends by writing one (a cleaned dataset, a report, a saved model).

The examples on this page run in a small private file system inside your browser, so you can create and read files safely.

Beginner

Writing and reading a text file

Use open(filename, mode) inside a with block. with closes the file automatically when the block ends, even if an error happens, which prevents lost data and locked files. The mode says what you want to do:

ModeMeaning
"r"read (the default); error if the file does not exist
"w"write: creates the file, or wipes it if it exists
"a"append: add to the end, keeping what is there

Always say encoding="utf-8", so accents, other alphabets and emoji work the same on every computer.

Python
with open("notes.txt", "w", encoding="utf-8") as f:
    f.write("first line\n")          # \n ends the line: write does not add it for you
    f.write("second line\n")

with open("notes.txt", "a", encoding="utf-8") as f:
    f.write("added later\n")

with open("notes.txt", encoding="utf-8") as f:
    text = f.read()                  # the whole file as one string
print(text)

with open("notes.txt", encoding="utf-8") as f:
    for line in f:                   # one line at a time: good for big files
        print("->", line.strip())    # strip removes the \n at the end
Open in playground ↗
Beginner

Paths with pathlib

A path says where a file is, such as data/sales.csv. Windows writes paths with \ and macOS/Linux with /; the pathlib module handles the difference for you. Build paths with / between parts, and ask a path questions with its methods.

Python
from pathlib import Path

data = Path("data")
data.mkdir(exist_ok=True)            # make a folder (no error if it is already there)
report = data / "report.txt"         # join parts with /
report.write_text("all good\n", encoding="utf-8")   # quick write
print(report, report.exists(), report.suffix, report.stem)
print(report.read_text(encoding="utf-8"))

for p in Path(".").glob("*.txt"):    # every .txt file in the current folder
    print("found:", p.name)
Open in playground ↗
Beginner

Relative and absolute paths

A relative path like data/sales.csv is looked up from the program's current working directory: the folder the terminal was in when you ran it. If you run your script from a different folder, the file is "not found" even though it exists. Check where you are with Path.cwd(). An absolute path starts from the top of the drive (C:\Users\sita\… or /Users/sita/…) and works from anywhere, but only on that computer. In projects, keep data in a folder next to your code and use relative paths.

Beginner

CSV: tables as text

CSV (comma-separated values) is the simplest table format: one row per line, columns separated by commas, usually with a header row of column names. Every spreadsheet program can save as CSV. Python's csv module handles the tricky parts (commas inside quotes, for example). csv.DictReader gives each row as a dictionary keyed by the header names.

Python
import csv

rows = [
    {"name": "Sita", "city": "Brisbane", "score": 82},
    {"name": "Leo",  "city": "Sydney",   "score": 91},
    {"name": "Mia",  "city": "Brisbane", "score": 77},
]
with open("scores.csv", "w", newline="", encoding="utf-8") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "city", "score"])
    writer.writeheader()
    writer.writerows(rows)

print(open("scores.csv", encoding="utf-8").read())

total = 0
with open("scores.csv", newline="", encoding="utf-8") as f:
    for row in csv.DictReader(f):
        score = int(row["score"])      # everything read from CSV is text: convert!
        total += score
        print(row["name"], "from", row["city"], "scored", score)
print("average:", round(total / len(rows), 1))
Open in playground ↗
Beginner

JSON: nested records as text

JSON is the format websites and APIs use to send data. It looks almost exactly like Python dictionaries and lists, with small differences: strings must use double quotes, and it uses true, false and null where Python has True, False and None. The json module converts both ways: json.dump writes Python data to a file, json.load reads it back.

Python
import json

settings = {"user": "Sita", "dark_mode": True, "recent": ["a.csv", "b.csv"], "limit": None}
with open("settings.json", "w", encoding="utf-8") as f:
    json.dump(settings, f, indent=2)

print(open("settings.json", encoding="utf-8").read())

with open("settings.json", encoding="utf-8") as f:
    loaded = json.load(f)
print(loaded["recent"][0], loaded["dark_mode"])

text = json.dumps({"ok": True})      # dumps/loads: to and from a string instead of a file
print(text, json.loads(text))
Open in playground ↗
Try it

Try it: JSON ⇄ Python

Edit the JSON and see the Python value it becomes. Try breaking a JSON rule (a trailing comma, single quotes) and read the error.

Interactive: this part runs in your browser.

Beginner

When files go wrong

File errors are common and usually easy to fix. FileNotFoundError: wrong name or wrong folder (check Path.cwd() and the spelling, including the extension). PermissionError: the file is open in another program (often Excel) or the folder is protected. UnicodeDecodeError: the file is not UTF-8; try encoding="latin-1" or encoding="utf-8-sig" for CSVs saved by Excel. The Errors lesson shows how to handle these with try and except.

Beginner

Practice: check yourself

Write the code, then press ✔ Check my answer: hidden tests run your code and tell you exactly what is right. ▶ Run just shows the output. Stuck? Use a hint; the solution is there if you need it.

Exercise 1. Write and read back

Write the three names in names to names.txt, one per line. Then read the file back into a list loaded (without the new-line characters).

Exercise 2. Average from a CSV

The setup creates marks.csv with columns name,mark. Read it with csv.DictReader and set average to the average mark (rounded to 1 decimal) and best to the name with the highest mark.

Exercise 3. Save settings as JSON

Save settings to settings.json with json.dump, then load it back into loaded with json.load.

Beginner

Summary

Open files inside with blocks, choose the mode ("r", "w", "a"), and always set encoding="utf-8". Use pathlib for paths, csv.DictReader/DictWriter for tables, and json.load/dump for nested data. Later, pandas reads CSV and JSON in one line, but it rests on exactly these ideas.

Key words

file
data saved on disk that outlives your program
mode
"r" read, "w" write (wipes), "a" append
with
a block that closes the file automatically
encoding
how characters are stored as bytes; use UTF-8
path
the location of a file: relative (from the working folder) or absolute
CSV
a table as text: rows on lines, columns separated by commas
JSON
nested data as text: objects (dicts) and arrays (lists)

Common mistakes

  • Opening with "w" when you meant to add: it wipes the file. Use "a".
  • Forgetting that everything read from a text or CSV file is a string: convert with int() or float().
  • Running the script from another folder, so a relative path is not found.
  • Forgetting \n when writing lines, so everything ends up on one line.
  • Writing CSV without newline="" on Windows, which adds blank lines between rows.
  • JSON with single quotes or a trailing comma: it is not valid JSON.
Check yourself

Quiz

1. Why open files with with open(...) as f:?

2. What does mode "w" do to an existing file?

3. You read a number from a CSV with DictReader. What type is it?

4. Which module converts between Python dictionaries and JSON text?

5. Your script says FileNotFoundError for "data/sales.csv", but the file exists. What is the most likely cause?

Go deeper

📚 Further reading

Trusted, mostly free sources: official documentation, well-known textbooks and courses. When you want to confirm or go deeper on anything here, start with these.

Ready