6.4.2 – Read CSV text files

CSV records and fields

Here is the complete content of a simple file called activities.csv:

Quiz,Monday,18
Robotics,Wednesday,12

Each line is one record. The first record has three fields: Quiz, Monday and 18. A CSV file may have a header row, but this one does not. It is still text: 18 is stored as the characters 1 and 8, not automatically as an integer.

When Python reads the first line, the value will normally include its line-feed character:

line = "Quiz,Monday,18\n"
clean_line = line.strip("\n")
fields = clean_line.split(",")

The state changes like this:

strip("\n") returns a string with line-feed characters removed from its ends. It does not remove the commas. split(",") then returns a list of the substrings between those commas. The last item, fields[2], remains the string "18".

The programs in this lesson are given simple, unquoted CSV data: no field contains a comma or a line break, and there are no blank or malformed rows. That guarantee matters because split(",") is not a complete parser for more complex quoted CSV fields.

Reading CSV data

Opening a file in read mode creates a file object positioned at the start of the file:

session_file = open("sessions.csv", "r")

The "r" means read. A program should call close() when it has finished with the file:

session_file.close()

Pearson's Programming Language Subset gives three useful ways to take lines from a CSV file. Each reads from the file's current position.

Read one line:

line = session_file.readline()

This returns the next line as a string, or the empty string at end of file.

Read all remaining lines:

rows = session_file.readlines()

This returns a list of strings. Direct iteration processes the remaining lines without first collecting them:

for line in session_file:
    print(line.strip("\r\n"))

Use this complete original sessions.csv file for the examples. Assume every shown record ends with \n, including the final one.

Maya,Python,14
Owen,Networks,11
Zara,Logic,16

If the file has just been opened, this code stores only the first line:

session_file = open("sessions.csv", "r")
first_line = session_file.readline()
session_file.close()

The predicted value of first_line is "Maya,Python,14\n". By contrast, this version reads every line into a list:

session_file = open("sessions.csv", "r")
rows = session_file.readlines()
session_file.close()

The predicted value is:

["Maya,Python,14\n", "Owen,Networks,11\n", "Zara,Logic,16\n"]

For processing every record, direct iteration avoids having to store all the lines first:

session_file = open("sessions.csv", "r")

for line in session_file:
    fields = line.strip("\n").split(",")
    print("{}: {}".format(fields[0], fields[2]))

session_file.close()

The first loop pass can be traced as follows:

The supplied predicted output for all three passes is:

Maya: 14
Owen: 11
Zara: 16

After readlines() or a complete file loop reaches the end, another read from the same file object has no record left to return. Closing and opening the file again creates a new file object positioned at the start.