append new row to old csv file python
PythonCsvAppendWriterPython Problem Overview
I am trying to add a new row to my old csv file. Basically, it gets updated each time I run the Python script.
Right now I am storing the old csv rows values in a list and then deleting the csv file and creating it again with the new list value.
Wanted to know are there any better ways of doing this.
Python Solutions
Solution 1 - Python
with open('document.csv','a') as fd:
fd.write(myCsvRow)
Opening a file with the 'a'
parameter allows you to append to the end of the file instead of simply overwriting the existing content. Try that.
Solution 2 - Python
I prefer this solution using the csv
module from the standard library and the with
statement to avoid leaving the file open.
The key point is using 'a'
for appending when you open the file.
import csv
fields=['first','second','third']
with open(r'name', 'a') as f:
writer = csv.writer(f)
writer.writerow(fields)
If you are using Python 2.7 you may experience superfluous new lines in Windows. You can try to avoid them using 'ab'
instead of 'a'
this will, however, cause you https://stackoverflow.com/questions/34283178/typeerror-a-bytes-like-object-is-required-not-str-in-python-and-csv in Python 3.6. Adding the newline=''
, as Natacha suggests, will cause you a backward incompatibility between Python 2 and 3.
Solution 3 - Python
Based in the answer of @G M and paying attention to the @John La Rooy's warning, I was able to append a new row opening the file in 'a'
mode.
> Even in windows, in order to avoid the newline problem, you must declare it as newline=''
.
>
> Now you can open the file in 'a'
mode (without the b).
import csv
with open(r'names.csv', 'a', newline='') as csvfile:
fieldnames = ['This','aNew']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writerow({'This':'is', 'aNew':'Row'})
I didn't try with the regular writer (without the Dict), but I think that it'll be ok too.
Solution 4 - Python
Are you opening the file with mode of 'a' instead of 'w'?
See Reading and Writing Files in the python docs
> # 7.2. Reading and Writing Files
> open() returns a file object, and is most commonly used with two arguments: open(filename, mode).
>
> >>> f = open('workfile', 'w')
> >>> print f
Solution 5 - Python
If the file exists and contains data, then it is possible to generate the fieldname
parameter for csv.DictWriter
automatically:
# read header automatically
with open(myFile, "r") as f:
reader = csv.reader(f)
for header in reader:
break
# add row to CSV file
with open(myFile, "a", newline='') as f:
writer = csv.DictWriter(f, fieldnames=header)
writer.writerow(myDict)
Solution 6 - Python
I use the following approach to append a new line in a .csv file:
pose_x = 1
pose_y = 2
with open('path-to-your-csv-file.csv', mode='a') as file_:
file_.write("{},{}".format(pose_x, pose_y))
file_.write("\n") # Next line.
[NOTE]:
mode='a'
is append mode.
Solution 7 - Python
If you use pandas, you can append your dataframes to an existing CSV file this way:
df.to_csv('log.csv', mode='a', index=False, header=False)
With mode='a' we ensure that we append, rather than overwrite, and with header=False we ensure that we append only the values of df rows, rather than header + values.
Solution 8 - Python
# I like using the codecs opening in a with
field_names = ['latitude', 'longitude', 'date', 'user', 'text']
with codecs.open(filename,"ab", encoding='utf-8') as logfile:
logger = csv.DictWriter(logfile, fieldnames=field_names)
logger.writeheader()
# some more code stuff
for video in aList:
video_result = {}
video_result['date'] = video['snippet']['publishedAt']
video_result['user'] = video['id']
video_result['text'] = video['snippet']['description'].encode('utf8')
logger.writerow(video_result)