Best method for reading newline delimited files and discarding the newlines?

PythonFileReadline

Python Problem Overview


I am trying to determine the best way to handle getting rid of newlines when reading in newline delimited files in Python.

What I've come up with is the following code, include throwaway code to test.

import os

def getfile(filename,results):
   f = open(filename)
   filecontents = f.readlines()
   for line in filecontents:
     foo = line.strip('\n')
     results.append(foo)
   return results

blahblah = []

getfile('/tmp/foo',blahblah)

for x in blahblah:
    print x

Python Solutions


Solution 1 - Python

lines = open(filename).read().splitlines()

Solution 2 - Python

Here's a generator that does what you requested. In this case, using rstrip is sufficient and slightly faster than strip.

lines = (line.rstrip('\n') for line in open(filename))

However, you'll most likely want to use this to get rid of trailing whitespaces too.

lines = (line.rstrip() for line in open(filename))

Solution 3 - Python

What do you think about this approach?

with open(filename) as data:
    datalines = (line.rstrip('\r\n') for line in data)
    for line in datalines:
        ...do something awesome...

Generator expression avoids loading whole file into memory and with ensures closing the file

Solution 4 - Python

for line in file('/tmp/foo'):
    print line.strip('\n')

Solution 5 - Python

Just use generator expressions:

blahblah = (l.rstrip() for l in open(filename))
for x in blahblah:
    print x

Also I want to advise you against reading whole file in memory -- looping over generators is much more efficient on big datasets.

Solution 6 - Python

I use this

def cleaned( aFile ):
    for line in aFile:
        yield line.strip()

Then I can do things like this.

lines = list( cleaned( open("file","r") ) )

Or, I can extend cleaned with extra functions to, for example, drop blank lines or skip comment lines or whatever.

Solution 7 - Python

I'd do it like this:

f = open('test.txt')
l = [l for l in f.readlines() if l.strip()]
f.close()
print l

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionsolarceView Question on Stackoverflow
Solution 1 - PythonCurt HagenlocherView Answer on Stackoverflow
Solution 2 - PythonTimoLinnaView Answer on Stackoverflow
Solution 3 - PythonPaweł PrażakView Answer on Stackoverflow
Solution 4 - PythonDavid ZView Answer on Stackoverflow
Solution 5 - PythonartyomView Answer on Stackoverflow
Solution 6 - PythonS.LottView Answer on Stackoverflow
Solution 7 - PythonrouterguyView Answer on Stackoverflow