How to remove trailing whitespace in code, using another script?


Python Problem Overview

Something like:

import fileinput

for lines in fileinput.FileInput("test.txt", inplace=1):
 	lines = lines.strip()
 	if lines == '': continue
 	print lines

But nothing is being printed on stdout.

Assuming some string named foo:

foo.lstrip() # to remove leading white space
foo.rstrip() # to remove trailing whitespace
foo.strip()  # to remove both lead and trailing whitespace

Python Solutions

Solution 1 - Python

fileinput seems to be for multiple input streams. This is what I would do:

with open("test.txt") as file:
    for line in file:
        line = line.rstrip()
        if line:

Solution 2 - Python

You don't see any output from the print statements because FileInput redirects stdout to the input file when the keyword argument inplace=1 is given. This causes the input file to effectively be rewritten and if you look at it afterwards the lines in it will indeed have no trailing or leading whitespace in them (except for the newline at the end of each which the print statement adds back).

If you only want to remove trailing whitespace, you should use rstrip() instead of strip(). Also note that the if lines == '': continue is causing blank lines to be completely removed (regardless of whether strip or rstrip gets used).

Unless your intent is to rewrite the input file, you should probably just use for line in open(filename):. Otherwise you can see what's being written to the file by simultaneously echoing the output to sys.stderr using something like the following (which will work in both Python 2 and 3):

from __future__ import print_function
import fileinput
import sys

for line in (line.rstrip() for line in
                fileinput.FileInput("test.txt", inplace=1)):
    if line:
        print(line, file=sys.stderr)

Solution 3 - Python

If you're looking to tidy up for PEP8, this will trim trailing whitespace for your whole project:

import os

PATH = '/path/to/your/project'

for path, dirs, files in os.walk(PATH):
    for f in files:
        file_name, file_extension = os.path.splitext(f)
        if file_extension == '.py':
            path_name = os.path.join(path, f)
            with open(path_name, 'r') as fh:
                new = [line.rstrip() for line in fh]
            with open(path_name, 'w') as fh:
                [fh.write('%s\n' % line) for line in new]

Solution 4 - Python

This is the sort of thing that sed is really good at: $ sed 's/[ \t]*$//'. Be aware the you will probably need to literally type a TAB character instead of \t for this to work.

Solution 5 - Python

It seems, fileinput.FileInput is a generator. As such, you can only iterate over it once, then all items have been consumed and calling it's next method raises StopIteration. If you want to iterate over the lines more than once, you can put them in a list:


Then call rstrip on them.

Solution 6 - Python

Save as

#!/usr/bin/env python
Fix trailing whitespace and line endings (to Unix) in a file.
Usage: python

import os
import sys

def main():
    """ Parse arguments, then fix whitespace in the given file """
    if len(sys.argv) == 2:
        fname = sys.argv[1]
        if not os.path.exists(fname):
            print("Python file not found: %s" % sys.argv[1])
        print("Invalid arguments. Usage: python")

def fix_whitespace(fname):
    """ Fix whitespace in a file """
    with open(fname, "rb") as fo:
        original_contents =
    # "rU" Universal line endings to Unix
    with open(fname, "rU") as fo:
        contents =
    lines = contents.split("\n")
    fixed = 0
    for k, line in enumerate(lines):
        new_line = line.rstrip()
        if len(line) != len(new_line):
            lines[k] = new_line
            fixed += 1
    with open(fname, "wb") as fo:
    if fixed or contents != original_contents:
        print("************* %s" % os.path.basename(fname))
    if fixed:
        slines = "lines" if fixed > 1 else "line"
        print("Fixed trailing whitespace on %d %s" \
              % (fixed, slines))
    if contents != original_contents:
        print("Fixed line endings to Unix (\\n)")

if __name__ == "__main__":

Solution 7 - Python

It's a bit surprising seeing multiple answers suggesting to use python for this task, as there's no need to write a multi-line program for this.

Standard Unix tools like sed, awk or perl can achieve this easily straight from the command-line.

e.g anywhere you have perl (Windows, Mac, Linux) the following should achieve what the OP asked:

perl -i -pe 's/[ \t]+$//;' files...

Explanation of the arguments to perl:

-i   # run the edit "in place" (modify the original file)
-p   # implies a loop with a final print over every input line
-e   # next arg is the perl expression to apply (to every line)

s/[ \t]$// is a substitution regex s/FROM/TO/: replace every trailing (end of line) non-empty space (spaces or tabs) with nothing.


  • One liner, no programming needed
  • Works on multiple (any number) of files
  • Works correctly on standard-input (no file arguments given)

> Edit: > > Newer versions of perl support \h (any horizontal-space character), so the solution becomes even shorter: > > perl -i -pe 's/\h+$//;' files... >

More generally, if you want to modify any number of files directly from the command line, replacing every appearance of FOO with BAR, you may always use this generic template:

perl -i -pe 's/FOO/BAR/' files...


All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionkamalView Question on Stackoverflow
Solution 1 - PythonnmichaelsView Answer on Stackoverflow
Solution 2 - PythonmartineauView Answer on Stackoverflow
Solution 3 - PythonPartialOrderView Answer on Stackoverflow
Solution 4 - PythonHank GayView Answer on Stackoverflow
Solution 5 - PythonXORcistView Answer on Stackoverflow
Solution 6 - PythonCzarek TomczakView Answer on Stackoverflow
Solution 7 - PythonarielfView Answer on Stackoverflow