How to get files in a directory, including all subdirectories
PythonPython Problem Overview
I'm trying to get a list of all log files (.log) in directory, including all subdirectories.
Python Solutions
Solution 1 - Python
import os
import os.path
for dirpath, dirnames, filenames in os.walk("."):
for filename in [f for f in filenames if f.endswith(".log")]:
print os.path.join(dirpath, filename)
Solution 2 - Python
You can also use the glob module along with os.walk.
import os
from glob import glob
files = []
start_dir = os.getcwd()
pattern = "*.log"
for dir,_,_ in os.walk(start_dir):
files.extend(glob(os.path.join(dir,pattern)))
Solution 3 - Python
Checkout Python Recursive Directory Walker. In short os.listdir() and os.walk() are your friends.
Solution 4 - Python
A single line solution using only (nested) list comprehension:
import os
path_list = [os.path.join(dirpath,filename) for dirpath, _, filenames in os.walk('.') for filename in filenames if filename.endswith('.log')]
Solution 5 - Python
I have a solution:
import os
for logfile in os.popen('find . -type f -name *.log').read().split('\n')[0:-1]:
print logfile
or
import subprocess
(out, err) = subprocess.Popen(["find", ".", "-type", "f", "-name", "*.log"], stdout=subprocess.PIPE).communicate()
for logfile in out.split('\n')[0:-1]:
print logfile
These two take the advantage of find . -type f -name *.log
.
The first one is simpler but not guaranteed
for white-space when add -name *.log
,
but worked fine for simply find ../testdata -type f
(in my OS X environment).
The second one using subprocess seems more complicated, but this is the white-space safe one (again, in my OS X environment).
This is inspired by Chris Bunch, in the answer https://stackoverflow.com/a/3503909/2834102
Solution 6 - Python
If You want to list in current directory, You can use something like:
import os
for e in os.walk(os.getcwd()):
print e
Just change the
os.getcwd()
to other path to get results there.
Solution 7 - Python
Using standard library's pathlib
:
from pathlib import Path
working_dir = Path()
for path in working_dir.glob("**/*.log"):
print(path)
# OR if you need absolute paths
print(path.absolute())
# OR if you need only filenames without extension for further parsing
print(path.stem)