Finding a string in docker logs of container
DockerGrepDocker Problem Overview
What is the best way to find a specific string in the logs of a docker container? Let's say I want to see all requests, that are made in the "nginx" docker image that came from a ip starting with "127."
grep wont work as expected on docker logs command:
docker logs nginx | grep "127."
It prints all logs, but does not filter the result!
Docker Solutions
Solution 1 - Docker
this can happen if the container is logging to stderr, piping works only for stdout, so try:
docker logs nginx 2>&1 | grep "127."
"2>&1" will tells the shell to redirect stderr to stdout and give the output to grep using pipe.
Solution 2 - Docker
As vim fan I prefer to use less
and search with /
docker logs nginx 2>&1 | less
Solution 3 - Docker
docker logs <container_name> 2>&1 | grep <string>
Solution 4 - Docker
You could also use an anonymous pipe:
docker logs nginx 2> >(grep '127.')
Solution 5 - Docker
Additionally, I found it usefull to highlight some terms that I'm searching for in the logs. Especially on productive installations where a lot of log output is generated. In my case I want to highlight COUNT(*)
statements. But with a simple grep I can't see the entire SQL statement since it's a multi line statement. This is possible with -E
switch and some regex:
For example, the following snippet search for all queries that contain COUNT(*)
as well as count(*)
:
docker logs <containerName> -f | grep --line-buffered -i -E --color "select count\(\*\)|$"
Some explanation:
docker logs -f
tell docker to follow the logs, so the filter applys also to new entrys like when viewing it usingtail -f
- greps
--line-buffered
switch flushes the output on every line, which is required to grep in real time whendocker logs -f
is used -E
is an extended regex pattern, required to apply our pattern that allow us returning also the non matching results--color
highlights the matched parts (seems the default behaviour on my Ubuntu 16.04 LTS, but maybe not on other distributions, so I included it here to be safe)*
is escaped to disable its special glob functionality, where(
, and)
are masked to avoid their regex meaning as group, which is enabled by-E
switch
If the container logs to stderr, you can pipe them as Edoardo already wrote for a simple grep:
docker logs <containerName> -f 2>&1 | grep --line-buffered -i -E --color "select count\(\*\)|$"
The -f
switch could be omitted if no live grep is wanted. In both cases, you see the entire log buth with highlighted search term like this:
Solution 6 - Docker
To follow up on the comments and clarify this for anyone else hitting this issue. Here is the simplest way I can see to search an nginx container log.
docker logs nginx > stdout.log 2>stderr.log
cat stdout.log | grep "127."
IMO its kinda messy because you need to create and delete these potentially very large files. Hopefully we'll get some tooling to make it a bit more convenient.
Solution 7 - Docker
Run following command to extract container name for image nginx -
docker ps --filter ancestor=nginx
Copy container ID from last command & then extract log path for your container through below command
grep "127." `docker inspect --format={{.LogPath}} <ContainerName>`
Solution 8 - Docker
I generally use it with -f
option as well, when I am debugging the issue
docker logs -f nginx 2>&1 | grep "127."
It will show us, what we are expecting in real-time.
To include timestamps, add -t
docker logs -ft nginx 2>&1 | grep "127."
Solution 9 - Docker
First, use this command ( b1e3c456f07f is the container id ):
docker inspect --format='{{.LogPath}}' b1e3c456f07f
The result will be something like this:
/var/lib/docker/containers/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767-json.log
Second, use this command ( you can use vim if you like ):
nano /var/lib/docker/containers/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767/b1e3c456f07f2cb3ae79381ada33a034041a10f65174f52bc1792110b36fb767-json.log