How to redirect docker container logs to a single file?

LoggingDocker

Logging Problem Overview


I want to redirect all the logs of my docker container to single log file to analyse them. I tried

docker logs container > /tmp/stdout.log 2>/tmp/stderr.log

but this gives log in two different file. I already tried

docker logs container > /tmp/stdout.log

but it did not work.

Logging Solutions


Solution 1 - Logging

No need to redirect logs.

Docker by default store logs to one log file. To check log file path run command:

docker inspect --format='{{.LogPath}}' containername

/var/lib/docker/containers/f844a7b45ca5a9589ffaa1a5bd8dea0f4e79f0e2ff639c1d010d96afb4b53334/f844a7b45ca5a9589ffaa1a5bd8dea0f4e79f0e2ff639c1d010d96afb4b53334-json.log

Open that log file and analyse.

if you redirect logs then you will only get logs before redirection. you will not be able to see live logs.

EDIT:

To see live logs you can run below command

tail -f `docker inspect --format='{{.LogPath}}' containername`

Note:

This log file /var/lib/docker/containers/f844a7b45ca5a9589ffaa1a5bd8dea0f4e79f0e2ff639c1d010d96afb4b53334/f844a7b45ca5a9589ffaa1a5bd8dea0f4e79f0e2ff639c1d010d96afb4b53334-json.log will be created only if docker generating logs if there is no logs then this file will not be there. it is similar like sometime if we run command docker logs containername and it returns nothing. In that scenario this file will not be available.

Solution 2 - Logging

How about this option:

docker logs containername >& logs/myFile.log

It will not redirect logs which was asked for in the question, but copy them once to a specific file.

Solution 3 - Logging

docker logs -f <yourContainer> &> your.log &

Explanation:

  • -f (i.e.--follow): writes all existing logs and follows logging everything that comes next.
  • &> redirects both the standard output and standard error.
  • Likely you want to run that method in the background, thus the &.
  • You can separate stdout and stderr by: > output.log 2> error.log (instead of using &>).

Solution 4 - Logging

To capture both stdout & stderr from your docker container to a single log file run the following:

docker logs container > container.log 2>&1

Solution 5 - Logging

Assuming that you have multiple containers and you want to aggregate the logs into a single file, you need to use some log aggregator like fluentd. fluentd is supported as logging driver for docker containers.

So in docker-compose, you need to define the logging driver

  service1:
    image: webapp:0.0.1
    logging:
      driver: "fluentd"
      options:
        tag: service1 

  service2:
        image: myapp:0.0.1
        logging:
          driver: "fluentd"
          options:
            tag: service2

The second step would be update the fluentd conf to cater the logs for both service 1 and service 2

 <match service1>
   @type copy
   <store>
    @type file
    path /fluentd/log/service/service.*.log
    time_slice_format %Y%m%d
    time_slice_wait 10m
    time_format %Y%m%dT%H%M%S%z
  </store>
 </match> 
 <match service2>
    @type copy
   <store>
    @type file
    path /fluentd/log/service/service.*.log
    time_slice_format %Y%m%d
    time_slice_wait 10m
    time_format %Y%m%dT%H%M%S%
  </store>
 </match> 

In this config, we are asking logs to be written to a single file to this path
/fluentd/log/service/service.*.log

and the third step would be to run the customized fluentd which will start writing the logs to file.

Here is the link for step by step instructions

Bit Long, but correct way since you get more control over log files path etc and it works well in Docker Swarm too .

Solution 6 - Logging

First check your container id

docker ps -a

You can see first row in CONTAINER ID columns. Probably it looks like this "3fd0bfce2806" then type it in shell

docker inspect --format='{{.LogPath}}' 3fd0bfce2806

You will see something like this

/var/lib/docker/containers/3fd0bfce2806b3f20c2f5aeea2b70e8a7cff791a9be80f43cdf045c83373b1f1/3fd0bfce2806b3f20c2f5aeea2b70e8a7cff791a9be80f43cdf045c83373b1f1-json.log

then you can see it as

cat /var/lib/docker/containers/3fd0bfce2806b3f20c2f5aeea2b70e8a7cff791a9be80f43cdf045c83373b1f1/3fd0bfce2806b3f20c2f5aeea2b70e8a7cff791a9be80f43cdf045c83373b1f1-json.log

It would be in JSON format, you can use the timestamp to trace errors

Solution 7 - Logging

Bash script to copy all container logs to a specified directory:

#!/usr/bin/env bash

TARGET_DIR=~/logs/docker_logs
mkdir -p "$TARGET_DIR"
for name in `sudo docker ps --format '{{.Names}}'`;
do
    path=$(sudo docker inspect --format='{{.LogPath}}' $name)
    sudo cp -rf "$path" "$TARGET_DIR"/$name.log
done

Solution 8 - Logging

The easiest way that I use is this command on terminal:

docker logs elk > /home/Desktop/output.log

structure is:

docker logs <Container Name> > path/filename.log

Solution 9 - Logging

Below works for me,

docker logs <container-id> > mylogs.txt 2>&1

Solution 10 - Logging

If you work on Windows and use PowerShell (like me), you could use the following line to capture the stdout and stderr:

 docker logs <containerId> | Out-File 'C:/dev/mylog.txt'

I hope it helps someone!

Solution 11 - Logging

docker logs -f docker_container_name >> YOUR_LOG_PATH 2>&1 &

Solution 12 - Logging

Since Docker merges stdout and stderr for us, we can treat the log output like any other shell stream. To redirect the current logs to a file, use a redirection operator

$ docker logs test_container > output.log
docker logs -f test_container > output.log

Instead of sending output to stderr and stdout, redirect your application’s output to a file and map the file to permanent storage outside of the container.

$ docker logs test_container> /tmp/output.log

Docker will not accept relative paths on the command line, so if you want to use a different directory, you’ll need to use the complete path.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionKETAN PATILView Question on Stackoverflow
Solution 1 - Loggingpl_rockView Answer on Stackoverflow
Solution 2 - LoggingEddy HernandezView Answer on Stackoverflow
Solution 3 - LoggingjuanmirocksView Answer on Stackoverflow
Solution 4 - LoggingJean VelloenView Answer on Stackoverflow
Solution 5 - LoggingAbhishek GalodaView Answer on Stackoverflow
Solution 6 - Loggingkiran beethojuView Answer on Stackoverflow
Solution 7 - LoggingMugenView Answer on Stackoverflow
Solution 8 - LoggingitssaifurrehmanView Answer on Stackoverflow
Solution 9 - LoggingNaveenrajView Answer on Stackoverflow
Solution 10 - Loggingmirind4View Answer on Stackoverflow
Solution 11 - Loggingfocus zhengView Answer on Stackoverflow
Solution 12 - LoggingAnish VargheseView Answer on Stackoverflow