Save docker-compose logs to a file

Logging Problem Overview

I have unit tests running on my build server and would like to capture the log results for analysis when something fails. I have yet to find a way to redirect the output of docker-compose logs to a file, or to find where the log files themselves actually live.

I want the equivalent of:

docker-compose logs > logs.txt

Edit - clarification:

All of my docker containers produce useful logs, which a manual run of docker-compose logs reveals. I want to script this process to save those same logs to a file that is an artifact on my build server. Essentially, the output of docker-compose logs saved to a file, however docker-compose logs never exits.

Logging Solutions

Solution 1 - Logging

By default docker uses the json-file driver to record your containers logs and the raw json output of the logs can be found in:

/var/lib/docker/containers/[container-id]/[container-id]-json.log

You can get this location by running:

docker inspect --format='{{.LogPath}}' [container-id or container-name]

When you run docker-compose logs [service-name], docker-compose will attach to the service (container) you reference and the LogPrinter object will output the contents of the above file, but formatted so they're easier to read.

Solution 2 - Logging

I am not sure what you are trying to achieve. Are you trying to reproduce

docker-compose logs > logs.txt

within the compose file as an instruction? Or is your issue that the redirect does not "catch" the whole output?

In the later, you can do:

docker-compose logs --no-color >& logs.txt

docker-compose logs --no-color |& tee logs.txt

to both see the logs on the terminal and dump it to a file at the same time.

Solution 3 - Logging

In later release docker-compose 1.7.x+, it is fixed. see https://github.com/docker/compose/issues/2227 & https://github.com/docker/compose/releases/tag/1.7.0

Before that, there is another way to achieve it, the solution of accepted answer is to access host files directly, Which may be not applicable for remote/security case.

Below we can get the container name from docker-compose ps command and let the docker logs command to loop

docker-compose ps | tail -n +3 | awk '{print $1}' | xargs -n1 docker logs

Solution 4 - Logging

The docker-compose logs command does terminate (unless you add the --follow setting).

The likely problem here is that the logs are so large it takes some time to format and output them. You can reduce this issue if you limit the output by specifying either/both the number of lines you want to read and the container you want to read from.

Try:

docker-compose logs --no-color --tail=1000 <service-name> > logs.txt

NOTE: service-name is name of service taken from docker-compose file - not container name or container id

(Adjust the tail number as required. I've added "--no-color" to simplify the output but it is not needed.)

Solution 5 - Logging

The solution I made is based on Chris McKinnel's answer with some enhancements.

I needed to dump last 1MIO records from my nginx container and docker-compose logs --tail 1000000 nginx > last1mio.log was too slow and the output was not exactly the same as usual nginx files have.

This is solution that works for me:

docker inspect --format='{{.LogPath}}' nginx-1 \
| xargs tail -n 1000000 \
| jq -r -j .log > last1mio.log 2>&1

Notes:

you need to know your container name, in my case is nginx-1 - I use docker-compose option container_name: for having this constant
I am using JQ utility to parse json - with some special options to get the format the same as original nginx log files usually have.

In my case dump of last 1mio lines of nginx log takes cca 10s.

Solution 6 - Logging

To see where the logs ofthe container are we use docker inspect.

docker inspect --format={{.LogPath}} <container id>

This will give you the path of the json file where the logs are saved.

in my case /var/lib/docker/containers/xxxx

Then i go there and can coppy the json file.

Something strange is that i only had the last 2 weeks of the logs. Is there any way to get more?

Solution 7 - Logging

In my case, docker was using journald by default and my LogPath was blank.

I added the following to my docker-compose and then got a log to view:

logging:
  driver: "json-file"

View Log:

CONTNAME=
docker inspect --format='{{.LogPath}}' $CONTNAME | xargs tail -f

Content Type	Original Author	Original Content on Stackoverflow
Question	Ryan	View Question on Stackoverflow
Solution 1 - Logging	Chris McKinnel	View Answer on Stackoverflow
Solution 2 - Logging	creack	View Answer on Stackoverflow
Solution 3 - Logging	Larry Cai	View Answer on Stackoverflow
Solution 4 - Logging	Matthew Wilcoxson	View Answer on Stackoverflow
Solution 5 - Logging	Robert Lujo	View Answer on Stackoverflow
Solution 6 - Logging	Dani I.G.	View Answer on Stackoverflow
Solution 7 - Logging	FreeSoftwareServers	View Answer on Stackoverflow