How to Redirect Docker Logs to a Single File

This post discusses how easy it is to manage your log when you use containers. You can easily use Docker to redirect your logs to a single file. How cool is that? I was in the midst of a big deployment to AWS when I wrote this one. Docker is the only way to go.

The original post is over on Scalyr’s blog.

Sometimes, when troubleshooting or monitoring a Docker container, we need to see the application’s output streams. Containerized applications generate standard output (stdout) and standard error (stderr) like any other software. The Docker daemon merges these streams and directs them to one of several locations, depending on which logging driver is installed. The default driver makes the container output easy to access, and if we want to copy the information to another location, we can redirect docker logs to a file.

Let’s take a look at how we can manipulate logs generated by the default json-file log driver. This driver places the log output in a system directory in JSON formatted files but provides a command line tool for displaying log contents in its original format.

Viewing Container Logs

Let’s start with a simple container. We’ll create one that prints a message every second.

This command is an example from the Docker website. It executes a minimal Docker container with a Bourne shell script that logs a timestamp once every second. We’ll use it for a couple of examples.

So the easiest way to view container logs is with the docker logs command. Start the container and then display the logs.

This command displays all of the output the container has generated so far.

If there’s too much output for a single screen, pipe it to a buffering command like less.

This will still send all of the output generated so far through less, so you’ll need to page through to get to the end.

But sometimes you want to follow the logs as the container is running. There’s a command line option with an appropriate name for that: run docker logs with –follow.

After printing all of the container output so far, the command will continue to follow the logs. You can use the short form of the option -f, similar to the tail command, too.

The docker logs command also offers an array of convenient options, such as filtering output based on time and displaying timestamps. You can read more about them here. But what if you want to redirect the docker logs to a file for viewing offline?

Redirect Docker Logs to File

Since Docker merges stdout and stderr for us, we can treat the log output like any other shell stream. To redirect the current logs to a file, use a redirection operator.

To send the current logs and then any updates that follow, use –follow with the redirection operator.

This command saves the output for future reference, but what if you want to keep the logs and view them at the same time? Opening another shell and tailing the output file feels like a hack. There must be a better way.

So, let’s use tee to watch the output and save it to a file at the same time.

We still see the container output. Now, open another shell and check the output file.

The file is being updated too! We have the output on the terminal, and tee saves it to the file we specified at the same time.

Redirecting Stdout/Stderr in the Container

Of course, there’s another way to save your container logs to a file. Instead of sending output to stderr and stdout, redirect your application’s output to a file and map the file to permanent storage outside of the container.

When to eschew the standard output completely is a judgment call that you should make when you take into account how much output your application generates, how often you think you’ll need to refer to it, and what kind of logging infrastructure is available to you. If, for example, you need to store events for compliance purposes, a logging framework like Java’s Logback may be a better option than capturing stdout and stderr.

That said, let’s modify the previous example to save the output to a file. We’ll change the loop to print the date to a file in /tmp. We’ll also run the container with /tmp mapped to the same directory of our host. Note that Docker will not accept relative paths on the command line, so if you want to use a different directory, you’ll need to use the complete path.

Take a look at /tmp/output.log on the host system.

Now, take a look at docker logs.

There’s nothing there. Since we redirected the output to a file, there is nothing for Docker to capture. Of course, we would need to capture both stdout and stderr in a production application.

Use the Method That Works for You

We covered several different ways to capture docker logs for saving and analysis. The best method is the one that works for you. In a development environment, the docker logs command is a powerful tool that works well with other command line tools. You can use Docker’s built-in tools to view, filter, and copy logs to a file.

In a production environment, log management should be part of an enterprise strategy. Scalyr’s log aggregation tools help you aggregate, process, search, and analyze your logs. Contact us, or sign up for a free trial to see how our tools make it easier to find what you need in the vast ocean of log entries produced by your growing application infrastructure.



Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Proudly powered by WordPress | Theme: Baskerville 2 by Anders Noren.

Up ↑

%d bloggers like this: