How To Manage Logfiles with Logrotate on Ubuntu 22.04

Introduction

Logrotate is a system utility that manages the automatic rotation and compression of log files. If log files were not rotated, compressed, and periodically pruned, they could eventually consume all available disk space on a system.

Logrotate is installed by default on Ubuntu 22.04, and is set up to handle the log rotation needs of all installed packages, including rsyslog, the default system log processor.

In this article, we will explore the default Logrotate configuration, then configure log rotation for a fictional custom application.

Prerequisites

This tutorial assumes you have an Ubuntu 22.04 server, with a non-root sudo-enabled user, as described in Initial Server Setup with Ubuntu 22.04.

Logrotate is available on many other Linux distributions as well, but the default configuration may be quite different. Other sections of this tutorial will still apply as long as your version of Logrotate is similar to Ubuntu 22.04’s. Follow Step 1 to determine your Logrotate version.

Log into your server as your sudo-enabled user to begin.

Step 1 — Confirming Your Logrotate Version

Logrotate is installed by default on Ubuntu. However, if you need to install it, run the following commands to update your package list and retrieve the package:

sudo apt update
sudo apt install logrotate

If you’re using a non-Ubuntu server, first make sure Logrotate is installed by asking for its version information:

logrotate --version

Output
logrotate 3.19.0

    Default mail command:       /usr/bin/mail
    Default compress command:   /bin/gzip
    Default uncompress command: /bin/gunzip
    Default compress extension: .gz
    Default state file path:    /var/lib/logrotate/status
    ACL support:                yes
    SELinux support:            yes

If Logrotate is installed but the version number is significantly different, you may have issues with some of the configuration options that are explored in this tutorial. Refer to the documentation for your specific version of Logrotate by reading its manual (man) page:

man logrotate

You can also consult the online version of the Logrotate documentation. Next we’ll look at Logrotate’s default configuration structure on Ubuntu.

Step 2 — Exploring the Logrotate Configuration

Logrotate’s configuration information can generally be found in two places on Ubuntu:

• /etc/logrotate.conf: this file contains some default settings and sets up rotation for a few logs that are not owned by any system packages. It also uses an include statement to pull in configuration from any file in the /etc/logrotate.d directory.
• /etc/logrotate.d/: this is where any packages you install that need help with log rotation will place their Logrotate configuration. On a standard install you should already have files here for core system tools like apt, dpkg, rsyslog and so on.

By default, logrotate.conf will configure weekly log rotations, with log files owned by the root user and the syslog group, with four log files being retained at a time(rotate 4), and new empty log files being created after the current one is rotated (create).

Let’s take a look at a package’s Logrotate configuration file in /etc/logrotate.d. cat the file for the apt package utility:

cat /etc/logrotate.d/apt

Output
/var/log/apt/term.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

/var/log/apt/history.log {
  rotate 12
  monthly
  compress
  missingok
  notifempty
}

This file contains configuration blocks for two different log files in the /var/log/apt/ directory: term.log and history.log. They both have the same options. Any options not set in these configuration blocks will inherit the default values or those set in /etc/logrotate.conf. Any setting in a logrotate file will override logrotate’s default values, which are configured in /etc/logrotate.conf. The options set for the apt logs are:

• rotate 12: keep twelve old log files. This overrides the rotate 4 default.
• monthly: rotate once a month. This overrides the weekly default.
• compress: compress the rotated files. this uses gzip by default and results in files ending in .gz. The compression command can be changed using the compresscmd option.
• missingok: don’t write an error message if the log file is missing.
• notifempty: don’t rotate the log file if it is empty.

These configuration files also inherit the default create behaviour, which instructs Logrotate to create new logs after rotation. This could be overridden with nocreate, although that would effectively disable most of the other functionality.

There are many more configuration options available. You can read about all of them by typing man logrotate on the command line to bring up Logrotate’s manual page.

Next, we’ll set up a configuration file to handle logs for a fictional service called your-app.

Step 3 — Setting Up an Example Config

To manage log files using logrotate for applications outside of the pre-packaged and pre-configured system services, we have two options:

1. Create a new Logrotate configuration file and place it in /etc/logrotate.d/. This will be run daily as the root user along with all the other standard Logrotate jobs.
2. Create a new configuration file and run it outside of Ubuntu’s default Logrotate setup. This is only really necessary if you need to run Logrotate as a non-root user, or if you want to rotate logs more frequently than daily (an hourly configuration in /etc/logrotate.d/ would be ineffective, because the system’s Logrotate setup only runs once a day).

Let’s walk through these two options with some example setups.

Adding Configuration to /etc/logrotate.d/

We want to configure log rotation for a fictional web server that puts an access.log and error.log into /var/log/your-app/. It runs as the www-data user and group.

To add a configuration for the your-app log files to /etc/logrotate.d/, first open up a new file in the /etc/logrotate.d directory using nano or your preferred editor:

sudo nano /etc/logrotate.d/your-app

Add the following lines to your new configuration file:

/var/log/your-app/*.log {
	daily
	missingok
	rotate 14
	compress
	notifempty
	create 0640 www-data www-data
	sharedscripts
	postrotate
		systemctl reload your-app
	endscript
}

Some of the new configuration directives in this file are:

• create 0640 www-data www-data: this creates a new empty log file after rotation, with the specified permissions (0640), owner (www-data), and group (also www-data).
• sharedscripts: this flag means that any scripts added to the configuration are run only once per run, instead of for each file rotated. Since the path /var/log/your-app/*.log includes a wildcard *, this configuration would match any number of log files in the your-app directory. Without the sharedscripts option, the script specified in postrotate would run each time logrotate processes a log file without this option.
• postrotate to endscript: this block contains a script to run after the log file is rotated. In this case we’re reloading our example app. This is sometimes necessary to get your application to switch over to the newly created log file. Note that postrotate runs before logs are compressed. Compression could take a long time, and your software should switch to the new log file immediately. For tasks that need to run after logs are compressed, use the lastaction block instead.

To save and quit nano, press Ctrl+X, and when prompted, Y and then Enter. You can test the config file by doing a dry run:

sudo logrotate /etc/logrotate.conf --debug

This command calls logrotate, points it to the standard configuration file, and turns on debug mode.

Information will print out about which log files Logrotate is handling and what it would have done to them. If all looks well, you’re done. The standard Logrotate job will run once a day and include your new configuration.

Next, we’ll try a setup that doesn’t use Ubuntu’s default configuration at all.

Step 4 — Creating an Independent Logrotate Configuration

In this example we have an app running as our user sammy, generating logs that are stored in /home/sammy/logs/. We want to rotate these logs hourly, so we need to set this up outside of the /etc/logrotate.d structure provided by Ubuntu.

First, we’ll create a configuration file in our home directory. Open it in a text editor:

nano /home/sammy/logrotate.conf

Then paste in the following configuration:

/home/sammy/logs/*.log {
	hourly
	missingok
	rotate 24
	compress
	create
}

Save and close the file. We’ve encountered all these options in previous steps, but let’s summarize: this configuration will rotate the files hourly, compressing and keeping twenty-four old logs and creating a new log file to replace the rotated one.

You’ll need to customize the configuration to suit your application, but this is a good start.

To test that the configuration works, let’s make a log file. First cd to your user’s home directory using the cd ~ command. Then create a directory for the logs using the mkdir command. Finally create an empty file in the logs directory using the touch command. Run the following commands in order to complete these steps:

cd ~
mkdir logs
touch logs/access.log

Now that we have a blank log file in the right spot, let’s run the logrotate command.

Because the logs are owned by sammy we don’t need to use sudo. However, we do need to specify a state file. This file records what logrotate found and any actions that it took the last time it ran, so that it knows what to do the next time it runs. This state tracking is handled for us when using the default /etc/logrotate.conf configuration. The state file is stored in /var/lib/logrotate/status. Since we are not using the default configuration, we’ll need to configure the state file location manually.

We’ll have Logrotate put the state file right in our home directory for this example. It can go anywhere that’s accessible and convenient. Run the following command to use the /home/sammy/logrotate.conf that you created and to record the state that logrotate encounters:

logrotate /home/sammy/logrotate.conf --state /home/sammy/logrotate-state --verbose

Output
reading config file /home/sammy/logrotate.conf

Handling 1 logs

rotating pattern: /home/sammy/logs/*.log  hourly (24 rotations)
empty log files are rotated, old logs are removed
considering log /home/sammy/logs/access.log
  log does not need rotating

The --verbose flag will print out detailed information about what Logrotate is doing. In this case it did not rotate anything. This is Logrotate’s first time encountering this log file and it is zero hours old so shouldn’t be rotated.

If we examine at the state file using the cat utility, we’ll note that Logrotate recorded some information about the run:

cat /home/sammy/logrotate-state

Output
logrotate state -- version 2
"/home/sammy/logs/access.log" 2022-07-2-19:0:0

Logrotate noted the logs that it saw and when it last considered them for rotation. If we run this same command one hour later, the log will be rotated as expected.

If you want to force Logrotate to rotate the log file when it otherwise would not have, use the --force flag:

logrotate /home/sammy/logrotate.conf --state /home/sammy/logrotate-state --verbose --force

This is useful when testing postrotate and other scripts.

Finally, we need to set up a cron job to run Logrotate every hour. Open your user’s crontab:

crontab -e

This will open a text file. If it is your first time using cron, you may be prompted to choose a default text editor. If you do not have a preference, we recommend nano for new users. There may be some comments already in the file that explain the cron syntax. Move the cursor down to a new blank line at the end of the file and add the following:

crontab
14 * * * * /usr/sbin/logrotate /home/sammy/logrotate.conf --state /home/sammy/logrotate-state

This task will run on the 14th minute of every hour, every day. It runs nearly the same logrotate command we ran previously, though we expanded logrotate to its full path of /usr/sbin/logrotate to be safe. It’s good practice to use full paths when writing cron jobs. To learn more about cron, you can review our other tutorials.

Save the file and exit. This will install the crontab and our task will run on the specified schedule.

If we revisit our log directory in about an hour we should find the rotated and compressed log file access.log.1.gz (or .2.gz if you ran Logrotate with the --force flag).

Conclusion

In this tutorial we verified our Logrotate version, explored the default Ubuntu Logrotate configuration, and set up two different types of custom configurations. To learn more about the command line and configuration options available for Logrotate, you can read its manual page by running man logrotate in your terminal or by visiting the online documentation.