Newsyslog

• How newsyslog works
• How to configure
• Examples (rotating mysql and apache logs)

How newsyslog works

Logfiles has a tendency to grow, and they will eventually fill up your harddisk if nothing is done. But the utility Newsyslog can prevent this from happening. You can register any logfile in the configuration file of Newsyslog, and thereby tell Newsyslog how to manage each file.

In /var/log/ you'll find a number of logfiles. For example the mail logfile:

maillog
maillog.0.gz
maillog.1.gz
maillog.2.gz
maillog.3.gz
maillog.4.gz
maillog.5.gz
maillog.6.gz
maillog.7.gz

The file maillog is a logfile. The others are archive files created by newsyslog. In this example, it has been specified in the newsyslog configuration file that maillog must be turned over every night at midnight. It is also specified that the number of max archive files is 7. Last but not least, newsyslog has been told to compress the archive files.

Tonight at midnight the content of maillog will move to maillog.0.gz. The content of maillog.0.gz will move to maillog.1.gz and maillog.1.gz will move to maillog.2.gz and so on... Since the maximum number of archive files is 7, the content of the current maillog.7.gz will go straight to heaven...

But archiving logfiles don't have to take place at a specific time. It works well for frequently used logfiles which is growing rapidly. But why archive logfiles if they are empty? If a logfile is only rarely used, it gives more meaning to archive it depending on size.

How to configure newsyslog

Then configuration file resides in /etc/newsyslog.conf and looks something like this:

# logfilename          [owner:group] mode count size when [flag] [/pid_file] [sig_num]
/var/log/cron                         600  3     100  *     Z
/var/log/amd.log                      644  7     100  *     Z
/var/log/auth.log                     600  7     100  *     Z
/var/log/kerberos.log                 600  7     100  *     Z
/var/log/lpd-errs                     644  7     100  *     Z
/var/log/maillog                      640  7     *  @T00    Z
/var/log/sendmail.st                  640  10    *  168     B
/var/log/messages                     644  5     100  *     Z
/var/log/all.log                      600  7     *  @T00    Z
/var/log/slip.log       root:network  640  3     100  *     Z
/var/log/ppp.log        root:network  640  3     100  *     Z
/var/log/security                     600  10    100  *     Z
/var/log/wtmp                         644  3     *  @01T05  B
/var/log/daily.log                    640  7     *  @T00    Z
/var/log/weekly.log                   640  5     1  $W6D0   Z
/var/log/monthly.log                  640  12    *  $M1D0   Z
/var/log/console.log                  600  5     100  *     Z

Examples

Rotating MySQL log files

As you can see below I have two mysql logfiles which is currently not handled by newsyslog.

/var/log/mysql/
-rw-rw----  1 mysql  mysql     3027 Jul 20 19:10 mysql-err.log
-rw-rw----  1 mysql  mysql  2189789 Jul 28 20:53 mysql.log

The files are registeret in newsyslog.conf like this:

# logfilename                   [owner:group] mode count size when [flag]
/var/log/mysql/mysql.log          mysql:mysql   644  7    *    @T23  Z
/var/log/mysql/mysql-err.log      mysql:mysql   644  7    100  *     Z

Since the system frequently writes to mysql.log, I've chosen to archive it every night at 23:00. The error log however, hasn't been used the last eight days. It does not make sense to archive it every day, since it will often be empty. The archiving of mysql-err.log will therefore be based on size.

Every time cron runs newsyslog, the configuration file is read, so there's no need to restart anything. Just save your changes, and watch newsyslog go to work.

Rotating Apache log files

When rotating Apache log files, Apache must be restarted and we must also make sure that newsyslog does not leave a message in the new logfile. Fortunately newsyslog can do this automatically. The fields pid_file and sig_num in combination, takes care of the restart. And a B in the ZJB field tells newsyslog not to include the 'turned over' message.

Due to the limited with of this page, I have left out 'mode', 'count', 'size' and 'when' in this example:

# logfilename            [owner:group] [flag] [/pid_file]           [sig_num]
/path/to/log/access-log    640           B    /var/run/httpd.pid       30 

So why is this nessecary? When newsyslog rotates the log, the inode of the file changes, but Apache still tries to write to the old inode. Therefore we tell newsyslog to make a graceful restart (signal number 30). A quote from the Apache documentation:

httpd.apache.org/docs/2.0/stopping.html
The USR1 or graceful signal causes the parent process to advise the children to exit after their current request (or to exit immediately if they're not serving anything). The parent re-reads its configuration files and re-opens its log files. As each child dies off the parent replaces it with a child from the new generation of the configuration, which begins serving new requests immediately.

So this is a neat way of restarting Apache without the website users ever knowing it.

Leaving out the 'turned over' message with the B-flag, makes sure that readings from statistics applications (e.g. webalizer) does not get broken. This is also the reason that I have chosen not to compress logfiles with the Z-flag.