Linux Commands Examples

A great documentation place for Linux commands

logrotate

rotates, compresses, and mails system logs


see also : gzip

Synopsis

logrotate [-dv] [-f|--force] [-s|--state file] config_file ..


add an example, a script, a trick and tips

: email address (won't be displayed)
: name

Step 2

Thanks for this example ! - It will be moderated and published shortly.

Feel free to post other examples
Oops ! There is a tiny cockup. A damn 404 cockup. Please contact the loosy team who maintains and develops this wonderful site by clicking in the mighty feedback button on the side of the page. Say what happened. Thanks!

examples

1
source
            
sudo cp islandora_logrotate /etc/logrotate.d/islandora_logrotate
sudo chmod 644 /etc/logrotate.d/islandora_logrotate
1
source

How do I efficiently rename and compress old logs after turning on compression in logrotate?

As far as I know, logrotate cannot put old files to their place. So you need to do it by hand/by a small script.

Please notice, that depending on your configuration, logrotate rotates old log file only for limited times and then deletes them. So restoring old backup files may be useless until you changed your logrotate configuration so that it never deletes old logs (configuration variable rotate).

1
source

Capture logrotated files on change

You should use inoticoming. It is in the repos for Debian-family distros. If you are on another distro, you can probably compile from source.

SYNOPSIS

inoticoming [ global-options ] directory actions*

DESCRIPTION

Inoticoming is a daemon to watch a directory with Linux's inotify framework and trigger actions once files with specific names are placed in there.

You can use it as follows:

   inotcoming --initialsearch /directory/with/log/files --regexp \
   last_log_rotate_file10.txt mv {} /directory/to/store/log_rotate_files/`date +"%T"` \;

The action must be terminated by \;;

the option --initialsearch performs the following action as soon as the daemon starts;

the --regexp precedes a regular expression to be satisfied by the name of the file you wish to store elsewhere; here I assumed you know that the 10th log_rotate_file is the last one that will be kept by your system;

then the command follows, with {} identifying the file name which was matched by the --regexp option, and I have assumed you will want to move the file to a new directory, with a name reflecting the time of the operation. You can adjust this at will.

0
source

logrotate configuration file syntax - multiple wildcard entries possible?

Yes, you can use multiple wild cards. You can test your file without performing the actual rotations by doing this:

logrotate -d -f /etc/logrotate.conf

  • -d = Turns on debug mode. In debug mode, no changes will be made to the logs or to the logrotate state file.

  • -f = Tells logrotate to force the rotation, even if it doesn’t think this is necessary. Sometimes this is useful after adding new entries to logrotate, or if old log files have been removed by hand, as the new files will be created, and logging will con- tinue correctly.`

0
source

Log rotation of stdout?

the rotatelogs tool shipped with apache (in the bin dir) (see docs) takes input from stdin and rotates the log after some specific amount of time

0
source

logrotate configuration and execution

  1. Do I need to specify daily if I'm specifying a maxage of 1?

    Yes, you need daily as you want this section to execute everyday. This control how execution frequency of the section, while maxage control how long rotated files are kept. They are 2 different things.

  2. What exactly is missingok doing?

    missingok means logrotate will not complain/generate error if the log file for rotation does not exist. If this is not specified and the log file intended for rotation is missing, logrotate will generate an error log.

  3. Do I even need copytruncate if I'm specifying daily/maxage 1?

    If copytruncate is working for you now, don't change it. It apply to original log file

    copytruncate instruct logrotate to creates the copy of the original file (i.e rotate the original log file) and truncates the original file to zero byte size. This helps the respective service that belongs to that log file can write to the proper file.

  4. To run logrotate at mid-night

    There are 2 methods to do it.

    • Method 1 - Change when daily cron is run

      Look for following lines in /etc/crontab

      25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
      
      

      Change 25 6 (means 6:25AM), to 0 0. This also change ALL daily cron job starting time to mid-night.

    • Method 2 - Use custom crontab line

      Move logrotate out of default daily cron schedule

      mv /etc/cron.daily/logrotate /etc/logrotate.cronjob
      
      

      Create custom cron job. Add following line into /etc/crontab

      0 0 * * * root /etc/logrotate.cronjob
      
      
0
source

Logrotate Error

You're going to need to get more information.

Does the file exist, and does the user running cron have permission to read it?:

ls -l /var/log/mylog.1

See if logrotate can give more information. It should have a verbose flag you can turn on with '-v'.

0
source

Logrotate append to existing file and truncate original

From the logrotate man page:

 prerotate/endscript
          The  lines  between  prerotate and endscript (both of which must
          appear on lines by  themselves)  are  executed  (using  /bin/sh)
          before the log file is rotated and only if the log will actually
          be rotated. These directives may only appear inside a  log  file
          definition.  Normally,  the  absolute  path  to  the log file is
          passed as first argument to the script.   If   sharedscripts  is
          specified,  whole  pattern  is  passed  to the script.  See also
          postrotate.  See sharedscripts  and  nosharedscripts  for  error
          handling.

So, you should add an entry like this in logrotate.conf:

/var/log/vnc.log 
{
  rotate 6
  monthly
  compress
  missingok
  prerotate
      cat /var/log/vnc.log >> /archive/logs/vnc.log
  endscript
}

0
source

linux logrotate how to set rules for all log files in an directory(including files in sub-directory)

Yes the logrotate configuration which you have provided will work.

0
source

Log rotation with automatic *.log file discovery

Here is a simple Python script for performing auto-discovery and generating logrotate conf file:

0
source

Will cron.daily wait for jobs to finish before starting the next one?

I just created test cron job which executes every minute:

#!/bin/bash

echo "123" >> /home/user/test-file

sleep 100

echo "234" >> /home/user/test-file

So I did see overlap in htop. I mean there was periods of time when I saw two processes running simultaneously. 123 was echoed at the begging of the minute and 234 was echoed at 40 seconds of next minute (because of sleep 100). In period of 40 seconds to beginning of next minute there was just one process.

That means cron fires jobs and does not wait for them to finish.

I'll try to represent it graphically:

0 minute 00 seconds 123 <----first execution
1 minute 00 seconds 123 <----second execution
1 minute 40 seconds 234 <----first execution
2 minute 00 seconds 123 <----third execution
2 minute 40 seconds 234 <----second execution
3 minute 00 seconds 123 <----fourth execution
3 minute 40 seconds 234 <----third execution

At least this is true for jobs in the /etc/crontab with vixie-cron on fresh updated Gentoo.

As Paul discovered - stuff in cron.daily executes in the different way.

0
source

Logrotate not doing any rotation

I'm not sure if this is the problem but if Apache is still running it might have a lock on those log-files. Maybe the /usr/bin/killall -HUP httpd does not kill Apache quick enough.

Try shutting Apache down first and see if that helps:

service httpd stop
logrotate -f /etc/logrotate.d/apache
service httpd start

If you don't start Apache with service you need to use the appropriate command.

BTW. my /etc/logrotate.d/httpd (which is your /etc/logrotate.d/apache) looks like this:

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /bin/kill -USR1 `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
    endscript
}

0
source

Logrotate says "log needs rotating" and then does nothing

A few hints in order of importance:

-) logrotate configuration file seems ok but is there somebody calling logrotate? This is usually done by cron or anacron. Check this out since logrotate needs somebody to call it indeed (usually cron/anacron) otherwise configuring it is useless. Check in your cron/anacron configurations and check that the daemon is active.

-) hourly doesn't look to be a valid configuration option at least in my version, check this out to be sure

-) when you run it manually with the "-d" no changes will be made to files so for that run it's correct that nothing happened. So to make sure at least the configuration is 100% fine try it without -d. And eventually with -f to force the rotation so you can check for eventual permission or such troubles.

description

logrotate is designed to ease administration of systems that generate large numbers of log files. It allows automatic rotation, compression, removal, and mailing of log files. Each log file may be handled daily, weekly, monthly, or when it grows too large.

Normally, logrotate is run as a daily cron job. It will not modify a log more than once in one day unless the criterion for that log is based on the log’s size and logrotate is being run more than once each day, or unless the -f or --force option is used.

Any number of config files may be given on the command line. Later config files may override the options given in earlier files, so the order in which the logrotate config files are listed is important. Normally, a single config file which includes any other config files which are needed should be used. See below for more information on how to use the include directive to accomplish this. If a directory is given on the command line, every file in that directory is used as a config file.

If no command line arguments are given, logrotate will print version and copyright information, along with a short usage summary. If any errors occur while rotating logs, logrotate will exit with non-zero status.

options

-?, --help

Prints help message.

-d, --debug

Turns on debug mode and implies -v. In debug mode, no changes will be made to the logs or to the logrotate state file.

-f, --force

Tells logrotate to force the rotation, even if it doesn’t think this is necessary. Sometimes this is useful after adding new entries to a logrotate config file, or if old log files have been removed by hand, as the new files will be created, and logging will continue correctly.

-m, --mail <command>

Tells logrotate which command to use when mailing logs. This command should accept two arguments: 1) the subject of the message, and 2) the recipient. The command must then read a message on standard input and mail it to the recipient. The default mail command is /usr/bin/mail -s.

-s, --state <statefile>

Tells logrotate to use an alternate state file. This is useful if logrotate is being run as a different user for various sets of log files. The default state file is /var/lib/logrotate/status.

--usage

Prints a short usage message.

-v, --verbose

Turns on verbose mode, ie. display messages during rotation.

configuration file

logrotate reads everything about the log files it should be handling from the series of configuration files specified on the command line. Each configuration file can set global options (local definitions override global ones, and later definitions override earlier ones) and specify logfiles to rotate. A simple configuration file looks like this:

# sample logrotate configuration file
compress

/var/log/messages {
rotate 5
weekly
postrotate
/usr/bin/killall -HUP syslogd
endscript
}

"/var/log/httpd/access.log" /var/log/httpd/error.log {
rotate 5
mail www[:at:]my[:dot:]org
size 100k
sharedscripts
postrotate
/usr/bin/killall -HUP httpd
endscript
}

/var/log/news/* {
monthly
rotate 2
olddir /var/log/news/old
missingok
postrotate
kill -HUP ’cat /var/run/inn.pid’
endscript
nocompress
}

~/log/*.log {}

The first few lines set global options; in the example, logs are compressed after they are rotated. Note that comments may appear anywhere in the config file as long as the first non-whitespace character on the line is a #.

The next section of the config file defines how to handle the log file /var/log/messages. The log will go through five weekly rotations before being removed. After the log file has been rotated (but before the old version of the log has been compressed), the command /sbin/killall -HUP syslogd will be executed.

The next section defines the parameters for both /var/log/httpd/access.log and /var/log/httpd/error.log. Each is rotated whenever it grows over 100k in size, and the old logs files are mailed (uncompressed) to www[:at:]my[:dot:]org after going through 5 rotations, rather than being removed. The sharedscripts means that the postrotate script will only be run once (after the old logs have been compressed), not once for each log which is rotated. Note that log file names may be enclosed in quotes (and that quotes are required if the name contains spaces). Normal shell quoting rules apply, with , ", and \ characters supported.

The next section defines the parameters for all of the files in /var/log/news. Each file is rotated on a monthly basis. This is considered a single rotation directive and if errors occur for more than one file, the log files are not compressed.

The last section uses tilde expansion to rotate log files in the home directory of the current user. This is only available, if your glob library supports tilde expansion. GNU glob does support this.

Please use wildcards with caution. If you specify *, logrotate will rotate all files, including previously rotated ones. A way around this is to use the olddir directive or a more exact wildcard (such as *.log).

If the directory /var/log/news does not exist, this will cause logrotate to report an error. This error cannot be stopped with the missingok directive.

Here is more information on the directives which may be included in a logrotate configuration file:
compress

Old versions of log files are compressed with gzip(1) by default. See also nocompress.

compresscmd

Specifies which command to use to compress log files. The default is gzip(1). See also compress.

uncompresscmd

Specifies which command to use to uncompress log files. The default is gunzip(1).

compressext

Specifies which extension to use on compressed logfiles, if compression is enabled. The default follows that of the configured compression command.

compressoptions

Command line options may be passed to the compression program, if one is in use. The default, for gzip(1), is "-6" (biased towards high compression at the expense of speed). If you use a different compression command, you may need to change the compressoptions to match.

copy

Make a copy of the log file, but don’t change the original at all. This option can be used, for instance, to make a snapshot of the current log file, or when some other utility needs to truncate or parse the file. When this option is used, the create option will have no effect, as the old log file stays in place.

copytruncate

Truncate the original log file to zero size in place after creating a copy, instead of moving the old log file and optionally creating a new one. It can be used when some program cannot be told to close its logfile and thus might continue writing (appending) to the previous log file forever. Note that there is a very small time slice between copying the file and truncating it, so some logging data might be lost. When this option is used, the create option will have no effect, as the old log file stays in place.

create mode owner group, create owner group

Immediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated). mode specifies the mode for the log file in octal (the same as chmod(2)), owner specifies the user name who will own the log file, and group specifies the group the log file will belong to. Any of the log file attributes may be omitted, in which case those attributes for the new file will use the same values as the original log file for the omitted attributes. This option can be disabled using the nocreate option.

daily

Log files are rotated every day.

dateext

Archive old versions of log files adding a date extension like YYYYMMDD instead of simply adding a number. The extension may be configured using the dateformat and dateyesterday options.

dateformat format_string

Specify the extension for dateext using the notation similar to strftime(3) function. Only %Y %m %d and %s specifiers are allowed. The default value is -%Y%m%d. Note that also the character separating log name from the extension is part of the dateformat string. The system clock must be set past Sep 9th 2001 for %s to work correctly. Note that the datestamps generated by this format must be lexically sortable (i.e., first the year, then the month then the day. e.g., 2001/12/01 is ok, but 01/12/2001 is not, since 01/11/2002 would sort lower while it is later). This is because when using the rotate option, logrotate sorts all rotated filenames to find out which logfiles are older and should be removed.

dateyesterday

Use yesterday’s instead of today’s date to create the dateext extension, so that the rotated log file has a date in its name that is the same as the timestamps within it.

delaycompress

Postpone compression of the previous log file to the next rotation cycle. This only has effect when used in combination with compress. It can be used when some program cannot be told to close its logfile and thus might continue writing to the previous log file for some time.

extension ext

Log files with ext extension can keep it after the rotation. If compression is used, the compression extension (normally .gz) appears after ext. For example you have a logfile named mylog.foo and want to rotate it to mylog.1.foo.gz instead of mylog.foo.1.gz.

ifempty

Rotate the log file even if it is empty, overriding the notifempty option (ifempty is the default).

include file_or_directory

Reads the file given as an argument as if it was included inline where the include directive appears. If a directory is given, most of the files in that directory are read in alphabetic order before processing of the including file continues. The only files which are ignored are files which are not regular files (such as directories and named pipes) and files whose names end with one of the taboo extensions, as specified by the tabooext directive.

mail address

When a log is rotated out of existence, it is mailed to address. If no mail should be generated by a particular log, the nomail directive may be used.

mailfirst

When using the mail command, mail the just-rotated file, instead of the about-to-expire file.

maillast

When using the mail command, mail the about-to-expire file, instead of the just-rotated file (this is the default).

maxage count

Remove rotated logs older than <count> days. The age is only checked if the logfile is to be rotated. The files are mailed to the configured address if maillast and mail are configured.

maxsize size

Log files are rotated when they grow bigger than size bytes even before the additionally specified time interval (daily, weekly, monthly, or yearly). The related size option is similar except that it is mutually exclusive with the time interval options, and it causes log files to be rotated without regard for the last rotation time. When maxsize is used, both the size and timestamp of a log file are considered.

minsize size

Log files are rotated when they grow bigger than size bytes, but not before the additionally specified time interval (daily, weekly, monthly, or yearly). The related size option is similar except that it is mutually exclusive with the time interval options, and it causes log files to be rotated without regard for the last rotation time. When minsize is used, both the size and timestamp of a log file are considered.

missingok

If the log file is missing, go on to the next one without issuing an error message. See also nomissingok.

monthly

Log files are rotated the first time logrotate is run in a month (this is normally on the first day of the month).

nocompress

Old versions of log files are not compressed. See also compress.

nocopy

Do not copy the original log file and leave it in place. (this overrides the copy option).

nocopytruncate

Do not truncate the original log file in place after creating a copy (this overrides the copytruncate option).

nocreate

New log files are not created (this overrides the create option).

nodelaycompress

Do not postpone compression of the previous log file to the next rotation cycle (this overrides the delaycompress option).

nodateext

Do not archive old versions of log files with date extension (this overrides the dateext option).

nomail

Do not mail old log files to any address.

nomissingok

If a log file does not exist, issue an error. This is the default.

noolddir

Logs are rotated in the directory they normally reside in (this overrides the olddir option).

nosharedscripts

Run prerotate and postrotate scripts for every log file which is rotated (this is the default, and overrides the sharedscripts option). The absolute path to the log file is passed as first argument to the script. If the scripts exit with error, the remaining actions will not be executed for the affected log only.

noshred

Do not use shred when deleting old log files. See also shred.

notifempty

Do not rotate the log if it is empty (this overrides the ifempty option).

olddir directory

Logs are moved into directory for rotation. The directory must be on the same physical device as the log file being rotated, and is assumed to be relative to the directory holding the log file unless an absolute path name is specified. When this option is used all old versions of the log end up in directory. This option may be overridden by the noolddir option.

postrotate/endscript

The lines between postrotate and endscript (both of which must appear on lines by themselves) are executed (using /bin/sh) after the log file is rotated. These directives may only appear inside a log file definition. Normally, the absolute path to the log file is passed as first argument to the script. If sharedscripts is specified, whole pattern is passed to the script. See also prerotate. See sharedscripts and nosharedscripts for error handling.

prerotate/endscript

The lines between prerotate and endscript (both of which must appear on lines by themselves) are executed (using /bin/sh) before the log file is rotated and only if the log will actually be rotated. These directives may only appear inside a log file definition. Normally, the absolute path to the log file is passed as first argument to the script. If sharedscripts is specified, whole pattern is passed to the script. See also postrotate. See sharedscripts and nosharedscripts for error handling.

firstaction/endscript

The lines between firstaction and endscript (both of which must appear on lines by themselves) are executed (using /bin/sh) once before all log files that match the wildcarded pattern are rotated, before prerotate script is run and only if at least one log will actually be rotated. These directives may only appear inside a log file definition. Whole pattern is passed to the script as first argument. If the script exits with error, no further processing is done. See also lastaction.

lastaction/endscript

The lines between lastaction and endscript (both of which must appear on lines by themselves) are executed (using /bin/sh) once after all log files that match the wildcarded pattern are rotated, after postrotate script is run and only if at least one log is rotated. These directives may only appear inside a log file definition. Whole pattern is passed to the script as first argument. If the script exits with error, just an error message is shown (as this is the last action). See also firstaction.

rotate count

Log files are rotated count times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather than rotated.

size size

Log files are rotated only if they grow bigger then size bytes. If size is followed by k, the size is assumed to be in kilobytes. If the M is used, the size is in megabytes, and if G is used, the size is in gigabytes. So size 100, size 100k, size 100M and size 100G are all valid.

sharedscripts

Normally, prerotate and postrotate scripts are run for each log which is rotated and the absolute path to the log file is passed as first argument to the script. That means a single script may be run multiple times for log file entries which match multiple files (such as the /var/log/news/* example). If sharedscripts is specified, the scripts are only run once, no matter how many logs match the wildcarded pattern, and whole pattern is passed to them. However, if none of the logs in the pattern require rotating, the scripts will not be run at all. If the scripts exit with error, the remaining actions will not be executed for any logs. This option overrides the nosharedscripts option and implies create option.

shred

Delete log files using shred -u instead of unlink(). This should ensure that logs are not readable after their scheduled deletion; this is off by default. See also noshred.

shredcycles count

Asks GNU shred(1) to overwrite log files count times before deletion. Without this option, shred’s default will be used.

start count

This is the number to use as the base for rotation. For example, if you specify 0, the logs will be created with a .0 extension as they are rotated from the original log files. If you specify 9, log files will be created with a .9, skipping 0-8. Files will still be rotated the number of times specified with the rotate directive.

su user group

Rotate log files set under this user and group instead of using default user/group (usually root). user specifies the user name used for rotation and group specifies the group used for rotation.

tabooext [+] list

The current taboo extension list is changed (see the include directive for information on the taboo extensions). If a + precedes the list of extensions, the current taboo extension list is augmented, otherwise it is replaced. At startup, the taboo extension list contains .rpmsave, .rpmorig, ~, .disabled, .dpkg-old, .dpkg-dist, .dpkg-new, .cfsaved, .ucf-old, .ucf-dist, .ucf-new, .rpmnew, .swp, .cfsaved, .rhn-cfg-tmp-*

weekly

Log files are rotated if the current weekday is less than the weekday of the last rotation or if more than a week has passed since the last rotation. This is normally the same as rotating logs on the first day of the week, but it works better if logrotate is not run every night.

yearly

Log files are rotated if the current year is not the same as the last rotation.

files

/var/lib/logrotate.status

Default state file.

/etc/logrotate.conf

Configuration options.

notes

The killall(1) program in Debian is found in the psmisc package.


see also

gzip


authors

Erik Troan, Preston Brown, Jan Kaluza.

<logrotate-owner[:at:]fedoraproject[:dot:]org>
<http://fedorahosted.org/logrotate/>

Corrections and changes for Debian by Paul Martin <pm[:at:]debian[:dot:]org>

How can this site be more helpful to YOU ?


give  feedback