Table of contents
Introduction
http://www.rsyslog.com/doc/manual.html is an enhanced syslogd supporting, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part. It is quite compatible to stock sysklogd and can be used as a drop-in replacement. Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user.
Installation
Rsyslog is generally included in the CentOS and Redhat repositories and all you need to do is:
$ yum install rsyslog
This should not do anything to your current (usually syslog/sysklogd) system. It just installs rsyslog in parallel (and it’s up to you to turn it on). Everything is put in place for you (startup scripts, config file that is a mirror of syslog.conf, integration with log rotation, etc.) and you just have to:
chkconfig syslog off chkconfig rsyslog on service syslog stop service rsyslog start
Configuration
Rsyslog Server
Example setup
A syslog system user was created with the options as set below:
adduser --system --no-create-home -s /sbin/nologin -b /var/log/remote-syslog -c 'Syslog system user for logging' syslog
The directory {{/var/log/remote-syslog}} is a mounted LVM and not located on the main file system.
This is the configuration as a centralised rsyslog server where all rsyslog clients can write their log files to:
# rsyslog v5 configuration file # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html #### MODULES #### $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # provides kernel logging support (previously done by rklogd) # Provides TCP syslog reception $ModLoad imtcp ###################### # Directives ###################### # Set the default permissions for all log files. $FileOwner syslog $FileGroup syslog $FileCreateMode 0644 $DirCreateMode 0755 #### GLOBAL DIRECTIVES #### # Use default timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # Include all config files in /etc/rsyslog.d/ $IncludeConfig /etc/rsyslog.d/*.conf ######## TEMPLATES ######## $template RemoteLog,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/%app-name%.log" $template image-squid-access,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/squid/image/access.log" $template image-squid-cache,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/squid/image/cache.log" $template image-squid-store,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/squid/image/store.log" $template squidlog,"%msg%\n" #### RULESETS #### ### Local Logging ### $RuleSet local # Log all kernel messages to the console. #kern.* /dev/console # Log anything (except mail) of level info or higher. Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save boot messages also to boot.log local7.* /var/log/boot.log $DefaultRuleSet local ### Remote Ruleset ### $RuleSet remote local0.* ?image-squid-access;squidlog local1.* ?image-squid-cache;squidlog local2.* ?image-squid-store;squidlog *.* ?RemoteLog #### LISTENERS #### #debug2: channel 0: window 999407 sent adjust 49169 $InputTCPServerBindRuleset remote $InputTCPServerRun 10514
Rsyslog Client
The directory {{/var/log/rsyslog}} has to exist for rsyslog to put spool files.
This the configuration as a replacement for syslog with duplicating of log files to a remote syslog server:
# rsyslog v5 configuration file # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html #### MODULES #### $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad imfile $ModLoad immark # provides --MARK-- message capability #### GLOBAL DIRECTIVES #### # Use default timestamp format ####### Templates ######### $template squidlog,"%msg%\n" # Include all config files in /etc/rsyslog.d/ $IncludeConfig /etc/rsyslog.d/*.conf #### RULES #### ### Local Logging ### # System specific logs use local log files # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console *.info;mail.none;authpriv.none;cron.none /var/log/messages;RSYSLOG_TraditionalFileFormat authpriv.* /var/log/secure;RSYSLOG_TraditionalFileFormat mail.* -/var/log/maillog;RSYSLOG_TraditionalFileFormat cron.* /var/log/cron *.emerg * local7.* /var/log/boot.log local0.* /var/log/image/access.log;squidlog local1.* /var/log/image/cache.log;squidlog local2.* /var/log/image/store.log;squidlog *.* -/var/log/ ### Remote Ruleset ### # ### begin forwarding rule ### # The statement between the begin ... end define a SINGLE forwarding # rule. They belong together, do NOT split them. If you create multiple # forwarding rules, duplicate the whole block! # Remote Logging (we use TCP for reliable delivery) # # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. $WorkDirectory /var/log/rsyslog # where to place spool files $ActionQueueFileName fwdRule1 # unique name prefix for spool files $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueType LinkedList # run asynchronously $ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional local0.* @@10.157.51.183:10514;squidlog local1.* @@10.157.51.183:10514;squidlog local2.* @@10.157.51.183:10514;squidlog *.* @@10.157.51.183:10514 # ### end of the forwarding rule ###
Advanced Configuration
For advanced configuration options, please visit the rsyslog http://wiki.rsyslog.com or the http://www.rsyslog.com/doc/manual.html
Filters
Beyond using simple facility.severity action, a more advanced set of filters can be used that take information from the message. Properties like the program name, process ID or some defined tag. For example,
:programname, isequal, "named" /var/log/named/named.log
would be a configuration directive to redirect logs coming from the named process and filter then out to the log mentioned.
Available properties can be found in the “Available Properties” section of the rsyslog.conf man page and the comparison operations can be found under the “Property-Based Filters” section
Filter conditionals
more advanced configuration can be done using conditional statements. For example,
if $programname == 'prog1' then {
action(type="omfile" file="/var/log/prog1.log")
if $msg contains 'test' then
action(type="omfile" file="/var/log/prog1test.log")
else
action(type="omfile" file="/var/log/prog1notest.log")
}
if $fromhost == 'host1' then {
mail.* action(type="omfile" file="/var/log/host1/mail.log")
*.err /var/log/host1/errlog # this is also still valid
#
# more "old-style rules" ...
#
} else {
mail.* action(type="omfile" file="/var/log/mail.log")
*.err /var/log/errlog
#
# more "old-style rules" ...
#
}
Examples
configuration of Squid Proxy to use Rsyslog
If one application uses more than one output to syslog;
e.g. Squid proxy with access.log, store.log and cache.log
You have to assign them individual http://en.wikipedia.org/wiki/Syslog#Facility_Levels to make sure they get written to the correct output files on the remote server.
Example excerpt from squid.conf below:
access_log syslog:local0 format1 cache_log syslog:local1 cache_store_log syslog:local2
For shared servers with multiple servers, you have to make sure that you don’t reuse facility levels as different logs will be combined.
Known issues
Remote rsyslog logging under load causing applications to go into blocking state