Loading...
 

RSyslog



Introduction


http://www.rsyslog.com/doc/manual.html is an enhanced syslogd supporting, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part. It is quite compatible to stock sysklogd and can be used as a drop-in replacement. Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user.



Installation



Rsyslog is generally included in the CentOS and Redhat repositories and all you need to do is:


$ yum install rsyslog



This should not do anything to your current (usually syslog/sysklogd) system. It just installs rsyslog in parallel (and it's up to you to turn it on). Everything is put in place for you (startup scripts, config file that is a mirror of syslog.conf, integration with log rotation, etc.) and you just have to:


chkconfig syslog off
chkconfig rsyslog on
service syslog stop
service rsyslog start


Configuration


Rsyslog Server


Example setup


A syslog system user was created with the options as set below:

adduser --system --no-create-home -s /sbin/nologin -b /var/log/remote-syslog -c 'Syslog system user for logging' syslog


The directory {{/var/log/remote-syslog}} is a mounted LVM and not located on the main file system.


This is the configuration as a centralised rsyslog server where all rsyslog clients can write their log files to:


# rsyslog v5 configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)


# Provides TCP syslog reception
$ModLoad imtcp

######################
#   Directives
######################
# Set the default permissions for all log files.

$FileOwner syslog
$FileGroup syslog
$FileCreateMode 0644
$DirCreateMode 0755


#### GLOBAL DIRECTIVES ####

# Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf


######## TEMPLATES ########

$template RemoteLog,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/%app-name%.log"
$template image-squid-access,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/squid/image/access.log"
$template image-squid-cache,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/squid/image/cache.log"
$template image-squid-store,"/var/log/remote-syslog/%$year%/%$month%/%$day%/%fromhost%/squid/image/store.log"
$template squidlog,"%msg%\n"



#### RULESETS ####
### Local Logging ###
$RuleSet local
# Log all kernel messages to the console.
#kern.*                                                 /dev/console
# Log anything (except mail) of level info or higher. Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                              /var/log/secure
# Log all the mail messages in one place.
mail.*                                                  -/var/log/maillog
# Log cron stuff
cron.*                                                  /var/log/cron
# Everybody gets emergency messages
*.emerg                                                 *
# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log


$DefaultRuleSet local

### Remote Ruleset ###
$RuleSet remote
local0.* ?image-squid-access;squidlog
local1.* ?image-squid-cache;squidlog
local2.* ?image-squid-store;squidlog
*.* ?RemoteLog

#### LISTENERS ####

#debug2: channel 0: window 999407 sent adjust 49169
$InputTCPServerBindRuleset remote
$InputTCPServerRun 10514


Rsyslog Client


The directory {{/var/log/rsyslog}} has to exist for rsyslog to put spool files.


This the configuration as a replacement for syslog with duplicating of log files to a remote syslog server:


# rsyslog v5 configuration file

# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html

#### MODULES ####

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
#$ModLoad imfile
$ModLoad immark  # provides --MARK-- message capability

#### GLOBAL DIRECTIVES ####

# Use default timestamp format



####### Templates #########

$template squidlog,"%msg%\n"

# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/*.conf


#### RULES ####
### Local Logging ###

# System specific logs use local log files

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 /dev/console

*.info;mail.none;authpriv.none;cron.none                /var/log/messages;RSYSLOG_TraditionalFileFormat
authpriv.*                                              /var/log/secure;RSYSLOG_TraditionalFileFormat
mail.*                                                  -/var/log/maillog;RSYSLOG_TraditionalFileFormat
cron.*                                                  /var/log/cron
*.emerg                                                 *
local7.*                                                /var/log/boot.log
local0.*                                                /var/log/image/access.log;squidlog
local1.*                                                /var/log/image/cache.log;squidlog
local2.*                                                /var/log/image/store.log;squidlog
*.* -/var/log/


### Remote Ruleset ###

# ### begin forwarding rule ###
# The statement between the begin ... end define a SINGLE forwarding
# rule. They belong together, do NOT split them. If you create multiple
# forwarding rules, duplicate the whole block!
# Remote Logging (we use TCP for reliable delivery)
#
# An on-disk queue is created for this action. If the remote host is
# down, messages are spooled to disk and sent when it is up again.
$WorkDirectory /var/log/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down


# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional

local0.* @@10.157.51.183:10514;squidlog
local1.* @@10.157.51.183:10514;squidlog
local2.* @@10.157.51.183:10514;squidlog
*.* @@10.157.51.183:10514

# ### end of the forwarding rule ###


Advanced Configuration


For advanced configuration options, please visit the rsyslog http://wiki.rsyslog.com or the http://www.rsyslog.com/doc/manual.html

Filters

Beyond using simple facility.severity action, a more advanced set of filters can be used that take information from the message. Properties like the program name, process ID or some defined tag. For example,
/etc/rsyslog.d/10-bind.conf
:programname, isequal, "named"    /var/log/named/named.log

would be a configuration directive to redirect logs coming from the named process and filter then out to the log mentioned.

Available properties can be found in the "Available Properties" section of the rsyslog.conf man page and the comparison operations can be found under the "Property-Based Filters" section

Filter conditionals

more advanced configuration can be done using conditional statements. For example,
if $programname == 'prog1' then {
   action(type="omfile" file="/var/log/prog1.log")
   if $msg contains 'test' then
     action(type="omfile" file="/var/log/prog1test.log")
   else
     action(type="omfile" file="/var/log/prog1notest.log")
}

if $fromhost == 'host1' then {
   mail.* action(type="omfile" file="/var/log/host1/mail.log")
   *.err /var/log/host1/errlog # this is also still valid
   # 
   # more "old-style rules" ...
   #
} else {
   mail.* action(type="omfile" file="/var/log/mail.log")
   *.err /var/log/errlog
   # 
   # more "old-style rules" ...
   #
}



Examples

configuration of Squid Proxy to use Rsyslog



If one application uses more than one output to syslog;

e.g. Squid proxy with access.log, store.log and cache.log

You have to assign them individual http://en.wikipedia.org/wiki/Syslog#Facility_Levels to make sure they get written to the correct output files on the remote server.


Example excerpt from squid.conf below:

access_log syslog:local0 format1
cache_log syslog:local1
cache_store_log syslog:local2


For shared servers with multiple servers, you have to make sure that you don't reuse facility levels as different logs will be combined.

Known issues

Remote rsyslog logging under load causing applications to go into blocking state