Interacting with system logs is a crucial aspect of both security and system administration. Monitoring the log files of multiple hosts can get very unwieldy when these hosts are distributed across medium or large networks, or when they are parts of various different types of networks. In these cases, configuring remote logging may make the whole process a lot more comfortable.
Centralized logging to a specific logging host can reduce some of the administrative burden of log file administration. Log file aggregation, merging and rotation can be configured in one location, using the native tools of FreeBSD, such as syslogd(8) and newsyslog(8). In the following example configuration, host A, named logserv.example.com, will collect logging information for the local network. Host B, named logclient.example.com will pass logging information to the server system. In live configurations, both hosts require proper forward and reverse DNS or entries in /etc/hosts. Otherwise, data will be rejected by the server.
Log servers are machines configured to accept logging information from remote hosts. In most cases this is to ease configuration, in other cases it may just be a better administration move. Regardless of reason, there are a few requirements before continuing.
A properly configured logging server has met the following minimal requirements:
The firewall ruleset allows for UDP to be passed on port 514 on both the client and server;
syslogd has been configured to accept remote messages from client machines;
The syslogd server and all client machines must have valid entries for both forward and reverse DNS, or be properly configured in /etc/hosts.
To configure the log server, the client must be listed in /etc/syslog.conf, and the logging facility must be specified:
+logclient.example.com *.* /var/log/logclient.log
Note: More information on various supported and available facilities may be found in the syslog.conf(5) manual page.
Once added, all facility messages will be logged to the file specified previously, /var/log/logclient.log.
The server machine must also have the following listing placed inside /etc/rc.conf:
syslogd_enable="YES" syslogd_flags="-a logclient.example.com -v -v"
The first option will enable the syslogd daemon on boot up,
and the second option allows data from the specified client to be accepted on this
server. The latter part, using -v -v, will increase the
verbosity of logged messages. This is extremely useful for tweaking facilities as
administrators are able to see what type of messages are being logged under which
facility.
Multiple -a options may be specified to allow logging from
multiple clients. IP addresses and whole
netblocks may also be specified, see the syslog(3) manual page
for a full list of possible options.
Finally, the log file should be created. The method used does not matter, but touch(1) works great for situations such as this:
# touch /var/log/logclient.log
At this point, the syslogd daemon should be restarted and verified:
# /etc/rc.d/syslogd restart # pgrep syslog
If a PID is returned, the server has been restarted successfully, and client configuration may begin. If the server has not restarted, consult the /var/log/messages log for any output.
A logging client is a machine which sends log information to a logging server in addition to keeping local copies.
Similar to log servers, clients must also meet a few minimum requirements:
syslogd(8) must be configured to send messages of specific types to a log server, which must accept them;
The firewall must allow UDP packets through on port 514;
Both forward and reverse DNS must be configured or have proper entries in the /etc/hosts.
Client configuration is a bit more relaxed when compared to that of the servers. The client machine must have the following listing placed inside /etc/rc.conf:
syslogd_enable="YES" syslogd_flags="-s -v -v"
As before, these entries will enable the syslogd daemon on
boot up, and increases the verbosity of logged messages. The -s option prevents logs from being accepted by this client from
other hosts.
Facilities describe the system part for which a message is generated. For an example, ftp and ipfw are both facilities. When log messages are generated for those two services, they will normally include those two utilities in any log messages. Facilities are accompanied with a priority or level, which is used to mark how important a log message is. The most common will be the warning and info. Please refer to the syslog(3) manual page for a full list of available facilities and priorities.
The logging server must be defined in the client's /etc/syslog.conf. In this instance, the @ symbol is used to send logging data to a remote server and would look similar to the following entry:
*.* @logserv.example.com
Once added, syslogd must be restarted for the changes to take effect:
# /etc/rc.d/syslogd restart
To test that log messages are being sent across the network, use logger(1) on the client to send a message to syslogd:
# logger "Test message from logclient"
This message should now exist both in /var/log/messages on the client, and /var/log/logclient.log on the log server.
In certain cases, debugging may be required if messages are not being received on the log server. There are several reasons this may occur; however, the most common two are network connection issues and DNS issues. To test these cases, ensure both hosts are able to reach one another using the hostname specified in /etc/rc.conf. If this appears to be working properly, an alternation to the syslogd_flags option in /etc/rc.conf will be required.
In the following example, /var/log/logclient.log is empty, and the /var/log/messages files indicate no reason for the failure. To increase debugging output, change the syslogd_flags option to look like the following example, and issue a restart:
syslogd_flags="-d -a logclien.example.com -v -v"
# /etc/rc.d/syslogd restart
Debugging data similar to the following will flash on the screen immediately after the restart:
logmsg: pri 56, flags 4, from logserv.example.com, msg syslogd: restart syslogd: restarted logmsg: pri 6, flags 4, from logserv.example.com, msg syslogd: kernel boot file is /boot/kernel/kernel Logging to FILE /var/log/messages syslogd: kernel boot file is /boot/kernel/kernel cvthname(192.168.1.10) validate: dgram from IP 192.168.1.10, port 514, name logclient.example.com; rejected in rule 0 due to name mismatch.
It appears obvious the messages are being rejected due to a name mismatch. After reviewing the configuration bit by bit, it appears a typo in the following /etc/rc.conf line has an issue:
syslogd_flags="-d -a logclien.example.com -v -v"
The line should contain logclient, not logclien. After the proper alterations are made, a restart is issued with expected results:
# /etc/rc.d/syslogd restart logmsg: pri 56, flags 4, from logserv.example.com, msg syslogd: restart syslogd: restarted logmsg: pri 6, flags 4, from logserv.example.com, msg syslogd: kernel boot file is /boot/kernel/kernel syslogd: kernel boot file is /boot/kernel/kernel logmsg: pri 166, flags 17, from logserv.example.com, msg Dec 10 20:55:02 <syslog.err> logserv.example.com syslogd: exiting on signal 2 cvthname(192.168.1.10) validate: dgram from IP 192.168.1.10, port 514, name logclient.example.com; accepted in rule 0. logmsg: pri 15, flags 0, from logclient.example.com, msg Dec 11 02:01:28 trhodes: Test message 2 Logging to FILE /var/log/logclient.log Logging to FILE /var/log/messages
At this point, the messages are being properly received and placed in the correct file.
As with any network service, security requirements should be considered before implementing this configuration. At times, log files may contain sensitive data about services enabled on the local host, user accounts, and configuration data. Network data sent from the client to the server will not be encrypted nor password protected. If a need for encryption exists, it might be possible to use security/stunnel, which will transmit data over an encrypted tunnel.
Local security is also an issue. Log files are not encrypted during use or after log rotation. Local users may access these files to gain additional insight on system configuration. In those cases, setting proper permissions on these files will be critical. The newsyslog(8) utility supports setting permissions on newly created and rotated log files. Setting log files to mode 600 should prevent any unwanted snooping by local users.