Monday, 17 April 2017

Config Log4j to Send Kafka logs to Syslog

1. Configure Syslog Daemon for UDP Input

$ sudo vim /etc/rsyslog.conf

Uncomment these lines to accept UDP messages on the default port 514.
$ModLoad imudp
$UDPServerRun 514
local7.*      /var/log/kafka.log

2. Restart the rsyslog service so the changes take effect

$ sudo service rsyslog restart

3. Open your log4j.properties file:

$ vi /opt/kafka/config/log4j.properties



log4j.rootLogger =INFO, stdout, SYSLOG
log4j.appender.SYSLOG =org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.SyslogHost =localhost
log4j.appender.SYSLOG.Facility =LOCAL7
log4j.appender.SYSLOG.threshold =INFO
log4j.appender.SYSLOG.DatePattern ='.'yyyy-MM-dd-HH
log4j.appender.SYSLOG.layout =org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.ConversionPattern =kafka-broker: [%d] %p %m (%c)%n

Log4j comes out of the box with a SyslogAppender
Configure SyslogAppender to write these messages as UDP over localhost to the syslog daemon. The first field in the conversion pattern is the syslog appname.

4. Restart Kafka
$ sudo stop kafka-broker
$ sudo start kafka-broker

Reference:

1 comment:

  1. Great explanation on Lambda vs Kappa architecture! Data processing like this is crucial in various sectors, including healthcare, where managing and analyzing large datasets efficiently is key. For anyone interested in learning how to handle healthcare data and coding, medical coding courses in Delhi could be a great opportunity to explore this field!
    Medical Coding Courses in Delhi

    ReplyDelete