问题描述:

I am working on a Java project to parse logs where I am using Java Grok library for pattern recognition. I have given the pattern as follows:

%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\\[%{POSINT:syslog_pid}\\])?: %{GREEDYDATA:syslog_message}

When I try parsing the line,

Dec 23 14:30:01 louis CRON[619]: (www-data) CMD (php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log)

it gives the following output:

syslog_timestamp=Dec 23 14:30:01

syslog_hostname=louis

syslog_program=CRON

syslog_pid=619

syslog_message=(www-data) CMD (php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log)

I want to extract details from the syslog_message further to get stuff like facility and severity. How can I improve the grok pattern to get these details?

网友答案:

I am not an expert of Java Grok. You can use www-data as your facility and error as your severity.

Change your pattern to get this information.

syslog_facility=www-data syslog-severity=error

Regards, Ivo

网友答案:

As far as I see, you can't get the severity and facility from that message because it's not there. Assuming this is written by a syslog daemon, and assuming that syslog daemon is rsyslog, you can change the way things are written down by altering the template and select the properties you want.

You can write, for example, one JSON per line in a file with a template like this (basically copy-pasted from this rsyslog->Elasticsearch blog post):

template(name="jsonPerLine"
  type="list") {
    constant(value="{")
      constant(value="\"@timestamp\":\"")     property(name="timereported" dateFormat="rfc3339")
      constant(value="\",\"host\":\"")        property(name="hostname")
      constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")
      constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")
      constant(value="\",\"tag\":\"")   property(name="syslogtag" format="json")
      constant(value="\",\"message\":\"")    property(name="msg" format="json")
    constant(value="\"}\n")
}

This way, in your application you can parse this JSON and get all these fields, which is much cheaper than using regular expressions (which is what grok does).

You'd use this template to write to files later on in the configuration, for example:

action(
  type="omfile"
  file="/var/log/json_messages"
  template="jsonPerLine"
)

If you get errors from the syntax above, it may be that your distro has a really-really old version of rsyslog. If that's so, you can get packages from the official repositories.

相关阅读:
Top