Drop events based on Syslog Severity at Ingest

[mnietert]

Hey all,

I've added the following to props.conf to parse out PRI from _raw, and Severity/Facility codes from PRI. 

<p>props.conf<br>
[syslog]<br>
EXTRACT-PRI = ^<(?P<PRI>\d+)<br>
LOOKUP-syslog_facility = syslog_facility "Facility-code" AS "Facility-Code" OUTPUTNEW Facility AS Facility<br>
EVAL-Facility-Code = (PRI - (PRI % 8)) / 8<br>
EVAL-Severity-Code = PRI % 8<br>
LOOKUP-syslog_severity = syslog_severity "Sev-code" AS "Severity-Code" OUTPUTNEW Severity AS Severity</p>

Now, we'd like to drop events if the Severity-Level is not above a certain level (for example, we'd like to drop all debug messages at ingest). I know that first we will have to convert the Severity/Facility codes from EVAL to INGEST_EVAL values so that we can operate on their values at ingest, but what is the best way to filter all messages that are say, Severity-Code>=6 (dropping info and debug.) I was thinking of applying another transform with a REGEX command that forwards to a null queue, but it doesn't look like there's any great boolean evaluators available in transforms. In search I could just use WHERE Severity_Code>6, but that's not available at ingest...

[richgalloway]

If you have a REGEX that identifies the events you want to discard then use it in transforms.conf.  You are correct, though, when you say one cannot use expressions like foo>6 in regular expressions.

You should find it easier to use your syslog server (rsylog or syslog-ng) to do the filtering.