Getting Data In

Extracting fields from different syslog messages using props.conf transforms.conf


The syslog messages we receive from the firewall have multiple formats. A limited sample is listed below

Apr 30 15:26:20 %ASA-4-313004: Denied ICMP type=0, from laddr on interface outside to no matching session
Apr 29 13:39:31 %ASA-6-605004: Login denied from to inside: for user "dstrollo"
Apr 28 13:58:47 %ASA-3-710003: TCP access denied by ACL from to Internet:

Our requirement is to be able to extract all the fields in these messages for our analysts.

My question is whether the configuration below will work. If it doesn't it may mess up the existing index. Any advice will be much appreciated.

In props.conf, I set up a basic search to extract the syslog message in each record. The results are listed below

Denied ICMP type=0, from laddr on interface outside to no matching session
Login denied from to inside: for user "dstrollo"
TCP access denied by ACL from to Internet:

The props.conf is listed below



pulldown_type = 1

%Y = year, %m = month, %d = day, %H = hour, %M = minute, %S = seconds, %z = time zone offset

example = 2015-03-25T16:22:01-04:00

the - ( in -4:00 => %z ) is part of the timezone specification (UTC-04:00 versus UTC+04:00)

TIME_FORMAT = %Y-%m-%dT%H:%M:%S%z
EXTRACT-syslog = \s(?[^\s]+)\s(?[^:]+:)\s(?.+)

Then I use different transforms to process each message type separately at search time

In props.conf

EXTRACT-syslog = "as above"
REPORT-SyslogMsg = SyslogMsg1, SyslogMsg2, SyslogMsg3

In trasnforms.conf

REGEX = "extract fields in Msg1"

REGEX = "extract fields in Msg2"

REGEX = "extract fields in Msg3"

Thank you.

0 Karma


Thank you !,

0 Karma

Esteemed Legend

It will work and even if it doesn't, because you are using the "REPORT-" directive (Search-time) instead of the "TRANSFORMS-" (or "EXTRACT-") directive (Index-time), it will not do any permanent modifications so there is no risk. Is there really missing backslashes for this transform or did you just mess up the markdown for it:

I would make sure each of your 3 REGEX strings contains the string literal text for each individual message variation to avoid false extractions (e.g. "ASA-4-313004", "ASA-6-605004", and "ASA-3-710003").

P.S. You do not need the "EXTRACT-syslog" more than once.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In the last month, the Splunk Threat Research Team (STRT) has had 2 releases of new security content via the ...

Announcing the 1st Round Champion’s Tribute Winners of the Great Resilience Quest

We are happy to announce the 20 lucky questers who are selected to be the first round of Champion's Tribute ...

We’ve Got Education Validation!

Are you feeling it? All the career-boosting benefits of up-skilling with Splunk? It’s not just a feeling, it's ...