Splunk Search

How do I configure multi-value fields for RFC 5424-compliant syslog events?

hulahoop
Splunk Employee
Splunk Employee

Splunk understands old school BSD-style syslog events effortlessly. For RFC 5424-style events, multiple data structure elements are allowed. How do I get Splunk to understand there are multiple ID fields?

Here is a sample event:

<363>1 2009-11-12T21:35:53.45-08:00 hulahoop-macbook-pro.local BANKING - Transfer [Hula@12293 Amount="55.00" FromAccount="12345601" ToAccount="12345602"][Hoop@12293 timezone="America/Los_Angeles" bcId="DI4448" sessionId="TestUser" userId="User1" hostName="MyHost" companyId="Company1" userProduct="Banking" ipAddress="10.200.10.5"] Transfer successful

0 Karma
1 Solution

hulahoop
Splunk Employee
Splunk Employee

Edit your props.conf and transforms.conf and add a repeat match regex for the ID field:

Add to $SPLUNK_HOME/etc/system/local/props.conf:

[syslog]
REPORT-foo = syslogFields,repeatMatchIDField

Add to $SPLUNK_HOME/etc/system/local/transforms.conf:

[syslogFields]
REGEX = \S+ \S+ \S+ (\S+) (\S+) (\S+) \[.*\]+? (.*)$
FORMAT = APPNAME::$1 PROCID::$2 MSGID::$3 MSG::$4

[repeatMatchIDField]
REGEX = \[(\S+)
FORMAT = ID::$1
MV_ADD = true

View solution in original post

Ledion_Bitincka
Splunk Employee
Splunk Employee

A similar approach could be taken for other fields - since the fields can have the same name you would have to work with multivalued fields. Here is an example

[syslog]
REPORT-other-fields = otherFields
# disable auto kv
KV_MODE = none

[otherFields]
REGEX  = (\S+)="([^"])+"
FORMAT = $1::$2
MV_ADD = true
0 Karma

hulahoop
Splunk Employee
Splunk Employee

Edit your props.conf and transforms.conf and add a repeat match regex for the ID field:

Add to $SPLUNK_HOME/etc/system/local/props.conf:

[syslog]
REPORT-foo = syslogFields,repeatMatchIDField

Add to $SPLUNK_HOME/etc/system/local/transforms.conf:

[syslogFields]
REGEX = \S+ \S+ \S+ (\S+) (\S+) (\S+) \[.*\]+? (.*)$
FORMAT = APPNAME::$1 PROCID::$2 MSGID::$3 MSG::$4

[repeatMatchIDField]
REGEX = \[(\S+)
FORMAT = ID::$1
MV_ADD = true

gkanapathy
Splunk Employee
Splunk Employee

This will extract your IDs, but will not distinguish between parameters in different structures that have the same name. I'm not sure if there is a general way in Splunk REGEX to split them up into individual fields to avoid name conflicts.

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...