Splunk Search

How do I configure multi-value fields for RFC 5424-compliant syslog events?

hulahoop
Splunk Employee
Splunk Employee

Splunk understands old school BSD-style syslog events effortlessly. For RFC 5424-style events, multiple data structure elements are allowed. How do I get Splunk to understand there are multiple ID fields?

Here is a sample event:

<363>1 2009-11-12T21:35:53.45-08:00 hulahoop-macbook-pro.local BANKING - Transfer [Hula@12293 Amount="55.00" FromAccount="12345601" ToAccount="12345602"][Hoop@12293 timezone="America/Los_Angeles" bcId="DI4448" sessionId="TestUser" userId="User1" hostName="MyHost" companyId="Company1" userProduct="Banking" ipAddress="10.200.10.5"] Transfer successful

0 Karma
1 Solution

hulahoop
Splunk Employee
Splunk Employee

Edit your props.conf and transforms.conf and add a repeat match regex for the ID field:

Add to $SPLUNK_HOME/etc/system/local/props.conf:

[syslog]
REPORT-foo = syslogFields,repeatMatchIDField

Add to $SPLUNK_HOME/etc/system/local/transforms.conf:

[syslogFields]
REGEX = \S+ \S+ \S+ (\S+) (\S+) (\S+) \[.*\]+? (.*)$
FORMAT = APPNAME::$1 PROCID::$2 MSGID::$3 MSG::$4

[repeatMatchIDField]
REGEX = \[(\S+)
FORMAT = ID::$1
MV_ADD = true

View solution in original post

Ledion_Bitincka
Splunk Employee
Splunk Employee

A similar approach could be taken for other fields - since the fields can have the same name you would have to work with multivalued fields. Here is an example

[syslog]
REPORT-other-fields = otherFields
# disable auto kv
KV_MODE = none

[otherFields]
REGEX  = (\S+)="([^"])+"
FORMAT = $1::$2
MV_ADD = true
0 Karma

hulahoop
Splunk Employee
Splunk Employee

Edit your props.conf and transforms.conf and add a repeat match regex for the ID field:

Add to $SPLUNK_HOME/etc/system/local/props.conf:

[syslog]
REPORT-foo = syslogFields,repeatMatchIDField

Add to $SPLUNK_HOME/etc/system/local/transforms.conf:

[syslogFields]
REGEX = \S+ \S+ \S+ (\S+) (\S+) (\S+) \[.*\]+? (.*)$
FORMAT = APPNAME::$1 PROCID::$2 MSGID::$3 MSG::$4

[repeatMatchIDField]
REGEX = \[(\S+)
FORMAT = ID::$1
MV_ADD = true

gkanapathy
Splunk Employee
Splunk Employee

This will extract your IDs, but will not distinguish between parameters in different structures that have the same name. I'm not sure if there is a general way in Splunk REGEX to split them up into individual fields to avoid name conflicts.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...