How do I configure multi-value fields for RFC 5424-compliant syslog events?

[hulahoop]

Splunk understands old school BSD-style syslog events effortlessly. For RFC 5424-style events, multiple data structure elements are allowed. How do I get Splunk to understand there are multiple ID fields?

Here is a sample event:

<363>1 2009-11-12T21:35:53.45-08:00 hulahoop-macbook-pro.local BANKING - Transfer [Hula@12293 Amount="55.00" FromAccount="12345601" ToAccount="12345602"][Hoop@12293 timezone="America/Los_Angeles" bcId="DI4448" sessionId="TestUser" userId="User1" hostName="MyHost" companyId="Company1" userProduct="Banking" ipAddress="10.200.10.5"] Transfer successful

[Ledion_Bitincka]

A similar approach could be taken for other fields - since the fields can have the same name you would have to work with multivalued fields. Here is an example

[syslog]
REPORT-other-fields = otherFields
# disable auto kv
KV_MODE = none

[otherFields]
REGEX  = (\S+)="([^"])+"
FORMAT = $1::$2
MV_ADD = true

[hulahoop · Accepted solution]

Edit your props.conf and transforms.conf and add a repeat match regex for the ID field:

Add to $SPLUNK_HOME/etc/system/local/props.conf:

[syslog]
REPORT-foo = syslogFields,repeatMatchIDField

Add to $SPLUNK_HOME/etc/system/local/transforms.conf:

[syslogFields]
REGEX = \S+ \S+ \S+ (\S+) (\S+) (\S+) \[.*\]+? (.*)$
FORMAT = APPNAME::$1 PROCID::$2 MSGID::$3 MSG::$4

[repeatMatchIDField]
REGEX = \[(\S+)
FORMAT = ID::$1
MV_ADD = true

[gkanapathy]

This will extract your IDs, but will not distinguish between parameters in different structures that have the same name. I'm not sure if there is a general way in Splunk REGEX to split them up into individual fields to avoid name conflicts.