Splunk Search

Interesting regex/transforms.conf question

kubowler99
New Member

My dilemma:

We have a log file that dumps out info from an array.

Four fields:

Count
FieldA
FieldB
FieldC

In the log file, the field 'Count' will provide the number of entries in the array. There will then be 'Count' instances of each field (FieldA, FieldB, FieldC).

For example:

2012.02.28 00:02:00.000|Count: 1, FieldA[0]: abcdefg, FieldB[0]: 12345, FieldC[0]: 1234abcd
2012.02.28 00:02:01.000|Count: 2, FieldA[0]: abcdefg, FieldB[0]: 12345, FieldC[0]: 1234abcd, FieldA[1]: hijklmn, FieldB[1]: 67890, FieldC[1]: 5678efgh

We don't really care about FieldA[0] vs FieldA[1] for metrics, but we do want to know about the data for all instances of FieldA. Not sure if some combination of repetition and grouping will accomplish this or not. I will be testing it out, but was also looking for others' feedback and if they've done this before.

Is there a 'simple' (and I use that term lightly) way to dynamically parse the fields using Splunk (transforms.conf, etc.)?

Tags (1)
0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

Yes, this is a good use of multivalued fields and the MV_ADD property in transforms.conf. First, add a stanza in props.conf like:

[source::.../yourfile.log*]
REPORT-numberedfields = numberedfields

Then add the corresponding stanza in transforms.conf, which directs Splunk to read fieldnames followed by an ignored number, followed by the value, and, when duplicate field names are encountered, to accumulate into the same field:

[numberedfields]
REGEX = (\w+)\[\d+\]: ([^,]+)
FORMAT = $1::$2
MV_ADD = True
Get Updates on the Splunk Community!

Leveraging Detections from the Splunk Threat Research Team & Cisco Talos

  Now On Demand  Stay ahead of today’s evolving threats with the combined power of the Splunk Threat Research ...

New in Splunk Observability Cloud: Automated Archiving for Unused Metrics

Automated Archival is a new capability within Metrics Management; which is a robust usage & cost optimization ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...