Splunk Search

Interesting regex/transforms.conf question

kubowler99
New Member

My dilemma:

We have a log file that dumps out info from an array.

Four fields:

Count
FieldA
FieldB
FieldC

In the log file, the field 'Count' will provide the number of entries in the array. There will then be 'Count' instances of each field (FieldA, FieldB, FieldC).

For example:

2012.02.28 00:02:00.000|Count: 1, FieldA[0]: abcdefg, FieldB[0]: 12345, FieldC[0]: 1234abcd
2012.02.28 00:02:01.000|Count: 2, FieldA[0]: abcdefg, FieldB[0]: 12345, FieldC[0]: 1234abcd, FieldA[1]: hijklmn, FieldB[1]: 67890, FieldC[1]: 5678efgh

We don't really care about FieldA[0] vs FieldA[1] for metrics, but we do want to know about the data for all instances of FieldA. Not sure if some combination of repetition and grouping will accomplish this or not. I will be testing it out, but was also looking for others' feedback and if they've done this before.

Is there a 'simple' (and I use that term lightly) way to dynamically parse the fields using Splunk (transforms.conf, etc.)?

Tags (1)
0 Karma

Stephen_Sorkin
Splunk Employee
Splunk Employee

Yes, this is a good use of multivalued fields and the MV_ADD property in transforms.conf. First, add a stanza in props.conf like:

[source::.../yourfile.log*]
REPORT-numberedfields = numberedfields

Then add the corresponding stanza in transforms.conf, which directs Splunk to read fieldnames followed by an ignored number, followed by the value, and, when duplicate field names are encountered, to accumulate into the same field:

[numberedfields]
REGEX = (\w+)\[\d+\]: ([^,]+)
FORMAT = $1::$2
MV_ADD = True
Get Updates on the Splunk Community!

Registration for Splunk University is Now Open!

Are you ready for an adventure in learning?   Brace yourselves because Splunk University is back, and it's ...

Splunkbase | Splunk Dashboard Examples App for SimpleXML End of Life

The Splunk Dashboard Examples App for SimpleXML will reach end of support on Dec 19, 2024, after which no new ...

Understanding Generative AI Techniques and Their Application in Cybersecurity

Watch On-Demand Artificial intelligence is the talk of the town nowadays, with industries of all kinds ...