Changing Key Value Parsing to Capture fieldname(value)

[stevepraz]

We are trying to import a new source of data which has some entries with data in a key value pair format but it is not in a format that Splunk normally recognizes.

Here is an example:
2015/01/29 06:56:51.351 Low Messaging.Message.MessageUnpackaged.Request CoreId(ci1419.896@servername_te) MessageState(UnpackagedRequest) FinalState(false) MessageSize(236080) BackupFilename(/opt/software/common/data/backup/servername_te/month01/d29/h06/ci1419.896@servername_te_Consumed_2806450) ConsumptionFilename(c33) Protocol(RAW) ProtocolVersion(1.0)

So, I'm looking to capture that there is a field called MessageState and it's value is "UnpackagedRequest". The problem is that the fields differ (in names and quantity) from line to line so it would be difficult to configure them individually. I was hoping there is some way I can configure transform.conf or props.conf in order for Splunk to recognize FieldName(Value) the same way it recognizes FieldName=Value.

[chanfoli]

Instead of using a kv mode, here is another option I came up with for a custom dynamic search-time extraction using the above format:

For transforms.conf:

[stevescustomextraction]
REGEX = \s(\w+)\(([^\)]+)\)
FORMAT = $1::$2

For props.conf:

[stevessourcetypel]
REPORT-customextraction = stevescustomextraction

[lguinn2]

The problem you will have is that the DELIMS setting in transforms.conf allows you to specify only two delimiters: one that separates the pairs of key/values and one that separates the key from the value. So you could try this in transforms.conf:

DELIMS = ")", "("
CLEAN_KEYS = true

But I am not sure it will work. Instead, you could do the following for each field. In props.conf:

EXTRACT-e1=FinalState\((?<FinalState>.*?)\)
EXTRACT-e2= Protocol\((?< Protocol >.*?)\)
EXTRACT-e3= ConsumptionFilename\((?< ConsumptionFilename >.*?)\)

Note that by creating a list of field extractions in props.conf, each extraction is independent. So the fields can appear in any order and the extractions will work, even if not all fields appear in all events. I know this method will work.