Getting Data In

Changing Key Value Parsing to Capture fieldname(value)

stevepraz
Path Finder

We are trying to import a new source of data which has some entries with data in a key value pair format but it is not in a format that Splunk normally recognizes.

Here is an example:
2015/01/29 06:56:51.351 Low Messaging.Message.MessageUnpackaged.Request CoreId(ci1419.896@servername_te) MessageState(UnpackagedRequest) FinalState(false) MessageSize(236080) BackupFilename(/opt/software/common/data/backup/servername_te/month01/d29/h06/ci1419.896@servername_te_Consumed_2806450) ConsumptionFilename(c33) Protocol(RAW) ProtocolVersion(1.0)

So, I'm looking to capture that there is a field called MessageState and it's value is "UnpackagedRequest". The problem is that the fields differ (in names and quantity) from line to line so it would be difficult to configure them individually. I was hoping there is some way I can configure transform.conf or props.conf in order for Splunk to recognize FieldName(Value) the same way it recognizes FieldName=Value.

Tags (1)
0 Karma

chanfoli
Builder

Instead of using a kv mode, here is another option I came up with for a custom dynamic search-time extraction using the above format:

For transforms.conf:

[stevescustomextraction]
REGEX = \s(\w+)\(([^\)]+)\)
FORMAT = $1::$2

For props.conf:

[stevessourcetypel]
REPORT-customextraction = stevescustomextraction
0 Karma

lguinn2
Legend

The problem you will have is that the DELIMS setting in transforms.conf allows you to specify only two delimiters: one that separates the pairs of key/values and one that separates the key from the value. So you could try this in transforms.conf:

DELIMS = ")", "("
CLEAN_KEYS = true

But I am not sure it will work. Instead, you could do the following for each field. In props.conf:

EXTRACT-e1=FinalState\((?<FinalState>.*?)\)
EXTRACT-e2= Protocol\((?< Protocol >.*?)\)
EXTRACT-e3= ConsumptionFilename\((?< ConsumptionFilename >.*?)\)

Note that by creating a list of field extractions in props.conf, each extraction is independent. So the fields can appear in any order and the extractions will work, even if not all fields appear in all events. I know this method will work.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...