topic Splunk field seperators in Splunk Search

Splunk field seperators

timbCFCA — Mon, 12 Sep 2011 15:59:44 GMT

I love Splunk's ability to dynamically pull fields at runtime with name=value pairs.
I have several log formats which have a "key name: value" format or similar. The exact settings are:

There may potentially be spaces in both the key name and value
There are always two spaces between the colon and the value
The value may be blank
Each set of values is separated by three or four spaces

Subject:   Security ID:  S-1-5-18   Account Name:  ACCOUNT$   Account Domain:  DOMAIN   Logon ID:  0x3e7    Process Information:   New Process ID:  0x1bdc  Another Field:  Test Results negative

The regex I'm trying to use for my extraction is

\s\s\s([^\s]+):\s\s(.*)\s\s\s

Where am I going wrong?

Re: Splunk field seperators

Ayn — Mon, 12 Sep 2011 21:11:11 GMT

Use a regex that extracts the key and the value in transforms.conf.

[with_colon]
REGEX = \t([^\s:]+):\s(\S+)\t
FORMAT = $1::$2

Re: Splunk field seperators

gkanapathy — Tue, 13 Sep 2011 05:50:57 GMT

well, the field name may include spaces.

Re: Splunk field seperators

gkanapathy — Tue, 13 Sep 2011 05:53:53 GMT

I suppose the problem if the key or value may contain spaces is, how do you tell when one end and other begins, e.g., if you see:

one two: three four five: six seven eight nine: ten eleven: twelve

What would be the field names and what would be the values? Apparently one two is a field, but is its value three or three four? And so on.

If you can define that, then a regex can be created, but the idea is kind of what is in Ayn's answer.

Re: Splunk field seperators

timbCFCA — Tue, 13 Sep 2011 15:18:58 GMT

There should be a tab character between each set of fields so appending and prepending to the regex from Ayn should do the trick. I'll try it out and see how it goes.

Re: Splunk field seperators

Ayn — Tue, 13 Sep 2011 15:22:24 GMT

True, I missed that in the question! Editing to reflect on that, and the info that tabs are at the start and end of each k/v pair.

Re: Splunk field seperators

timbCFCA — Tue, 13 Sep 2011 19:58:36 GMT

Ayn, many thanks. I've done some additional research on the message format and updated the question accordingly. Could you take a look at what I need in the way of a regex?