Archive

Extracting tabular information using KV_MODE

Hazel
Communicator

Hello

I have a search that uses multikv in the search command to take my tablular event and split it up into fields, as follows

index=logchecker | multikv fields App LogFile FileCount FileSize LineCount MinDate MaxDate NoOfMinutes AveCharPerMinute

This gives me an event on each line, with it picking up each field, so I can just then pass these fields to a "table" command and it comes out nicely.

However, I'd like to find a way to do this automatically in props.conf to make things simpler. I looked it up and the documentation suggests just adding the following, but this doesn't seem to have any impact.

KV_MODE = true
CHECK_FOR_HEADER = true

Is there something else I need to add? My input is basically a script which echos lines of data as it searches through logs, outputting a first line of headers. I've changed props.conf to merge all the data together so I can then split it correctly based on the headers. Current props.conf

[logchecker]
BREAK_ONLY_BEFORE = <stuff>
DATETIME_CONFIG = CURRENT
KV_MODE = multi
CHECK_FOR_HEADER = true

Current Output

App,LogFile,FileCount,FileSize,LineCount,MinDate,MaxDate,NoOfMinutes,AveCharPerMinute
./configservice,vsol43a-6005,configservice.tc1.log, 21, 106659113, 845724,2010-04-06 06:24,2010-12-08 10:34,354490,300
./configservice,vsol43a-6005,configservice.tp1.log, 9, 42589938, 284878,2010-01-19 09:03,2010-12-08 10:44,465221,91

Any ideas how I can do this extraction automatically?

Thanks Hazel

Tags (1)
0 Karma

Ledion_Bitincka
Splunk Employee
Splunk Employee

Try using the following stanza in props.conf:

[logchecker]
SHOULD_LINEMERGE = false
LINE_BREAKER     = (ThiStringDoesNotExistInYourData)
TRUNCATE         = 100000
DATETIME_CONFIG  = CURRENT
KV_MODE          = multi

The first four lines of that stanza instruct Splunk to index the entire file content (up to 100KB, increase if necessary) as a single event and assign it the current timestamp. KV_MODE tells Splunk search to apply auto multikv to the events when they are retrieved from the index and before any further search time processing

gkanapathy
Splunk Employee
Splunk Employee

For a LINE_BREAKER that does not ever match, I use LINE_BREAKER = (?!)