topic Re: Extracting tabular information using KV_MODE in Splunk Search

Extracting tabular information using KV_MODE

Hazel — Wed, 08 Dec 2010 19:06:14 GMT

Hello

I have a search that uses multikv in the search command to take my tablular event and split it up into fields, as follows

index=logchecker | multikv fields App LogFile FileCount FileSize LineCount MinDate MaxDate NoOfMinutes AveCharPerMinute

This gives me an event on each line, with it picking up each field, so I can just then pass these fields to a "table" command and it comes out nicely.

However, I'd like to find a way to do this automatically in props.conf to make things simpler. I looked it up and the documentation suggests just adding the following, but this doesn't seem to have any impact.

KV_MODE = true
CHECK_FOR_HEADER = true

Is there something else I need to add? My input is basically a script which echos lines of data as it searches through logs, outputting a first line of headers. I've changed props.conf to merge all the data together so I can then split it correctly based on the headers. Current props.conf

[logchecker]
BREAK_ONLY_BEFORE = <stuff>
DATETIME_CONFIG = CURRENT
KV_MODE = multi
CHECK_FOR_HEADER = true

Current Output

App,LogFile,FileCount,FileSize,LineCount,MinDate,MaxDate,NoOfMinutes,AveCharPerMinute
./configservice,vsol43a-6005,configservice.tc1.log, 21, 106659113, 845724,2010-04-06 06:24,2010-12-08 10:34,354490,300
./configservice,vsol43a-6005,configservice.tp1.log, 9, 42589938, 284878,2010-01-19 09:03,2010-12-08 10:44,465221,91

Any ideas how I can do this extraction automatically?

Thanks Hazel

Re: Extracting tabular information using KV_MODE

Ledion_Bitincka — Thu, 09 Dec 2010 02:10:04 GMT

Try using the following stanza in props.conf:

[logchecker]
SHOULD_LINEMERGE = false
LINE_BREAKER     = (ThiStringDoesNotExistInYourData)
TRUNCATE         = 100000
DATETIME_CONFIG  = CURRENT
KV_MODE          = multi

The first four lines of that stanza instruct Splunk to index the entire file content (up to 100KB, increase if necessary) as a single event and assign it the current timestamp. KV_MODE tells Splunk search to apply auto multikv to the events when they are retrieved from the index and before any further search time processing

Re: Extracting tabular information using KV_MODE

gkanapathy — Thu, 03 Feb 2011 05:34:54 GMT

For a LINE_BREAKER that does not ever match, I use LINE_BREAKER = (?!)