topic Re: Problem using multikv to parse tabular data in Monitoring Splunk

Problem using multikv to parse tabular data

emiller42 — Mon, 14 Oct 2013 20:32:17 GMT

Hello!

I'm trying to create a custom multikv.conf stanza for the first time, and am having trouble getting it working. Here is an example of the events I'm working with:

Object ID Volume                    Aggregate     Storage Server          Daily Growth Rate   Days To Full Daily Growth Rate (%)
--------- ------------------------- ------------- ----------------------  ------------------- ------------ ---------------------
36906     volume_1                  aggr1_vmw_sas nas-vmw-prod02.foo.com           1074009984            1                  10.0
25031     volume_2                  aggr2_vmw_sas nas-vmw-prod03.foo.com             29670000           75                   0.6
25033     volume_3                  aggr2_vmw_sas nas-vmw-prod03.foo.com             23464000          146                   0.4
5430      volume_4                  aggr1_vmw_sas nas-vmw-prod02.foo.com             22181400          273                   0.3
Totals                                                                             1249102095

The multikv.conf stanza I created is as follows:

[volume_growth_rates]

#ignore the first two lines, as getting proper tokens with the spaces and % will be problematic
pre.start = "Object"
pre.linecount = 2
pre.ignore = _all_

#Manually define the header tokens
header.tokens = _token_list_, ObjectId, Volume, Aggregate, StorageServer, DailyGrowthRate, DaysToFull, DailyGrowthRatePct

#tokenize with space separators
body.end = "Totals"
body.tokens = _tokenize_, 0, " "

#ignore the 'Totals' line, as it doesn't really contain useful data.
post.start = "Totals"
post.linecount = 1
post.ignore = _all_

When I try to use this in search, nothing happens. Trying both:

... | multikv

and

... | multkv conf=volume_growth_rates

Any ideas on what I'm missing here?

Thanks!

Re: Problem using multikv to parse tabular data

Masa — Mon, 28 Sep 2020 15:50:10 GMT

Tested in v6.0.

multkv.conf is annoying.
It is acting funny. We cannot trust the spec file.... sad. Examples do not work....

Solution 1. Forget about multikv.conf and use general Field Extraction

props.conf

[source::.../Log/test_multikv_table.log]
REPORT-multikv = test_multikv_table_1 test_multikv_table_2
transforms.conf

[test_multikv_table_1]
REGEX = ^((\d+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+)\s+(\S+))
FORMAT = Row::$1 objectID::$2 volume::$3 aggregate::$4 storageServer::$5 dailyGrowthRate::$6 daysToFull::$7 growthRate::$8
MV_ADD = true

[test_multikv_table_2]
REGEX = ^Totals\s+(\S+)
FORMAT = Totals::$1

Solution 2. Stick with buggy multikv.conf

# Put multikv.conf in all of Splunk instance (Search Head and Peers)
- props.conf
[source::.../Log/test_multikv_table.log]
LINE_BREAKER = ([\n\r]+)Object ID
SHOULD_LINEMERGE = false
DATETIME_CONFIG  = CURRENT
KV_MODE = multi_volumeGrowthRates

multikv.conf

Bug: must be located in Search Peer's local directory, instead of bundle from SH

[volumeGrowthRates]

Not sure if "pre" is working

pre.start = "Object"
pre.end = "----.*"

pre.linecount = 2

pre.ignore = all

header setting

header.tokens = token_list,"ObjectId","Volume","Aggregate","StorageServer","DailyGrowthRate","DaysToFull","DailyGrowthRatePct"

Body needs to set ignoring pre and post lines

body.tokens = tokenize, 0, " "
body.ignore = regex "----|Totals"

Post... ignore is not working???

post.start = "Totals"
post.linecount = 1
post.ignore = all

Re: Problem using multikv to parse tabular data

stanwin — Mon, 03 Aug 2015 10:00:36 GMT

Maybe it is issue with identifying the header?

How does SPLUNk know when to ignore the header here?

example with the manual approach of assigning headers, we specify start of header & linecount:

# specify table header location and processing 
header.start = "Object ID" 
header.linecount = 1 
header.tokens = _tokenize_, -1," "

topic Re: Problem using multikv to parse tabular data in Monitoring Splunk

Problem using multikv to parse tabular data

Re: Problem using multikv to parse tabular data

Bug: must be located in Search Peer's local directory, instead of bundle from SH

Not sure if "pre" is working

pre.linecount = 2

header setting

Body needs to set ignoring pre and post lines

Post... ignore is not working???

Re: Problem using multikv to parse tabular data