Archive

multikv custom scripted input not mapping headers

Explorer

I have a custom scripted input generating output as follows which is being sucessfully indexed:

Timestamp | ServiceDescription | ServiceName | ServiceURI | ResponseTime | HTTPCode | Status | FailReason
2013-10-27T12-46-24 | abc.domain.com/Account/GetAPS | GetAPS | bus.domain.com:0000/Site/GetAPS.svc | 0.240 | 200 | Pass |
2013-10-27T12-46-25 | abc.domain.com/Account/GetAPS2 | GetAPS2 | bus.domain.com:0000/Site/GetAPS2.svc | 0.340 | 200 | Pass |
2013-10-27T12-46-26 | abc.domain.com/Account/GetAPS3 | GetAPS3 | bus.domain.com:0000/Site/GetAPS3.svc | 0.440 | 200 | Pass |
2013-10-27T12-46-27 | abc.domain.com/Account/GetAPS4 | GetAPS4 | bus.domain.com:0000/Site/GetAPS4.svc | 0.540 | 200 | Pass |

Based on advice from the answer to multikvconf for data with pipe delimeter my config is as follows:

inputs.conf

[script://./bin/customscript.sh]
interval = 600
source = customscript.sh
index = customindex
sourcetype = custominput
disabled = 0

multikv.conf

[custommultikv]
header.tokens = Timestamp,ServiceDescription,ServiceName,ServiceURI,ResponseTime,HTTPCode,Status,FailReason
header.start = "Timestamp"
header.linecount = 1
header.tokens = tokenize, -1, "|"
body.tokens = tokenize, 0, "|"

props.conf

[hostname]
NOBINARYCHECK = 1
SHOULDLINEMERGE = true
pulldown
type = 1
BREAKONLYBEFORE=(?!)
MAX_EVENTS=100000

The search that I am using is:

index="customindex" | multikv conf=custommultikv forceheader=1

The problem is that with or without * header.tokens * header.start * props.conf or * forceheader=1 the events are split but the fields are not created so the following, for example, returns no events:

index="mswmbpmallprod" splunkserver=dd894c3n12-9002 | multikv conf=ebcmultikv forceheader=1 | table Service_Description

Tags (2)
0 Karma

Builder

The following configuration worked for me:

[multitest] 
header.start = "Timestamp" 
header.linecount = 1 
header.tokens = _tokenize_, -1," | " 
body.tokens = _tokenize_, -1, " | " 

This will remove the hearder automatically from the results and then split each line into a separate event when using the following search sourcetype="multikvtest" | multikv conf=multitest

If you want to filter the results further by field value, you will only be able to do so by piping to the search command

Example:

| multikv conf=multitest | search field=value

Explorer

Thank you. Adding the spaces before and after the pipe (delimiter) and changing -1 to 0 in body.tokens worked like a charm.

It was a revelation though, that the fields do not show up in the field picker until you use another command after multikv like table etc (they show up imediately for bult-in multikv support like ps and top etc).

0 Karma