I have a custom scripted input generating output as follows which is being sucessfully indexed:
Timestamp | ServiceDescription | ServiceName | ServiceURI | ResponseTime | HTTPCode | Status | FailReason
2013-10-27T12-46-24 | abc.domain.com/Account/GetAPS | GetAPS | bus.domain.com:0000/Site/GetAPS.svc | 0.240 | 200 | Pass |
2013-10-27T12-46-25 | abc.domain.com/Account/GetAPS2 | GetAPS2 | bus.domain.com:0000/Site/GetAPS2.svc | 0.340 | 200 | Pass |
2013-10-27T12-46-26 | abc.domain.com/Account/GetAPS3 | GetAPS3 | bus.domain.com:0000/Site/GetAPS3.svc | 0.440 | 200 | Pass |
2013-10-27T12-46-27 | abc.domain.com/Account/GetAPS4 | GetAPS4 | bus.domain.com:0000/Site/GetAPS4.svc | 0.540 | 200 | Pass |
Based on advice from the answer to multikvconf for data with pipe delimeter my config is as follows:
interval = 600
source = customscript.sh
index = customindex
sourcetype = custominput
disabled = 0
header.tokens = Timestamp,ServiceDescription,ServiceName,ServiceURI,ResponseTime,HTTPCode,Status,FailReason
header.start = "Timestamp"
header.linecount = 1
header.tokens = tokenize, -1, "|"
body.tokens = tokenize, 0, "|"
NOBINARYCHECK = 1
SHOULDLINEMERGE = true
pulldowntype = 1
The search that I am using is:
index="customindex" | multikv conf=custommultikv forceheader=1
The problem is that with or without * header.tokens * header.start * props.conf or * forceheader=1 the events are split but the fields are not created so the following, for example, returns no events:
index="mswmbpmallprod" splunkserver=dd894c3n12-9002 | multikv conf=ebcmultikv forceheader=1 | table Service_Description
The following configuration worked for me:
[multitest] header.start = "Timestamp" header.linecount = 1 header.tokens = _tokenize_, -1," | " body.tokens = _tokenize_, -1, " | "
This will remove the hearder automatically from the results and then split each line into a separate event when using the following search sourcetype="multikvtest" | multikv conf=multitest
If you want to filter the results further by field value, you will only be able to do so by piping to the search command
Thank you. Adding the spaces before and after the pipe (delimiter) and changing -1 to 0 in body.tokens worked like a charm.
It was a revelation though, that the fields do not show up in the field picker until you use another command after multikv like table etc (they show up imediately for bult-in multikv support like ps and top etc).