I have a custom scripted input generating output as follows which is being sucessfully indexed:
Timestamp | Service_Description | Service_Name | Service_URI | Response_Time | HTTP_Code | Status | Fail_Reason
2013-10-27T12-46-24 | abc.domain.com/Account/GetAPS | GetAPS | bus.domain.com:0000/Site/GetAPS.svc | 0.240 | 200 | Pass |
2013-10-27T12-46-25 | abc.domain.com/Account/GetAPS2 | GetAPS2 | bus.domain.com:0000/Site/GetAPS2.svc | 0.340 | 200 | Pass |
2013-10-27T12-46-26 | abc.domain.com/Account/GetAPS3 | GetAPS3 | bus.domain.com:0000/Site/GetAPS3.svc | 0.440 | 200 | Pass |
2013-10-27T12-46-27 | abc.domain.com/Account/GetAPS4 | GetAPS4 | bus.domain.com:0000/Site/GetAPS4.svc | 0.540 | 200 | Pass |
Based on advice from the answer to multikvconf for data with pipe delimeter my config is as follows:
inputs.conf
[script://./bin/customscript.sh]
interval = 600
source = customscript.sh
index = customindex
sourcetype = custominput
disabled = 0
multikv.conf
[custommultikv]
header.tokens = Timestamp,Service_Description,Service_Name,Service_URI,Response_Time,HTTP_Code,Status,Fail_Reason
header.start = "Timestamp"
header.linecount = 1
header.tokens = tokenize, -1, "|"
body.tokens = tokenize, 0, "|"
props.conf
[hostname]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
pulldown_type = 1
BREAK_ONLY_BEFORE=(?!)
MAX_EVENTS=100000
The search that I am using is:
index="customindex" | multikv conf=custommultikv forceheader=1
The problem is that with or without * header.tokens * header.start * props.conf or * forceheader=1 the events are split but the fields are not created so the following, for example, returns no events:
index="mswm_bpm_all_prod" splunk_server=dd894c3n12-9002 | multikv conf=ebcmultikv forceheader=1 | table Service_Description
The following configuration worked for me:
[multitest]
header.start = "Timestamp"
header.linecount = 1
header.tokens = _tokenize_, -1," | "
body.tokens = _tokenize_, -1, " | "
This will remove the hearder automatically from the results and then split each line into a separate event when using the following search sourcetype="multikvtest" | multikv conf=multitest
If you want to filter the results further by field value, you will only be able to do so by piping to the search command
Example:
Thank you. Adding the spaces before and after the pipe (delimiter) and changing -1 to 0 in body.tokens worked like a charm.
It was a revelation though, that the fields do not show up in the field picker until you use another command after multikv like table etc (they show up imediately for bult-in multikv support like ps and top etc).