Dashboards & Visualizations

multikv custom scripted input not mapping headers

YisroelB
Explorer

I have a custom scripted input generating output as follows which is being sucessfully indexed:

Timestamp | Service_Description | Service_Name | Service_URI | Response_Time | HTTP_Code | Status | Fail_Reason
2013-10-27T12-46-24 | abc.domain.com/Account/GetAPS | GetAPS | bus.domain.com:0000/Site/GetAPS.svc | 0.240 | 200 | Pass |
2013-10-27T12-46-25 | abc.domain.com/Account/GetAPS2 | GetAPS2 | bus.domain.com:0000/Site/GetAPS2.svc | 0.340 | 200 | Pass |
2013-10-27T12-46-26 | abc.domain.com/Account/GetAPS3 | GetAPS3 | bus.domain.com:0000/Site/GetAPS3.svc | 0.440 | 200 | Pass |
2013-10-27T12-46-27 | abc.domain.com/Account/GetAPS4 | GetAPS4 | bus.domain.com:0000/Site/GetAPS4.svc | 0.540 | 200 | Pass |

Based on advice from the answer to multikvconf for data with pipe delimeter my config is as follows:

inputs.conf

[script://./bin/customscript.sh]
interval = 600
source = customscript.sh
index = customindex
sourcetype = custominput
disabled = 0

multikv.conf

[custommultikv]
header.tokens = Timestamp,Service_Description,Service_Name,Service_URI,Response_Time,HTTP_Code,Status,Fail_Reason
header.start = "Timestamp"
header.linecount = 1
header.tokens = tokenize, -1, "|"
body.tokens = tokenize, 0, "|"

props.conf

[hostname]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
pulldown_type = 1
BREAK_ONLY_BEFORE=(?!)
MAX_EVENTS=100000

The search that I am using is:

index="customindex" | multikv conf=custommultikv forceheader=1

The problem is that with or without * header.tokens * header.start * props.conf or * forceheader=1 the events are split but the fields are not created so the following, for example, returns no events:

index="mswm_bpm_all_prod" splunk_server=dd894c3n12-9002 | multikv conf=ebcmultikv forceheader=1 | table Service_Description

Tags (2)
0 Karma

RicoSuave
Builder

The following configuration worked for me:

[multitest] 
header.start = "Timestamp" 
header.linecount = 1 
header.tokens = _tokenize_, -1," | " 
body.tokens = _tokenize_, -1, " | " 

This will remove the hearder automatically from the results and then split each line into a separate event when using the following search sourcetype="multikvtest" | multikv conf=multitest

If you want to filter the results further by field value, you will only be able to do so by piping to the search command

Example:

| multikv conf=multitest | search field=value

YisroelB
Explorer

Thank you. Adding the spaces before and after the pipe (delimiter) and changing -1 to 0 in body.tokens worked like a charm.

It was a revelation though, that the fields do not show up in the field picker until you use another command after multikv like table etc (they show up imediately for bult-in multikv support like ps and top etc).

0 Karma
Get Updates on the Splunk Community!

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...