Dashboards & Visualizations

multikv custom scripted input not mapping headers

YisroelB
Explorer

I have a custom scripted input generating output as follows which is being sucessfully indexed:

Timestamp | Service_Description | Service_Name | Service_URI | Response_Time | HTTP_Code | Status | Fail_Reason
2013-10-27T12-46-24 | abc.domain.com/Account/GetAPS | GetAPS | bus.domain.com:0000/Site/GetAPS.svc | 0.240 | 200 | Pass |
2013-10-27T12-46-25 | abc.domain.com/Account/GetAPS2 | GetAPS2 | bus.domain.com:0000/Site/GetAPS2.svc | 0.340 | 200 | Pass |
2013-10-27T12-46-26 | abc.domain.com/Account/GetAPS3 | GetAPS3 | bus.domain.com:0000/Site/GetAPS3.svc | 0.440 | 200 | Pass |
2013-10-27T12-46-27 | abc.domain.com/Account/GetAPS4 | GetAPS4 | bus.domain.com:0000/Site/GetAPS4.svc | 0.540 | 200 | Pass |

Based on advice from the answer to multikvconf for data with pipe delimeter my config is as follows:

inputs.conf

[script://./bin/customscript.sh]
interval = 600
source = customscript.sh
index = customindex
sourcetype = custominput
disabled = 0

multikv.conf

[custommultikv]
header.tokens = Timestamp,Service_Description,Service_Name,Service_URI,Response_Time,HTTP_Code,Status,Fail_Reason
header.start = "Timestamp"
header.linecount = 1
header.tokens = tokenize, -1, "|"
body.tokens = tokenize, 0, "|"

props.conf

[hostname]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
pulldown_type = 1
BREAK_ONLY_BEFORE=(?!)
MAX_EVENTS=100000

The search that I am using is:

index="customindex" | multikv conf=custommultikv forceheader=1

The problem is that with or without * header.tokens * header.start * props.conf or * forceheader=1 the events are split but the fields are not created so the following, for example, returns no events:

index="mswm_bpm_all_prod" splunk_server=dd894c3n12-9002 | multikv conf=ebcmultikv forceheader=1 | table Service_Description

Tags (2)
0 Karma

RicoSuave
Builder

The following configuration worked for me:

[multitest] 
header.start = "Timestamp" 
header.linecount = 1 
header.tokens = _tokenize_, -1," | " 
body.tokens = _tokenize_, -1, " | " 

This will remove the hearder automatically from the results and then split each line into a separate event when using the following search sourcetype="multikvtest" | multikv conf=multitest

If you want to filter the results further by field value, you will only be able to do so by piping to the search command

Example:

| multikv conf=multitest | search field=value

YisroelB
Explorer

Thank you. Adding the spaces before and after the pipe (delimiter) and changing -1 to 0 in body.tokens worked like a charm.

It was a revelation though, that the fields do not show up in the field picker until you use another command after multikv like table etc (they show up imediately for bult-in multikv support like ps and top etc).

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...