Dashboards & Visualizations

multikv custom scripted input not mapping headers

YisroelB
Explorer

I have a custom scripted input generating output as follows which is being sucessfully indexed:

Timestamp | Service_Description | Service_Name | Service_URI | Response_Time | HTTP_Code | Status | Fail_Reason
2013-10-27T12-46-24 | abc.domain.com/Account/GetAPS | GetAPS | bus.domain.com:0000/Site/GetAPS.svc | 0.240 | 200 | Pass |
2013-10-27T12-46-25 | abc.domain.com/Account/GetAPS2 | GetAPS2 | bus.domain.com:0000/Site/GetAPS2.svc | 0.340 | 200 | Pass |
2013-10-27T12-46-26 | abc.domain.com/Account/GetAPS3 | GetAPS3 | bus.domain.com:0000/Site/GetAPS3.svc | 0.440 | 200 | Pass |
2013-10-27T12-46-27 | abc.domain.com/Account/GetAPS4 | GetAPS4 | bus.domain.com:0000/Site/GetAPS4.svc | 0.540 | 200 | Pass |

Based on advice from the answer to multikvconf for data with pipe delimeter my config is as follows:

inputs.conf

[script://./bin/customscript.sh]
interval = 600
source = customscript.sh
index = customindex
sourcetype = custominput
disabled = 0

multikv.conf

[custommultikv]
header.tokens = Timestamp,Service_Description,Service_Name,Service_URI,Response_Time,HTTP_Code,Status,Fail_Reason
header.start = "Timestamp"
header.linecount = 1
header.tokens = tokenize, -1, "|"
body.tokens = tokenize, 0, "|"

props.conf

[hostname]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
pulldown_type = 1
BREAK_ONLY_BEFORE=(?!)
MAX_EVENTS=100000

The search that I am using is:

index="customindex" | multikv conf=custommultikv forceheader=1

The problem is that with or without * header.tokens * header.start * props.conf or * forceheader=1 the events are split but the fields are not created so the following, for example, returns no events:

index="mswm_bpm_all_prod" splunk_server=dd894c3n12-9002 | multikv conf=ebcmultikv forceheader=1 | table Service_Description

Tags (2)
0 Karma

RicoSuave
Builder

The following configuration worked for me:

[multitest] 
header.start = "Timestamp" 
header.linecount = 1 
header.tokens = _tokenize_, -1," | " 
body.tokens = _tokenize_, -1, " | " 

This will remove the hearder automatically from the results and then split each line into a separate event when using the following search sourcetype="multikvtest" | multikv conf=multitest

If you want to filter the results further by field value, you will only be able to do so by piping to the search command

Example:

| multikv conf=multitest | search field=value

YisroelB
Explorer

Thank you. Adding the spaces before and after the pipe (delimiter) and changing -1 to 0 in body.tokens worked like a charm.

It was a revelation though, that the fields do not show up in the field picker until you use another command after multikv like table etc (they show up imediately for bult-in multikv support like ps and top etc).

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...