Getting Data In

multikv.conf for data with pipe delimeter

Parameshwara
Path Finder

multikv.conf

[testmultikv]
pre.linecount    = 1
header.linecount = 1
header.tokens    = _tokenize_, -1, "1"
body.tokens      = _tokenize_, 0, "1"

Sample data file:

School|Month|Subject_A_Score|Subject_B_Score
SchoolA|January|0|20
SchoolB|January|50|99
SchoolC|January|11|88
...

Search:

index=xxx | multikv conf=testmultikv | table School Month Subject...

Search results does not pick up the defined fields. What is missing in the configuration file?

Tags (2)

andreas
Explorer

A working configuration for multikv.conf is:

[testmultikv]
header.linecount = 1
header.tokens = _tokenize_, -1, "|"
body.tokens = _tokenize_, 0, "|"

(no pre section, and "|" (pipe) instead of "1" (one))

And you have to ensure that your whole data file will be indexed as one event, because multikv works on "table-formatted events".

The whole "table"

School|Month|Subject_A_Score|Subject_B_Score
SchoolA|January|0|20
SchoolB|January|50|99
SchoolC|January|11|88
...

has to be one event.

You can do this by setting BREAK_ONLY_BEFORE for the sourcetype to a pattern that never match like (?!) and SHOULD_LINEMERGE to true. Depending on your file size (number of lines) you probably also need to increase MAX_EVENTS (default is 500).

Sample sourcetype definition in props.conf:

[schooldata]
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=(?!)
MAX_EVENTS=10000
NO_BINARY_CHECK=1

Parameshwara
Path Finder

School|Month|SubjectA|SubjectB is the first line in my data and there are no empty lines before.

my inputs.conf

[monitor:///opt/testdata/multikv]
disabled = false
followTail = 0
host = datav1
index = multikv
sourcetype = datav1

"index=multikv sourcetype=datav1" gives only 1 event which is the 4 lines of my data (1 header, 3 data)
 

"index=multikv sourcetype=datav1 | multikv conf=testmultikv | table School Month SubjectA SubjectB" gives "No results found"
 

I'm running Splunk 5.0.2, build 149561.

0 Karma

andreas
Explorer

Is this line
School|Month|SubjectA|SubjectB
the first line in your data file? No empty line(s) before?

How does your inputs.conf look like? (Did you use sourcetype=datav1?)

What is the output of the search
index=multikv sourcetype=datav1
(should be one event containing all the data)

and what is the output of
index=multikv sourcetype=datav1 | multikv conf=testmultikv | table School Month SubjectA SubjectB

Which Splunk version are you using?

0 Karma

Parameshwara
Path Finder

multikv.conf:

[testmultikv]
header.linecount = 1
header.tokens = _tokenize_, -1, "|"
body.tokens = _tokenize_, 0, "|"

props.conf

[datav1]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
pulldown_type = 1
BREAK_ONLY_BEFORE=(?!)
MAX_EVENTS=100000

my data is:

School|Month|SubjectA|SubjectB
SchoolZ|January|0|20
SchoolX|January|50|99
SchoolM|January|11|88

my search is:

index=multikv | multikv conf=testmultikv

What I get is the first data row becomes the field. In 'interesting field' one items appears as SchoolZ|January|0|20.

0 Karma
Get Updates on the Splunk Community!

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...