Getting Data In
Provide Splunk Cloud feedback in this confidential UX survey by June 17
for a chance to win a $200 Amazon gift card!

Why the header from psv file sometimes being indexed and fields are not extracted while in most times it is ok

mlevsh
Builder

One of our teams on-boards psv logs and while the data on-boarded correctly in most case, sometimes the header is not recognized and field extraction is not happening. 
props.conf:

[status_psv]
INDEXED_EXTRACTIONS = PSV
KV_MODE = none
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
HEADER_FIELD_LINE_NUMBER = 1
TIMESTAMP_FIELDS = rqid
TIME_FORMAT = %s%6Q
MAX_DAYS_HENCE = 5


what are the possible reasons for some logs ST being ignored

Labels (3)
Tags (2)
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Can you be certain that there always is a header line and that it is always line 1? Have you tried to set HEADER_FIELD_LINE_NUMBER to '0' to let Splunk figure it out?

Also, is it possible that the file is being overwritten vs. being rolled while it is read?

0 Karma

mlevsh
Builder

@s2_splunk 

Thank you for you suggestions!

Majority (99.9%) of logs have header set as a 1st line and this Sourcetype works for other logs.
What makes this issue harder to troubleshoot that  the files are deleted after they ingested.
We were not successful to catch the actual log file so far.  the team will try HEADER_FIELD_LINE_NUMBER = 0 (which is default anyway) , but they want to troubleshoot without this change first

 

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!