topic splunk not parsing txt file from forwarder correctly in Getting Data In

splunk not parsing txt file from forwarder correctly

bcusick — Thu, 10 Jul 2014 18:03:46 GMT

Hi,

One of my forwarders is monitoring a directory where timestamped files populate every five minutes. The text output in the file includes a few-line header from the script along with all the results. The results are in the format "user ~ message"

For some reason, splunk is reading the ENTIRE file (source) as one log event. This is what's in my inputs file:

[monitor://D:\badPwdCount\logs\HIGH*.txt]

sourcetype = OVD_PW

disabled = false

SHOULD_LINEMERGE= false

crcSalt =

SOURCE is in between <> for crcSALT but it won't show up in comments. The issue is that Splunk is either picking up the entire file as an event, or a few events at a time as 1 event. Is a separate timestamp/unique identifier required for each line?

Re: splunk not parsing txt file from forwarder correctly

martin_mueller — Thu, 10 Jul 2014 19:03:37 GMT

What settings are you using for that sourcetype in props.conf on the indexer(s)?
Additionally, do post some sample data.

Re: splunk not parsing txt file from forwarder correctly

lguinn2 — Fri, 11 Jul 2014 00:10:08 GMT

SHOULD_LINEMERGE= false does not go in inputs.conf - it goes in props.conf like so

[OVD_PW]
SHOULD_LINEMERGE= false

I think that is the problem here.

Re: splunk not parsing txt file from forwarder correctly

ofrachon — Fri, 11 Jul 2014 15:12:19 GMT

You should tell Splunk how an event looks like in order to split the events correctly.

Could you post a sample of your data ?

Re: splunk not parsing txt file from forwarder correctly

bcusick — Mon, 04 Aug 2014 14:55:56 GMT

I added a field to the log records for "line number" which is basically "log record" and this worked like a charm. 🙂