topic Re: How to set up props.conf so that each source is a single event in Getting Data In

How to set up props.conf so that each source is a single event

mux — Thu, 30 Jan 2014 14:12:23 GMT

Currently we are monitoring a directory with batch jobs logs in it and it is not breaking correctly.

[monitor:///home/prod/department/interface/joblogs/*]
sourcetype = joblogs
index = finance
disabled = false

And I am trying to break the files up here.

[joblogs]
BREAK_ONLY_BEFORE = (E r r o r\s+L o g|J o b\s+L o g)
NO_BINARY_CHECK=1
SHOULD_LINEMERGE = True

We are setting the source to the log file name so each different file has a unique source name. I do not want to change the sourcetype name or split the sourcetypes by file. Ideally I am looking to break each file into one event based on the source so that each different file would be 1 event within the index. It is currently breaking the logs into several events.

Re: How to set up props.conf so that each source is a single event

kristian_kolb — Thu, 30 Jan 2014 16:32:22 GMT

Could you post a few samples? And also, indicate
at what points the file is currently being broken.

Re: How to set up props.conf so that each source is a single event

Lowell — Thu, 30 Jan 2014 16:40:54 GMT

Does the word "Error Log" really have spaces between every letter like that? Literally "E r r o r L o g" or is it possible that you have some kind of character set issue? (like UTF-16 or something). That's a long-shot, but it may be relevant.

Re: How to set up props.conf so that each source is a single event

lguinn2 — Thu, 30 Jan 2014 16:54:03 GMT

No problem, do this

[joblogs]
NO_BINARY_CHECK=1
SHOULD_LINEMERGE = false
LINE_BREAKER = ((*FAIL))
TRUNCATE = 99999999

I used this answer for the info.

Re: How to set up props.conf so that each source is a single event

mux — Thu, 30 Jan 2014 17:12:57 GMT

Thank you I believe this will do the trick, I will double check in the AM after the batch jobs have run tonight.