topic Re: How can I make splunk universal forwarders treat each newline as a separate event in Getting Data In

How can I make splunk universal forwarders treat each newline as a separate event

ilanz — Mon, 26 Mar 2012 15:49:16 GMT

Hi, my setup is two nodes, each has a Splunk Universal Forwarder which reads a logs directory and sends those logs to a central Splunk indexer.

Let's say I have a log file that looks like this:

4:01:02.316 PM 2012-03-26 16:01:02,3[CLIENT,ALL][Thread:162] INFO - {"JSON1":"hello"}

5:02:02.316 PM 2012-03-28 17:00:02,3[CLIENT,ALL][Thread:162] INFO - {"JSON2":"goodbye"}

In the case above, each line if forwarded as a separate event,

Now, let's say I change my log file format, I'll remove the prefixed timestamps to make it loo like this:

{"JSON1":"hello"}

{"JSON2":"goodbye"}

In that case, splunk treats it as a single event that is if I perform a search I actually get:

{"JSON1":"hello"}{"JSON2":"goodbye"}

How can I configure splunk correctly so it will treat each json entry in my log as a separate event, given my setup usage of universal forwarders.

Thanks beforehand!

Re: How can I make splunk universal forwarders treat each newline as a separate event

yannK — Mon, 26 Mar 2012 16:55:22 GMT

Hi, if your events are :
- really separated by a new line (in the file)
- or send to splunk one after the other (syslog by example)

then the solution is to use a sourcetype that define the events are "singleline events"
see SHOULD_LINEMERGE in http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Indexmulti-lineevents

example
inputs.conf [monitor:///var/log/json] sourcetype=myjson

props.conf (on the indexer)
[myjson]
SHOULD_LINEMERGE= false

Re: How can I make splunk universal forwarders treat each newline as a separate event

ilanz — Tue, 27 Mar 2012 07:05:54 GMT

Thanks for your help!