How can I make splunk universal forwarders treat each newline as a separate event

[ilanz]

Hi, my setup is two nodes, each has a Splunk Universal Forwarder which reads a logs directory and sends those logs to a central Splunk indexer.

Let's say I have a log file that looks like this:

4:01:02.316 PM 2012-03-26 16:01:02,3[CLIENT,ALL][Thread:162] INFO - {"JSON1":"hello"}

5:02:02.316 PM 2012-03-28 17:00:02,3[CLIENT,ALL][Thread:162] INFO - {"JSON2":"goodbye"}

In the case above, each line if forwarded as a separate event,

Now, let's say I change my log file format, I'll remove the prefixed timestamps to make it loo like this:

{"JSON1":"hello"}

{"JSON2":"goodbye"}

In that case, splunk treats it as a single event that is if I perform a search I actually get:

{"JSON1":"hello"}{"JSON2":"goodbye"}

How can I configure splunk correctly so it will treat each json entry in my log as a separate event, given my setup usage of universal forwarders.

Thanks beforehand!

[yannK · Accepted solution]

Hi, if your events are :
- really separated by a new line (in the file)
- or send to splunk one after the other (syslog by example)

then the solution is to use a sourcetype that define the events are "singleline events"
see SHOULD_LINEMERGE in http://docs.splunk.com/Documentation/Splunk/4.3.1/Data/Indexmulti-lineevents

example

inputs.conf
[monitor:///var/log/json]
sourcetype=myjson

props.conf (on the indexer)
[myjson]
SHOULD_LINEMERGE= false

[ilanz]

Thanks for your help!