Getting Data In

How do I troubleshoot linebreak / linemerge issues?

Richard_
Engager

I'm having a problem where multiple events are getting combined into a single event and I haven't been able to figure out how to fix it.

For example, the following two events are being listed as one event by splunk:
msg,server,City,recruit,0.12032890319824,code=0&desc=585,0,585,2013-06-06 15:45:58,10.80.74.124
msg,client,Hero,train,805012,[{"mod":"Hero","cash":0,"gold":1800,"cardnumber":0,"heroid":46,"type":3,"act":"train"}],2013-06-06 15:45:58,10.80.74.124

I have a splunk forwarder sending data to my main splunk indexer. I'm using apps to specify the inputs/outputs for the forwarder.
The sourcetype for the data in question is kotr_logknight

On the indexer I have created a props.conf that contains:
[kotr_logknight]
SHOULD_LINEMERGE = false

My understanding is that this should disable line merging so that my events shouldn't get combined. However, it doesn't seem to affect the behavior at all.
(I also tried putting props.conf on the forwarder and in with the app, and neither of those seemed to make any difference either)

How can I track down what is happening during indexing to understand why the configuration setting doesn't seem to be doing what I expect?

Tags (2)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Use a sample of data and import it into the data preview.
http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/Overviewofdatapreview

Also the props,conf with the sourcetype definition has to be on the instance parsing the events (indexers, or heavy forwarders if any)

View solution in original post

skoelpin
SplunkTrust
SplunkTrust

You're correct about making changes in the props.conf on the indexer. But you will need to write a regular expression to successfully break those lines so Splunk will see them as a new event

^msg\,\w+

0 Karma

yannK
Splunk Employee
Splunk Employee

Use a sample of data and import it into the data preview.
http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/Overviewofdatapreview

Also the props,conf with the sourcetype definition has to be on the instance parsing the events (indexers, or heavy forwarders if any)

Richard_
Engager

That was super helpful. I've never added data that way so that I didn't know of the existance of the data previewer.

My problem ended up being a stupid mistake. I was changing prop.conf on the wrong machine. I thought I was on the indexer, but I wasn't.

Thanks!

Richfez
SplunkTrust
SplunkTrust

The GUI for this gives you a partially interactive way to test things. Being able to immediately see results as you change things was very useful.

0 Karma
Get Updates on the Splunk Community!

Announcing General Availability of Splunk Incident Intelligence!

Digital transformation is real! Across industries, companies big and small are going through rapid digital ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...