Getting Data In

How do I troubleshoot linebreak / linemerge issues?

Richard_
Engager

I'm having a problem where multiple events are getting combined into a single event and I haven't been able to figure out how to fix it.

For example, the following two events are being listed as one event by splunk:
msg,server,City,recruit,0.12032890319824,code=0&desc=585,0,585,2013-06-06 15:45:58,10.80.74.124
msg,client,Hero,train,805012,[{"mod":"Hero","cash":0,"gold":1800,"cardnumber":0,"heroid":46,"type":3,"act":"train"}],2013-06-06 15:45:58,10.80.74.124

I have a splunk forwarder sending data to my main splunk indexer. I'm using apps to specify the inputs/outputs for the forwarder.
The sourcetype for the data in question is kotr_logknight

On the indexer I have created a props.conf that contains:
[kotr_logknight]
SHOULD_LINEMERGE = false

My understanding is that this should disable line merging so that my events shouldn't get combined. However, it doesn't seem to affect the behavior at all.
(I also tried putting props.conf on the forwarder and in with the app, and neither of those seemed to make any difference either)

How can I track down what is happening during indexing to understand why the configuration setting doesn't seem to be doing what I expect?

Tags (2)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Use a sample of data and import it into the data preview.
http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/Overviewofdatapreview

Also the props,conf with the sourcetype definition has to be on the instance parsing the events (indexers, or heavy forwarders if any)

View solution in original post

skoelpin
SplunkTrust
SplunkTrust

You're correct about making changes in the props.conf on the indexer. But you will need to write a regular expression to successfully break those lines so Splunk will see them as a new event

^msg\,\w+

0 Karma

yannK
Splunk Employee
Splunk Employee

Use a sample of data and import it into the data preview.
http://docs.splunk.com/Documentation/Splunk/5.0.3/Data/Overviewofdatapreview

Also the props,conf with the sourcetype definition has to be on the instance parsing the events (indexers, or heavy forwarders if any)

Richard_
Engager

That was super helpful. I've never added data that way so that I didn't know of the existance of the data previewer.

My problem ended up being a stupid mistake. I was changing prop.conf on the wrong machine. I thought I was on the indexer, but I wasn't.

Thanks!

Richfez
SplunkTrust
SplunkTrust

The GUI for this gives you a partially interactive way to test things. Being able to immediately see results as you change things was very useful.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...