I have a scenario where the events should not be split, but after trying a lot of options it still seems to be not working. Its an health check log,
Relay is RUNNING - PID 123123 deal publisher is RUNNING - PID 80345 C ADAPTER is RUNNING - PID 99342 M ADAPTER is RUNNING - PID 662521 SMA is RUNNING - PID 12321321
I just want all them to be an single event, but it keeps on splitting in different ways.
TRUNCATE = 0
MAX_EVENTS = 5
I have tried SHOULD_LINEMERGE = true as well, but doesnt seems to be working. I just want no to break and all the 5 lines get indexed as a single event.
Thanks in advance.
just an update on this issue(just in case anyone has the same issue) the events were being written in the log file with a delay, by default the time_before_close is 3 seconds. I had to change this to 5 seconds and also also set the multiline_event_extra_waittime=true in inputs.conf
time_before_close = 5 multiline_event_extra_waittime = true
From Splunk documentation on these parameters,
time_before_close = <integer>
* Modtime delta required before Splunk can close a file on EOF.
* Tells the system not to close files that have been updated in past
* Defaults to 3.
multiline_event_extra_waittime = [true|false]
*By default, Splunk Enterprise sends an event delimiter when (1) it reaches EOF of a file it monitors and (2) the last char it reads is a newline.
*In some cases, it takes time for all lines of a multiple-line event to arrive.
*Set to true to delay sending an event delimiter until the time that Splunk Enterprise closes the file, as defined by the time_before_close attribute, to allow all event lines to arrive.
*Default to false.
I am quite certain that this configuration will work. It is highly likely that problem is not in the configuration settings but in your delpoyment or testing approach. Answer these questions.
1: Are you overriding/resetting/modifying the sourcetype using a setting in
transforms.conf and if so, are you using the original sourcetype or the modified one (you MUST use the original one)?
2: Have you deployed this
props.conf file to the FIRST FULL INSTANCE of splunk (e.g. HF/IF or Indexers)?
3: Have you restarted all Splunk instances on those nodes?
4: Are you using
_index_earliest=-5m in your search SPL when you test to make sure that you are only looking at newly-indexed events?
It's a distributed setup with 2 SH and 4 IDX
1. No transforms.conf
2. I have deployed it in the indexers through the deployment server
3. Yes, I have restarted all the splunk instances
4. I'm using last 15 min in the search time range picker.
I have tried with
All time and
_index_earliest=-5m but still the same result. 5 lines in the above sample is getting split into 5 different events (with same time stamp). All those 5 lines should be a single event and not split at all
if you set SHOULD_LINEMERGE=true you need another option
you could set
BREAK_ONLY_BEFORE = ^Relay. If you know that your events always start like this
Tested it in splunk data upload, and I actually have the problem that splunk is not splitting the events until "BREAK_ONLY_BEFORE = ^Relay" is set. Not that some other conf is messing with your settings.
Thanks for the response.
I have tried the below props.conf and still its not working, the events are getting split differently each time.
SHOULD_LINEMERGE=true BREAK_ONLY_BEFORE = ^Relay
The events start with Relay as shown in the sample, I have tried adding TRUNCATE=0 and MAX_EVENTS = 5 along with the above configs but doesn't make any difference