topic Re: How does Splunk divide events? in Getting Data In

How does Splunk divide events?

sylim_splunk — Tue, 27 Jan 2015 23:30:36 GMT

There is an application putting SOAP logs, request and response, in a small delay of 0 ~10 secs into the log file - I want this to be indexed as a single event, but Splunk indexes it into 2 events - request and response separately. This doesn't always happen, but quite frequently. How can I achieve this?

Re: How does Splunk divide events?

sylim_splunk — Tue, 27 Jan 2015 23:35:13 GMT

This is not what Splunk can support. In order to get data indexed in a timely way, we presume an event is complete IF:
- The last character of the file is a newline
- The size of the file is not a multiple of 512 bytes
So if your app writes out a complete line and the stops, Splunk will break it.

Re: How does Splunk divide events?

aakwah — Tue, 27 Jan 2015 23:40:42 GMT

Hello,

To do this put the following 2 lines in forwarder's props.conf:

SHOULD_LINEMERGE=true
MAX_EVENTS = 2

Regards,
Ahmed Elakwah

Re: How does Splunk divide events?

sylim_splunk — Tue, 29 Sep 2020 16:43:21 GMT

If there's a gap more than 3 seconds between each data flush, either by app or by OS the event will be indexed separately because of this behavior. To work around it increase the value of time_before_close to, like 10 secs.

[monitor://D:\LogFiles\Test]
time_before_close = 10