Getting Data In

How does Splunk divide events?

sylim_splunk
Splunk Employee
Splunk Employee

There is an application putting SOAP logs, request and response, in a small delay of 0 ~10 secs into the log file - I want this to be indexed as a single event, but Splunk indexes it into 2 events - request and response separately. This doesn't always happen, but quite frequently. How can I achieve this?

Tags (2)
1 Solution

sylim_splunk
Splunk Employee
Splunk Employee

This is not what Splunk can support. In order to get data indexed in a timely way, we presume an event is complete IF:
- The last character of the file is a newline
- The size of the file is not a multiple of 512 bytes
So if your app writes out a complete line and the stops, Splunk will break it.

View solution in original post

aakwah
Builder

Hello,

To do this put the following 2 lines in forwarder's props.conf:

SHOULD_LINEMERGE=true
MAX_EVENTS = 2

Regards,
Ahmed Elakwah

0 Karma

sylim_splunk
Splunk Employee
Splunk Employee

This is not what Splunk can support. In order to get data indexed in a timely way, we presume an event is complete IF:
- The last character of the file is a newline
- The size of the file is not a multiple of 512 bytes
So if your app writes out a complete line and the stops, Splunk will break it.

sylim_splunk
Splunk Employee
Splunk Employee

If there's a gap more than 3 seconds between each data flush, either by app or by OS the event will be indexed separately because of this behavior. To work around it increase the value of time_before_close to, like 10 secs.

[monitor://D:\LogFiles\Test]
time_before_close = 10

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...