Getting Data In

How to send events over long-lived TCP connection?

giraffe
Explorer

Our application generates many small monitoring events. We are feeding these to Splunk by TCP. We would like to keep a single connection open between each of our processes and the Splunk TCP input. However, when we do this the events do not appear when sent over the TCP connection, but only when the TCP stream is closed. Then they all appear together as a single event.

How do we configure the TCP input to generate one event per line as they stream in, instead of treating all data received over the connection as a single event?

Tags (2)
1 Solution

Ayn
Legend

You need to tell Splunk when to break the incoming data into new events. Right now Splunk doesn't find anything to break the events on, so it waits until the end of the input stream. By default Splunk will create a new event when it encounters something it recognizes as a timestamp, or when the maximum defined number of lines in an event has been reached (default 256). These settings are specified in props.conf (docs: http://docs.splunk.com/Documentation/Splunk/latest/admin/propsconf - the settings you're looking for are MAX_EVENTS and BREAK_ONLY_BEFORE_DATE).

Could you paste some sample events? That way we could help with coming up with working settings for breaking your log stream into individual events.

View solution in original post

0 Karma

Vladimir
Path Finder

You can add timestamp/unix time to the begging of event line. For example:

1320663001,event1
1320663002,event2

In this case splunk will divide events

unix time is a "date +%s" command in linux systems, or $sTimeStamp = get-date ((Get-Date).touniversaltime()) -uformat "%s" in PowerShell

Ayn
Legend

You need to tell Splunk when to break the incoming data into new events. Right now Splunk doesn't find anything to break the events on, so it waits until the end of the input stream. By default Splunk will create a new event when it encounters something it recognizes as a timestamp, or when the maximum defined number of lines in an event has been reached (default 256). These settings are specified in props.conf (docs: http://docs.splunk.com/Documentation/Splunk/latest/admin/propsconf - the settings you're looking for are MAX_EVENTS and BREAK_ONLY_BEFORE_DATE).

Could you paste some sample events? That way we could help with coming up with working settings for breaking your log stream into individual events.

0 Karma

giraffe
Explorer
0 Karma

giraffe
Explorer

I've found a props.conf in etc/system/defaults. Can I create one in /etc/system/local? And if so, will the definitions in there be merged with the one in etc/system/defaults or will the defaults be ignored?

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...