Getting Data In

Is it possible to selectively line merge syslog-ng truncated events?

JSkier
Communicator

I have a syslog server receiving and filtering into files and then forwarding data to my indexers. One source uses udp, and sometimes a single event gets truncated into typically two lines.

2017-05-24T12:40:19.694287-02:00 server1 blah: [timestamp] ..... [more stu -TRUNCATES-
2017-05-24T12:40:19.694287-02:00 server1 blah: ff this is a continuation of one event] status warn [end]

Is it possible to have Splunk selectively line merge when this happens? For example, most of events are one line and below the message limit, but occasionally these two liner snow flakes come along that need to be merged somehow.

I have looked at the sending appliance and it cannot up the msg size. Also, it can also only send out syslog via UDP so it is bound to the protocol limitation. I haven't seen much for syslog-ng filtering that can address this.

mjeffery_splunk
Splunk Employee
Splunk Employee

Can your network appliance be configured to send syslog over TCP instead?

Also, who makes this network appliance?

0 Karma

JSkier
Communicator

It's a LoadBalancer.org appliance; uses rsyslog to send.

TCP still truncates, the sending from rsyslog does not seem to honor the message size setting of rsyslog.

0 Karma
Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...