Getting Data In

What is the best and fastest way to transport .log files by syslog?

yangban
Explorer

Hi we are trying to transport several .log files to a Forwarder by syslog.

We used some bash scripts to do so, but it was just way too slow for us.(nearly 1MB/min)

Is there anyone know a faster way?

Thanks.

0 Karma
1 Solution

FrankVl
Ultra Champion

You might want to take inspiration from this .conf17 talk: https://conf.splunk.com/conf-online.html?search=FN122172#/

Which uses a simple python script to send data to a HEC receiver.

Alternatively:
install a UF on the system where the log files are.
Or write a faster bash script (perhaps you can share what you had now).
Or configure an rsyslog/syslog-ng daemon to read from those files and then forward to a syslog receiver on the Forwarder.

View solution in original post

0 Karma

FrankVl
Ultra Champion

You might want to take inspiration from this .conf17 talk: https://conf.splunk.com/conf-online.html?search=FN122172#/

Which uses a simple python script to send data to a HEC receiver.

Alternatively:
install a UF on the system where the log files are.
Or write a faster bash script (perhaps you can share what you had now).
Or configure an rsyslog/syslog-ng daemon to read from those files and then forward to a syslog receiver on the Forwarder.

0 Karma

yangban
Explorer

Thanks! I might try HEC!

0 Karma
Get Updates on the Splunk Community!

The All New Performance Insights for Splunk

Splunk gives you amazing tools to analyze system data and make business-critical decisions, react to issues, ...

Good Sourcetype Naming

When it comes to getting data in, one of the earliest decisions made is what to use as a sourcetype. Often, ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...