Getting Data In

What is the best way to send data to Splunk HTTP Event collector? UDP vs forwarder?

New Member

Hi,
Can someone please help guide me based on experience? What is the best mechanism to stream data to Splunk? As part of our organization we have built a custom logger service that can make a REST call (internally this service would then use HTTP event collector) and send data to Splunk. Some of these apps had issues in Splunk HTTP collector with JavaScript/typescript. The constraint with forwarder is that many of these apps have multiple clients wherein it might be difficult to achieve a Forwarder setup. Lastly, with UDP we have seen few reviews that show event loss issues.

Thanks

0 Karma

Super Champion

The best possible option in large enterprises is to use a log aggregration layer and Splunk to read it from it.
eg.

  • use syslog (rsyslog/syslog-ng) to collect the data which comes as stream and log into a good directory structure
  • A good pattern to use is /myfilesystem/data/%port%/%fromhost-ip%/%priority%.log
  • And based on type of data and the requirements, you configure your splunk inputs.conf with relevant sourcetype and index
  • Ensure all data in Splunk have index-time settings configured correctly
0 Karma

Champion
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!