Getting Data In

Splunk HEC Integration. Where to enable? Deployment Server or Log Server?

lloydknight
Builder

Hello,

Here's our Splunk setup:

3 Indexers (not clustered)
1 Search Head/Deployment Server
1 Log Server (acts like Heavy Forwarder)

All Splunk instances are reporting to Deployment Server and indexes are app-based.
Log Server is basically a log server where other integrations that doesn't require Forwarder use this to dump their logs and we push an application from Deployment server to monitor the logs in the Log server.

Questions:
1. Where should I enable HEC?
This will be my first time using an HEC and I'm not sure if I'll enable it to Log server or Deployment Server.

2. If I will enable it to our Log Server, will there be API logs that will be dumped/generated somewhere that I can just monitor using deployed monitor apps from Deployment Server?

3. If I will enable it to our Deployment Server, my concern here is if it will generate actual logs which should be done in Log Server and if it will not generate logs on the server, what is the behavior of the API logs upon enabling it to Deployment server?

4. Will the API logs directly be indexed on the defined index? (i.e. index=api_logs)

5. High level explanation for Indexer Acknowledgement?

Can someone kindly answer all my 5 questions?

PS.
Already read this btw:
http://dev.splunk.com/view/event-collector/SP-CAAAE73

Much appreciated!

0 Karma
1 Solution

skalliger
Motivator

Hi,

you might want to read this about HEC: https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splu...

Should answer 1-4 (1. syslog, 2. I don't get your question, but the blog should clear that, 3. In general, you want to forward your splunk data to your indexers anyways, see link below, 4. yes).

Indexer acknowledgment works like this: a fowarder sends data to the indexer. An indexer has to acknowledge the data before the forwarder sends other data. In case of no ACK, the data will be re-sent. An indexer acknowledges data after it has been written to the disk.

Additional links:
https://docs.splunk.com/Documentation/Splunk/6.6.0/DistSearch/Forwardsearchheaddata
https://docs.splunk.com/Documentation/Splunk/6.6.0/Forwarding/Protectagainstlossofin-flightdata

Skalli

View solution in original post

0 Karma

skalliger
Motivator

Hi,

you might want to read this about HEC: https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregated-data-collection-in-splu...

Should answer 1-4 (1. syslog, 2. I don't get your question, but the blog should clear that, 3. In general, you want to forward your splunk data to your indexers anyways, see link below, 4. yes).

Indexer acknowledgment works like this: a fowarder sends data to the indexer. An indexer has to acknowledge the data before the forwarder sends other data. In case of no ACK, the data will be re-sent. An indexer acknowledges data after it has been written to the disk.

Additional links:
https://docs.splunk.com/Documentation/Splunk/6.6.0/DistSearch/Forwardsearchheaddata
https://docs.splunk.com/Documentation/Splunk/6.6.0/Forwarding/Protectagainstlossofin-flightdata

Skalli

0 Karma

lloydknight
Builder

big help. thanks

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...