Sending Events from micro service to Customer's Splunk environment

[himanshuSharma]

We are looking to integrate Splunk SIEM with our microservice, we are looking to send events from service to Splunk and then configure alerts based on eventType. 

As we understand there are 2 approaches Universal Log forwarder and HTTP Event collector. We are inclining more towards using HEC as it has the ability to send ack for events as well and challenge with Universal Log forwarder is that it needs to be managed by customer where Splunk will be running and volume of the events is also not that much.
Can someone help us in understanding cost involved in both approaches and scaling of HEC is number of events increases due to a spike.
Also should we go with building a Technology Add-on or app which can be used along with Splunk Enterprise Security.
We want to implement this for Enterprise as well as Cloud.

#SplunkAddOnbuilder

[himanshuSharma]

Thanks @richgalloway  for your inputs,
Does the volume of data being sent to Splunk helps in determining which method to use between HEC and UF
For our use case we plan to send events which has associated information (a json ~400 bytes.) and we may not be sending more than 5000 such events/day.
You also mentioned about the client to get Acks for events sent via HEC and we do plan to have that.
Based on the volume and our use case do you suggest we go with HEC?
Also , while building and add-on is it possible to add a query which will identify specific events as alerts and ship that with add-on which customer can install in their Splunk setup?

[richgalloway]

Both HEC and the UF support ack.  While HEC does support higher volume, but both have good throughput.  We'd need to know more about how much data you intend to send to determine which is better.

The data send to HEC has to be in a particular format and ACKs must be checked periodically, so there must be a client that has to be maintained by the customer.

There is no additional cost (from Splunk) for either approach.

Yes, you will want an add-on, especially if you use the UF (but may also be needed for HEC).  The add-on ensures the data is onboarded properly and defines the fields to be extracted.