Procedure to forward data from Cloud Splunk to on-prem splunk?

[ansif]

Use case:

Need to forward events from one Splunk to another splunk for which I am using HEC.But what is the procedure if events forwarding Splunk (source of HEC) is in cloud?

Do I need to engage Splunk support to open port,allow HEC (custom app) installation and etc? Please help.

[ansif]

Raised a support case and they suggested to install splunk stream app. This is exactly what I got:

Yes interaction with Splunk cloud to On-prem is possible, we need to use stream app for this purpose. Please refer below splunk document to know how it works and verify whether it suites your requirement. However, based on how this work you might need to architect your environment to achieve.

https://docs.splunk.com/Documentation/StreamApp/7.1.3/DeployStreamApp/DeployStreaminSplunkCloud
https://docs.splunk.com/Documentation/StreamApp/7.1.3/DeployStreamApp/SetupStream
https://docs.splunk.com/Documentation/StreamApp/7.1.3/DeployStreamApp/ConfigureStreamForwarder

Similar way I can use HEC to send data right?

[amiracle]

That is because you are using the REST API to interact between Stream (on the Splunk Cloud Search Head) and the endpoints. This is covered by the Splunk Cloud Service Definition (http://docs.splunk.com/Documentation/SplunkCloud/latest/Service/SplunkCloudservice) under API interaction. Sending data from Splunk Cloud via a forwarder is not currently permitted, which is what the original poster was asking to do.

[ansif]

@amiracle: So if I use alert action to send to HEC(onprem Splunk) it works right ? There is no special permission required from Splunk to allow network connection and all?

[woodcock]

This is a terrible idea and will cost you a bunch of extra money in AWS export surcharges. You would be WAY better off forwarding to both from the origination point, rather than pipelining from one to the next. If you have to pipeline it, then pipeline to on-prem first and then to Cloud.

[jkat54]

Yes, open a ticket with splunk support and ask them how to forward data out of the cloud to your on prem hec. Good luck!

[ansif]

Just checking here if anyone went through this.Thanks for your suggestion though.

[amiracle]

Splunk Cloud does not forward data out, so this request will be declined. Per the Splunk Cloud Service Description (https://docs.splunk.com/Documentation/SplunkCloud/latest/Service/SplunkCloudservice#Enterprise_Secur...) the only way to do "Data Egress from Splunk Cloud" is through DDSS (Data Export), REST API or for Splunk UBA.

[jkat54]

There’s only one way to know for sure, and some customers have enough money to make anything happen...

[khourihan_splun]

I downvoted this post because silly answer

[jkat54]

Can you suggest a better approach?

[amiracle]

If you are sending via HEC, why not send it to your on Prem HEC first, then forward it to Splunk Cloud using the index and forward capability of your on Prem Splunk indexers?

[ansif]

The thing is ,I need to forward events from Splunk cloud which is already set up with all alerts to another Splunk (on-prem) for auto ticketing.So this use case cant index data in onprem splunk before it goes to splunk cloud.

[joshd]

Why not just use the on-prem instance as a hybrid search head then no need to forward the data?