topic How to forward to Splunk cloud from AWS and on prem? in Getting Data In

How to forward to Splunk cloud from AWS and on prem?

FraserC1 — Fri, 12 Aug 2022 14:59:02 GMT

Hi,

Our setup is as follows:

Managed Splunk Cloud instance
Heavy Forwader (on-prem)
Syslog server (on-prem)

Our on prem servers have universal forwarders on them and forwarder to the HF which then sends to splunk cloud.

We are starting to spin up EC2 instances in AWS and want to do the same monitoring, so UF installed on the instance and forwarding to splunk cloud.

My question is how do we do this?
It seems a bit daft to send our logs back to our on-premis HF to then send to the cloud.

So should we create a HF in our AWS VPC and point all our ec2 instances towards that?

How has everyone else tackled this issue?

Cheers,
Fraser

Re: Forwarding to Splunk cloud from AWS and on prem

richgalloway — Fri, 02 Aug 2019 13:28:59 GMT

You should be able to set up a UF in AWS the same way you did for your on-prem HF. If doesn't matter if it's UF or HF as the outputs.conf settings are the same. You will, however, need to check your AWS security groups to make sure the UF is allowed to connect to Splunk Cloud.

As an aside, are you sure you need the intermediate HF in your on-prem space? It's a bottleneck, single point of failure, and impairs performance.

Re: Forwarding to Splunk cloud from AWS and on prem

FraserC1 — Fri, 02 Aug 2019 13:50:23 GMT

Okay I was thinking we can just use a UF instead.

I agree about the bottleneck and single point of failure but we were told it is best practice to point towards an HF before sending to the cloud.
It is also where all our SaaS add-ons are configured so we do need it in some capacity.

Re: Forwarding to Splunk cloud from AWS and on prem

dhihoriya_splun — Tue, 13 Aug 2019 10:34:32 GMT

Hi @FraserC1

Option 1:
If you want to use UF then you can directly send data to Splunk cloud but the UF will not parse the data as it will only forward the data to the Splunk cloud indexer and for that, you have to just put the config in outputs.conf of UF and in this case parsing and indxing will be done by Splunk cloud indexer.

Option2:
If you will use HF only then it will be a better option, As it will parse the data and will send it to Splunk cloud for indexing and in this case we don't have to use UF and need to put the same config in outputs.conf as per option1.

Re: Forwarding to Splunk cloud from AWS and on prem

neerajs_81 — Fri, 12 Aug 2022 06:53:56 GMT

For Option # 1 " If you want to use UF then you can directly send data to Splunk cloud ...", what config exactly do we need to put in the outputs.conf of the UF to make it fwd to Cloud instance ? All we have in the URL/hostname of the managed Search head instance . Are you saying we need to put this hostname in the outputs.conf and Splunk will do the rest and as in sending the data from SH to its indexer tier ?

Re: Forwarding to Splunk cloud from AWS and on prem

neerajs_81 — Fri, 12 Aug 2022 06:55:10 GMT