Getting Data In

How to forward to Splunk cloud from AWS and on prem?

FraserC1
Path Finder

Hi,

Our setup is as follows:

  • Managed Splunk Cloud instance
  • Heavy Forwader (on-prem)
  • Syslog server (on-prem)

Our on prem servers have universal forwarders on them and forwarder to the HF which then sends to splunk cloud.

We are starting to spin up EC2 instances in AWS and want to do the same monitoring, so UF installed on the instance and forwarding to splunk cloud.

My question is how do we do this?
It seems a bit daft to send our logs back to our on-premis HF to then send to the cloud.

So should we create a HF in our AWS VPC and point all our ec2 instances towards that?

How has everyone else tackled this issue?

Cheers,
Fraser

Labels (1)
0 Karma

neerajs_81
Builder

For Option # 1 " If you want to use UF then you can directly send data to Splunk cloud ...",  what config exactly do we need to put in the outputs.conf of the UF to make it fwd to Cloud instance ?    All  we have in the URL/hostname of the managed Search head instance  .  Are you saying we need to put this hostname in the outputs.conf and Splunk will do the rest and as in sending the data from SH to its indexer tier ?

0 Karma

dhihoriya_splun
Splunk Employee
Splunk Employee

Hi @FraserC1

Option 1:
If you want to use UF then you can directly send data to Splunk cloud but the UF will not parse the data as it will only forward the data to the Splunk cloud indexer and for that, you have to just put the config in outputs.conf of UF and in this case parsing and indxing will be done by Splunk cloud indexer.

Option2:
If you will use HF only then it will be a better option, As it will parse the data and will send it to Splunk cloud for indexing and in this case we don't have to use UF and need to put the same config in outputs.conf as per option1.

0 Karma

neerajs_81
Builder

For Option # 1 " If you want to use UF then you can directly send data to Splunk cloud ...",  what config exactly do we need to put in the outputs.conf of the UF to make it fwd to Cloud instance ?    All  we have in the URL/hostname of the managed Search head instance  .  Are you saying we need to put this hostname in the outputs.conf and Splunk will do the rest and as in sending the data from SH to its indexer tier ?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You should be able to set up a UF in AWS the same way you did for your on-prem HF. If doesn't matter if it's UF or HF as the outputs.conf settings are the same. You will, however, need to check your AWS security groups to make sure the UF is allowed to connect to Splunk Cloud.

As an aside, are you sure you need the intermediate HF in your on-prem space? It's a bottleneck, single point of failure, and impairs performance.

---
If this reply helps you, Karma would be appreciated.
0 Karma

FraserC1
Path Finder

Okay I was thinking we can just use a UF instead.

I agree about the bottleneck and single point of failure but we were told it is best practice to point towards an HF before sending to the cloud.
It is also where all our SaaS add-ons are configured so we do need it in some capacity.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...