topic Re: spunk cloud- getting data in in Getting Data In

What are the steps of ingesting data into Splunk cloud?

cyber22 — Wed, 09 Feb 2022 18:34:37 GMT

Can someone walk me through the steps of ingesting data into splunk cloud. I have read the documentation but it gets confusing.

Re: spunk cloud- getting data in

richgalloway — Wed, 09 Feb 2022 18:16:24 GMT

Given there is a fair amount of documentation on the topic, it's not reasonable to expect full coverage of it here. Specific questions are more likely to get helpful answers.

There are many ways to get data into Splunk Cloud and which one to use will depend on the data source, your Splunk Cloud "experience", and other factors. Tell us more about what data want to ingest and we should be able to offer some tips on how to do it.

Re: spunk cloud- getting data in

cyber22 — Wed, 09 Feb 2022 18:26:16 GMT

firewall/network

windows logs

Re: spunk cloud- getting data in

PickleRick — Wed, 09 Feb 2022 20:33:00 GMT

With windows you typically set up a Universal Forwarder on monitored machine(s), define inputs for the event logs you want to pull, point your output to your cloud instance and that's pretty much it.

With the "network/firewall" whatever that means it can be more complicated. I assume that you'll be getting events from those devices by meand of syslog. So you need something to listen for syslog events and write them to splunk. Might be a simple Universal Forwarder (but using raw tcp/udp inputs on UF in production environment is not a best idea), might be SC4S instance, might be rsyslog or whatever you want. There are many different ways to handle syslog.

Re: What are the steps of ingesting data into Splunk cloud?

moliminous — Thu, 10 Feb 2022 00:29:57 GMT

Each data source is different, but I noticed you tagged this for Windows so I'll post this guide:
https://docs.splunk.com/Documentation/SplunkCloud/8.2.2112/Admin/WindowsGDI#Overview

Essentially you login to the Splunk Cloud Search Head, download the Universal Forwarder app and you distribute that app to the /opt/splunkforwarder/etc/apps/ directory of the machines you want to send data to the Cloud.

Depending on your needs and network architecture, it could get more complicated, but that is the simple version.

So each Windows Server would need a Splunk UF (Universal Forwarder) and the Spunk Cloud UF app/ta/add-on (TA stands for Technical Add-on) to be able to send and collect data.

Each data source also needs a configuration telling it what data to collect.
This is often achieved by using a Splunk TA aka add-on on Splunkbase:
https://splunkbase.splunk.com/

You can download the Splunk UF here:
https://www.splunk.com/en_us/download/universal-forwarder.html

For larger environments, the UF and required addons are usually distributed via a Splunk Deployment Server.
Also, often data is sent through one or more Forwarders before Cloud to minimize firewall rules, or depending on your network architecture needs.

All data sources need to be able to send data via tcp/9997 to Splunk Cloud.

So the breakdown of steps is:

Create an index on Splunk Cloud to receive your data
Download the Cloud TA (called Cloud Universal Forwarder) from Splunk Cloud Search Head
Install a UF and the Cloud TA onto your data source
1. The Cloud TA needs to be untar'd to /opt/splunkforwarder/etc/apps/
2. Or it can be distributed via Splunk Deployment Server
Install one or more add-ons aka TAs to /opt/splunkforwarder/etc/apps/
Configure and enable one or more 'inputs' or data to send by editing the inputs.conf within each TA/add-on
1. There is usually a template inputs.conf in the default folder of each add-on.
2. Create a /local folder (same level as /default) in each TA and copy that inputs.conf in there
3. Edit it and enable one or more inputs to send data to Splunk

There actually is an 'outputs.conf' but the Splunk Cloud TA/UF handles that to securely send to Splunk Cloud.

Re: What are the steps of ingesting data into Splunk cloud?

shubham92 — Fri, 25 Mar 2022 18:25:19 GMT

It totally depends on the log source you are dealing with.

Windows/Linux: Install UF, add Splunk Cloud Credential File. Edit input.conf file if you want to change the Index.

Firewall Logs: If you have a Syslog server in place, install a UF on it and redirect the logs from the Syslog folder to it. If you do not have a Syslog server, you can use a Heavy Forwarder configured as a Syslog Receiver.

Cloud-Based: Check for supported apps. Most of them support API based integration, which is easy to do. Each app includes the steps to follow.

Let me know if you have any specific devices in question. I am no expert, but will definitely try to help you out.