Splunk SOAR not forwarding data to splunk

[anya25]

I'm trying to use the Splunk App for SOAR to forward logs and events from SOAR to Splunk Enterprise.

The servers seem to be connected (test connectivity works) but the data (events, playbook runs etc.) isn't being indexed and doesn't appear in search in Splunk.

I tried reindexing the data through SOAR but it didn't work.

Adding audit input in the app is working fine, but data isn't being indexed in real time according to the supposed indexes (I did create them using the "Create Indexes" button in the app)

Did anyone experience anything similar or has any idea as to what might be the issue?

[marnall]

Did you set up your SOAR to forward logs?

Go to Administration->Administration Settings->Forwarder Settings->New Group

Then add your indexers, e.g.:

indexer1:9997

Check the boxes for which logs you would like to see.

Add an optional TCP token if it applies for your environment.

Then if you save this configuration, your SOAR should start sending logs to Splunk Enterprise.

Ref:

https://docs.splunk.com/Documentation/SOARApp/1.0.57/Install/ConnectremotesearchSOAR6.2

https://docs.splunk.com/Documentation/SOARonprem/latest/Admin/Forwarders

[anya25]

Yes, I already set this up

[marnall · Accepted solution]

OK that is good. Do you see any logs coming from your SOAR host in the internal index at index=_internal ?

If yes, then can you see any errors when you filter the source to splunkd.log? (for me it's source="/opt/phantom/splunkforwarder/var/log/splunk/splunkd.log")

If no, then can you SSH into the SOAR machine and then read that splunkd.log file looking for errors? Usually the file is located at /opt/phantom/splunkforwarder/var/log/splunk/splunkd.log

(depending on how big the logfile is, you could use "cat splunkd.log | grep ERROR")

[anya25]

I don't see any events when filtering index=_internal and source=<path_to_splunkd.log> (with my path obviously)

but I do see errors when looking in the splunkd.log file in my SOAR machine - lots of "connection to host <indexer>:9997 failed", which is weird because 9997 is open on the splunk indexer, the machines are in the same segment and the "test connectivity" worked.

[anya25]

Sorry, my mistake - the IP address in the errors in the log file belongs to antoher Splunk server that is turned off.

I don't see any errors with the correct IP.

[marnall]

Excellent, it sounds like it is working with the right IP