topic Re: Understand which hosts sends logs to which Splunk Component in Splunk Enterprise

Understand which hosts sends logs to which Splunk Component

SplunkExplorer — Thu, 29 Feb 2024 09:56:05 GMT

Hi Splunkers, today I have a problem about understanding how and where Log Sources sends logs to Splunk.
In this particular Splunk On Prem environments, no documentation has been done, except the HLD.
So, we have to understand, for each log source, what Splunk component it reaches and how.
For example, if I have a Domain Controller, we must establish:

Where it sends logs? Directly to Indexers? To a HF?
A UF is installed on it? If not, how it send logs? WMI? WEF? Other

And so on.

Now, List of servers sending logs to Heavy forwarder is a community discussion where I started from @scelikok suggested search, changed it in:

index=_internal component=TcpOutputProc | stats count values(host) as host by idx | fields - count

and it helped me a lot: I'm able, for each Splunk Component of env (IDS, HF and so on) to understand what Log sources send them data.

So, what's the problem? The above search return data forwarded by another Splunk component.
I mean, in the output, field idx has always format ip/hostname:9997, so it means that data are coming from a server with UF or from another Splunk host (we have some intermediate forwarder, so sometimes I can see data ingested by an HF coming from another HF). What about data sent not with a Splunk agent/host?
For example, suppose I have this flow:

Log source with Syslog -> Splunk HF receive on port 514

With above search, I cannot see those sources (and I know for sure they exist on our env).
How can I recover it? The syslog is only an example, the key point here is: I must complete my search with all log sources that do not use UF and/or any other Splunk element, but other forwarding tool/protocol (syslog, API, WEF, and so on).

Re: Understand which hosts sends logs to which Splunk Component

isoutamo — Thu, 29 Feb 2024 14:37:22 GMT

Hi
Have you look this doc https://docs.splunk.com/Documentation/Splunk/9.2.0/InheritedDeployment/Introduction ?
r. Smo

Re: Understand which hosts sends logs to which Splunk Component

inventsekar — Thu, 29 Feb 2024 15:31:17 GMT

Hi @SplunkExplorer

> Where it sends logs? Directly to Indexers? To a HF?

A Splunk UF generally will send the logs to indexer. but if your indexer is overloaded and if you want to do some preprocessing beforehand, then you should use a HF(from UF, send the logs to HF.. HF will do some parsing tasks, then it will send the logs to indexer)

> A UF is installed on it? If not, how it send logs? WMI? WEF? Other

yes, WMI options is available. and if you can not install the UF, then you can use a syslog server to collect the logs from all systems that dont have UF and send it to a HF or indexer..