Getting Data In

Data collection Methods, is there a way to see what feeds a datasource?

packet_hunter
Contributor

I am working on a matrix of data sources for my splunk deployment.
I need to map my data sources -collection method (for example firewalls, DC(s), servers and other appliances) from forwarders or syslog or other log feeds.
Many of the data sources were setup prior to my involvement.
How would one take on the is challenge and no - forwarder monitoring is not enabled.
Thank you

Tags (2)
0 Karma
1 Solution

lguinn2
Legend

I suggest a couple of things. First, just get a list of basic info in your Splunk instance by running this search command

| tstats count where index=* by host index sourcetype source

Of course, the host that appears in the list might be overridden at some point during parsing, but it is often the same host name as the original forwarder or source. It's a good starting point.

Second, the Splunk indexers will keep some information in their internal logs, which are automatically indexed into _internal. Splunk forwarders send their internal logs to the indexers by default. So even if you don't have forwarder monitoring, you should still have the fowarder logs unless this has been explicitly disabled by your predecessor. And you should turn on forwarder monitoring, it doesn't cost anything to do it.

You can also run a variety of searches against the internal index. The Metrics component only collects information about the top 10 sources during each time interval, so it will not be complete information, but this search may still be helpful:

index=_internal sourcetype=splunkd component=Metrics group=per_source_thruput 
| chart sum(ev) by series host | rename series as source

The above search shows the number of events for each source. If you don't care about that, you can get a more complete list from this search:

index=_internal sourcetype=splunkd component=WatchedFile | stats list(file) as Files by host

This shows all the file inputs. For searches in the _internal index, the host name is the name of the forwarder that is sending the data. In other indexes, the host name for the file will be based on the setting in inputs.conf, so these host names could be different.

And there is more, but this should get you started.

View solution in original post

0 Karma

lguinn2
Legend

I suggest a couple of things. First, just get a list of basic info in your Splunk instance by running this search command

| tstats count where index=* by host index sourcetype source

Of course, the host that appears in the list might be overridden at some point during parsing, but it is often the same host name as the original forwarder or source. It's a good starting point.

Second, the Splunk indexers will keep some information in their internal logs, which are automatically indexed into _internal. Splunk forwarders send their internal logs to the indexers by default. So even if you don't have forwarder monitoring, you should still have the fowarder logs unless this has been explicitly disabled by your predecessor. And you should turn on forwarder monitoring, it doesn't cost anything to do it.

You can also run a variety of searches against the internal index. The Metrics component only collects information about the top 10 sources during each time interval, so it will not be complete information, but this search may still be helpful:

index=_internal sourcetype=splunkd component=Metrics group=per_source_thruput 
| chart sum(ev) by series host | rename series as source

The above search shows the number of events for each source. If you don't care about that, you can get a more complete list from this search:

index=_internal sourcetype=splunkd component=WatchedFile | stats list(file) as Files by host

This shows all the file inputs. For searches in the _internal index, the host name is the name of the forwarder that is sending the data. In other indexes, the host name for the file will be based on the setting in inputs.conf, so these host names could be different.

And there is more, but this should get you started.

0 Karma

packet_hunter
Contributor

Thank you Lisa!!!

0 Karma

packet_hunter
Contributor

Not sure if you are still looking at this... but in my splunk DMC instance, under settings>monitor console> Forwarders: Instance, I can see the KB/s data rate per universal forwarder, however I cannot see a GB/day volume anywhere.

Do you have some search syntax to convert KB/S to GB/day volume? Or is there another report somewhere?

Thank you

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...