Getting Data In

How do I find out what heavy forwarder a device is sending logs from?

kcooper
Communicator

We have a DMZ heavy forwarder (HF) that sends logs from the devices on the DMZ environment to our Splunk server. I need to know the name of the devices that are sending the data through the HF.

How can I get that information?

0 Karma

FrankVl
Ultra Champion

Your title and question are confusing. What do you want to find out? The name of the HF that was involved in processing a certain event, or the name of the original source host from which the event originated?

What kind of data feed are we talking about here? How are they sending through the HF? What kind of Splunk input is used? In case you're interested in the original source host: what does the data look like, does that contain the original host name?

0 Karma

dkeck
Influencer

HI,

try this query : index=_internal sourcetype=splunkd host=<heavyforwardername> group=per_host_thruput | stats values(series)

dkeck
Influencer

Please accept the answer if it helped you 🙂 thank you

0 Karma

kcooper
Communicator

Thank you. This worked great.

0 Karma

FrankVl
Ultra Champion

I highly doubt that works great, since the per_x_thruput metrics logs are incomplete. As the docs specify:

Note: The per_x_thruput categories are not complete. Remember that by default metrics.log shows the 10 busiest of each type, for each sampling window. If you have 2000 active forwarders, you cannot expect to see the majority of them in this data. You can adjust the sampling quantity, but this will increase the chattiness of metrics.log and the resulting indexing load and _internal index size. The sampling quantity is adjustable in limits.conf, [metrics] maxseries = num.

https://docs.splunk.com/Documentation/Splunk/7.2.3/Troubleshooting/Aboutmetricslog

0 Karma

kcooper
Communicator

I have skimmed the list of devices that was returned and it looks good but I am not sure at this time if all the devices are listed.

What would you suggest?

0 Karma

FrankVl
Ultra Champion

As I mentioned in my original comment to your question: that highly depends on how the data is coming in.

One thing you can do to allow filtering for data coming through a certain HF using file monitor inputs is to put the log files that splunk reads in a folder that is named after the HF's hostname. That way, the HF's name shows up in the source field.

Another solution is to add a custom meta data field, to explicitly label each event with the HF it passed through.

0 Karma

kcooper
Communicator

we can't have the all the log files in one. they are all different devices.
I will look into adding a custom meta data field - that is a good idea
thx

0 Karma

FrankVl
Ultra Champion

It doesn't all have to be in one folder. Just have the HF name somewhere in the path. For example, say currently you have:
/opt/logs/typeA/foo.log
/opt/logs/typeB/bar.log

Just create something like /opt/hf.dmz.com/logs/ as a symbolic link to /opt/logs/ and update splunk inputs accordingly. Which results in source values like:

/opt/hf.dmz.com/logs/typeA/foo.log
/opt/hf.dmz.com/logs/typeB/bar.log

0 Karma

dkeck
Influencer

What would you suggest then?

0 Karma

dkeck
Influencer

Then please hit accept to award points 🙂 thank you

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...