topic Re: I see traffic from forwarder to indexers, but I cant find anything when I search on the search heads. in Splunk ITSI

I see traffic from forwarder to indexers, but I cant find anything when I search on the search heads.

millinkan — Sun, 07 Jun 2020 00:26:57 GMT

I have installed a forwarder on my apache serer and I see traffic (logs) moving from the web server to the indexers.
When I run the command below on my search heads (plus ITSI), I get nothing.

| eventcount summarize=false index=* index=_* | dedup index | fields index

my input.conf:
[monitor:///web/JBossWeb/jws-3.0/https/logs/access.log.$(date +%Y.%m.%d)]
sourcetype=apache_access
disabled = 0
index = apache

[monitor:///web/JBossWeb/jws-3.0/https/logs/error.log.$(date +%Y.%m.%d)]
sourcetype=apache_error
disabled = 0
index = apache

Please help.
Thank you.

Re: I see traffic from forwarder to indexers, but I cant find anything when I search on the search heads.

skalliger — Wed, 18 Mar 2020 09:59:55 GMT

What are you trying to do? Get an eventcount? Because that's what the search does. There's a small mistake in your search, should be | eventcount summarize=false index=* index=_* | dedup index | fields index

Also, you can write | eventcount summarize=false index=* index=_* | stats values(index) instead.

Skalli

Re: I see traffic from forwarder to indexers, but I cant find anything when I search on the search heads.

millinkan — Wed, 18 Mar 2020 10:13:09 GMT

Thanks for the alternate search query.

Re: I see traffic from forwarder to indexers, but I cant find anything when I search on the search heads.

millinkan — Thu, 30 Apr 2020 08:08:04 GMT

I used the below in my inputs.conf and it worked.

[monitor:///web/JBossWeb/jws-3.0/httpd/logs/error.log.*]
sourcetype=apache_error
disabled = 0
index = linux
crcSalt=
ignoreOlderThan = 0d

Thanks for you assistance

Re: I see traffic from forwarder to indexers, but I cant find anything when I search on the search heads.

lloydknight — Thu, 30 Apr 2020 09:47:15 GMT

you cannot search anything on the SH (assuming no data on index=apache) but you see traffic logs (assuming the forwarder is already connected to the indexers)

have you tried checking on splunkd.log if there are any errors? are the sources being monitored? (run ./splunk list monitor on the UF)