Getting Data In

How to troubleshoot why my heavy forwarder is not receiving Windows event logs from the universal forwarder?

chanamoluk
Explorer

I want to send "wineventlog:security " logs to Heavy forwarder(KIWISERVER) and below are the configuration files that I have created on the Universal forwarder

inputs.conf:

[WinEventLog://Security]
disabled = 0
index = activedirectory
sourcetype=adlog_003

outputs.conf:

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = xxx.xx.xxx.xx:9997

[tcpout-server://xxx.xx.xxx.xx9997]

When i see the "Splunkd" log it shows "Connected to idx=xxx.xx.xxx.xx:9997" but i'm unable to see the events in splunk search index=active*

sample **splunkd log file :**

12-17-2016 01:09:30.162 -0500 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='C:\Program Files\SplunkUniversalForwarder\var\log\splunk\license_usage_summary.log'.
12-17-2016 01:09:30.162 -0500 INFO  WatchedFile - Will begin reading at offset=424312 for file='C:\Program Files\SplunkUniversalForwarder\var\log\splunk\metrics.log'.
12-17-2016 01:09:30.178 -0500 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='C:\Program Files\SplunkUniversalForwarder\var\log\splunk\remote_searches.log'.
12-17-2016 01:09:30.178 -0500 INFO  WatchedFile - Will begin reading at offset=854 for file='C:\Program Files\SplunkUniversalForwarder\var\log\splunk\conf.log'.
12-17-2016 01:09:30.287 -0500 INFO  TcpOutputProc - Connected to idx=xxx.xx.xxx.xx:9997

Please let me know what mistake I have done.....

noresults

0 Karma

mattymo
Splunk Employee
Splunk Employee

Are you searching from your searchhead or from the heavyforwarder?

can you see internal logs from the forwarder?

index=_internal host=<yourUF>

check metrics.log for your sourcetype. (sidenote: i don't think you need to be setting the sourcetype like that for a windows input, are you using any windows apps or TAs?)

index=_internal host=<yourUF> source=*metrics.log 

check inputstatus on the forwarder using the splunk list inputstatus command (depends on uf version - 6.3 or 6.4+ i think?)

https://docs.splunk.com/Documentation/Splunk/6.5.1/Admin/AbouttheCLI

- MattyMo
0 Karma

chanamoluk
Explorer

i'm searching on "Heavy forwarder" and i cannot see internal logs as well.

i think this is a firewall issue?

0 Karma

coltwanger
Contributor

I think the issue is you are forwarding your logs to the indexers, and the HF is not configured to search the data on the indexers. You need to search from the indexers themselves, or from a Search Head that has access to the data on the indexers.

0 Karma

mattymo
Splunk Employee
Splunk Employee

Yep, like coltwanger said,

if your HF is not set up for distributed search, you wont see the logs from there!

Check from the search head, ideally.

- MattyMo
0 Karma

tkomatsubara_sp
Splunk Employee
Splunk Employee

( I don't know why I can't add comment)

I'd like to suggest Just to monitor some file and send it to your index in order to identify the root cause.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...