Monitoring Splunk

Troubleshoot a MonitorNoHandle input

jeremyhagand61
Communicator

I have a UF (7.3.1) configured with the Splunk TA for Windows Inf. 6.0. It is a Domain Controller and has about 16 different inputs configured. The DC is running Windows Server 2016 Core.

This morning it was working fine.

I started work on sending the DNS queries for the local AD domain to the nullQueue and mistakenly put the props and transforms sections into the Deployment App without thinking and reloaded the deployment server.

After the UF restarted, the DNS logs stopped coming in altogether. I reversed the changes in the local\props and transforms in the deployment app, but after numerous restarts of every component, the DNS logs still don't appear in the index.

I've turned on debug logging on the UF and couldn't see anything relevant. I used btool to confirm that the DNS log file monitor is enabled in the running config on the UF and that the prop/transforms config is absent.

At this point I've run out of ideas. For some reason the "MonitorNoHandle" stanza doesn't put much in the splunkd.log file except when it is broken.

Can anyone help with some systematic steps to troubleshoot this. Thing I know are:
1: The DNS log entries are being created
2: All other configured inputs on the UF are being forwarded and indexed

Here is the config of the input in question, as shown from using btool:

[MonitorNoHandle://C:\Windows\System32\Dns\dns.log]
disabled = 0
host = HOSTNAME_REDACTED
index = msad
interval = 60
sourcetype = MSAD:NT6:DNS

Here it is from the inputs.conf file

    [MonitorNoHandle://C:\Windows\System32\Dns\dns.log]
    sourcetype=MSAD:NT6:DNS
    disabled=0
    index = msad
0 Karma

jg91
Path Finder

I Have the same problem, did you solved it?

0 Karma

jeremyhagand61
Communicator

Some further info from the DEBUG log:

09-0
2-2019 17:01:44.287 +1000 DEBUG TcpOutputProc - Pushed eventId=4862 on chanId=21 to back of tcp client (tcp output) queue. source:source::MonitorNoHandle|host::REDACTED|MSAD:NT6:DNS|
09-02-2019 17:01:44.287 +1000 DEBUG TcpOutputProc - Pushed eventId=4862 on chanID=0 to back of tcp client (tcp output) queue

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...