On a Windows 2008 domain controller, DNS debug logging enabled, so that queries can be captured by Splunk. The DNS debug log is called "D:\dns-log\dns-log.txt"
That file is being monitored by Splunk and it successfully is added to Splunk indexer.
Problem:
This log file keeps disappearing from its host after some number of hours or days.
If we restart the Microsoft DNS service, the log file is recreated and Splunk resumes indexing, minus the lost period of time when the file was missing.
Is there any way that the Splunk Universal Forwarder could be causing the file to be deleted?
I have not yet enabled Windows file auditing because that is quite resource intensive on the host.
You should use MonitorNoHande input type instead of monitor since dns log component is very sensitive to file access collisions.
I've already read the "godlessheathenmemoirs" article and the "0x80000000 Logs write-through transactions" setting is enabled in my environment. It hasn't made it any better.
Was you able to find a solution for you problem? I think I may have a similiar issue.
I came across interesting article when digging for info on the same subject - (not enough karma to paste the link) try googling for "Gathering detailed DNS debug logs from AD DNS" or go to godlessheathenmemoirs blog at blogspot dot com and look at August 2011 archive.
Haven't tried the tricks myself yet (there are always other priorities) but it might help.
Did you ever find the answer to this? It is happening to me too. It seems to be when the log file reached its maximum size it is deleted and recreated and during this process the file creation fails.
Perhaps it fails to recreate the file because Splunk has a file handle on it.
Then this is a non-destructive monitor.
Something else is removing your file.
If your inputs.conf stanza for this input is of the type
[monitor://<path>]
Then this is non-destructive. If your inputs.conf stanza for this input is of the type
[batch://<path>]
Then this could be destructive, although such a stanza should also have the line
move_policy = sinkhole
to be properly destructive (e.g., index the file and delete it).
Thanks for the idea -- but we are not using that type of config.
Ours is:
[monitor://D:\dns-log]
disabled = 0
index = WinEvtDns
sourcetype = dns