I'm having trouble getting a single file to be indexed. I have successfully monitored all files in a directory before, but I'm not sure what is causing the specified file to not be monitored. Here are two stanzas I have tested in inputs.conf, neither with success.
[monitor://C:\Program Files (x86)\Sell\Sell.FT Service\Client.log] index = ClientProcessor sourcetype = Client_log disabled = false [monitor://C:\Program Files (x86)\Sell\Sell.FT Service] index = ClientProcessor sourcetype = Client_log disabled = false whitelist = (.*\.log)
I created a new .txt file and copied a couple logs over to the new file. I then added another stanza to monitor that file, and the new file was indexed but not the original. I have tried .txt and .log suffixes for LPClient. Here is the current
[monitor://C:\Program Files (x86)\Sell\Sell.FT Service\LPClient.txt] index = LP sourcetype = LPClient_log disabled = 0 [monitor://C:\Program Files (x86)\Sell\Sell.FT Service\NewTextDocument.txt] index = LP sourcetype = LPClient_log disabled = 0
Do you get any warnings or errors in
splunkd.log related to this files? As well to list the inputs using
btool to see if your config is being applied:
splunk btool inputs list
Also check the inputs status to see if Splunk is monitoring the directory:
splunk list inputstatus
Hope this helps ...
The last logs in splunkd.log are for the download and install of the app from the deployment server. The input stanza to monitor the directory is listed with the btool command. However, I am unable to run the last command due to privileges on my account.
Here's the entry from btool
[monitor://C:\Program Files (x86)\Sell\Sell.FT Service] _rcvbuf = 1572864 allowSslCompression = true allowSslRenegotiation = true baseline = 0 dedicatedIoThreads = 2 disabled = false enableSSL = 1 evt_dc_name = evt_dns_name = evt_resolve_ad_obj = 0 host = APPS01 index = ClientProcessor interval = 60 maxSockets = 0 maxThreads = 0 port = 8088 sourcetype = Client_log sslVersions = *,-ssl2 useDeploymentServer = 0 whitelist = (.*\.log)
Looking at the
whitelist option .... why do you use a capturing group here? Have you tried to use
whitelist = .*\.log
here is a list of troubleshooting steps:
splunk btool inputs list monitor if your config is applied correctly and check
splunk list inputstatus if Splunk is reading the directory and the files?
Check if your service account has permission to read files in that directory?
Check on any intermediate parsing layer, if there are nullQueues configured?
index=_internal sourcetype=splunkd source=*metrics.log series=*Sell.FT Service* over all time to get some information if data was sent?
Search the index over all time, maybe you have some timestamping issue?
tcpdump the traffic to see if the input instance is sending out events and they get lost somewhere?
Was the file already indexed, if so clean the fish bucket index so Splunk will re-index it or use the option
crcSalt on your input - search docs for more information on both topics.
This list is almost never-ending ... Good luck and I hope you find the missing puzzle piece.
Thanks for the info.
splunk list inputstatus led me to this, type = unreadable file type. I am unsure how to resolve this as the file type is .log and it is an auto-generated log file.
I can't answer why, but I opened one of the files and saved it, and splunk then indexed the logs across all hosts.
I'm having the exact same thing. file type keeps reporting "unreadable file type" until I open te file and save it (without any changes made). After savind the file gets indexed and gets status "Finished reading".
Did you get this issue solved?
just converted it to an answer, feel free to accept it if it answered your question.