topic Re: Why is latest data in monitored paths in my Windows folders not getting indexed? in Getting Data In

Why is latest data in monitored paths in my Windows folders not getting indexed?

shariinPH — Mon, 30 Mar 2015 07:36:05 GMT

I am monitoring certain paths in my Windows folders..
I have already done the following:
Put crcSalt on my inputs.conf
Commanded clan all in my forwarder
Commanded my indexer to clan event data on that certain index.

I am confused why it's not indexing past and latest data.
Someone help me pls.
Thanks!

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

satishsdange — Mon, 30 Mar 2015 07:41:59 GMT

Could you please share some more information such as inputs.conf, outputs.conf,
output of "tail -100f splunkd.log | grep TcpOutputProc" on UF, enabled receiving?

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

shariinPH — Mon, 30 Mar 2015 07:53:13 GMT

hi satishsdaange!
here's my inputs.conf on my universal forwarder

[monitor://G:\OperationData\Atr_RdngINDRA\ZDanning\...\*-*funn.txt]
disabled = false
index = dun_trial
sourcetype = findun
ignoreOlderThan = 25d
_TCP_ROUTING=maydev

my props.conf in my indexer is:

[findun]
DATETIME_CONFIG = NONE
MAX_TIMESTAMP_LOOKAHEAD = 150
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = false
pulldown_type = 1

note that i already removed my crcSalt

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

satishsdange — Mon, 30 Mar 2015 07:56:12 GMT

could you please run on UF "tail -100f splunkd.log | grep TcpOutputProc" & share result

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

shariinPH — Mon, 30 Mar 2015 08:00:31 GMT

hi, im using windows os for my splunk

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

shariinPH — Mon, 30 Mar 2015 08:01:52 GMT

but i have checked the file splunk.log and copied some txt

03-30-2015 14:54:37.228 +0800 ERROR TailingProcessor - File will not be read, seekptr checksum did not match (file=G:\OperationData\Mtr_RdngINDRA\ZDunning\030815\0200-20150225dunn.txt).  Last time we saw this initcrc, filename was different.  You may wish to use a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.
03-30-2015 14:54:37.259 +0800 ERROR TailingProcessor - File will not be read, seekptr checksum did not match (file=G:\OperationData\Mtr_RdngINDRA\ZDunning\030815\0300-20150225dunn.txt).  Last time we saw this initcrc, filename was different.  You may wish to use a CRC salt on this source.  Consult the documentation or file a support case online at http://www.splunk.com/page/submit_issue for more info.

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

satishsdange — Mon, 30 Mar 2015 09:14:50 GMT

Well, I wanted to check whether UF is forwarding data to indexers or not? Could you please confirm that?

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

shariinPH — Mon, 30 Mar 2015 09:33:08 GMT

on my other monitored paths, yes it is forwarding data, but not all the files and logs were forwarded. what do you think is the problem with this?

Re: Why is latest data in monitored paths in my Windows folders not getting indexed?

woodcock — Mon, 10 Aug 2015 15:16:10 GMT

It turns out that ignoreOlderThan works differently than many people assume. It does NOT examine events; rather it tracks the modification time of the monitored file. If many of the files haven't been written to for upwards of a month, Splunk will stop monitoring them and once Splunk makes this decision, it is permanent. So even if the files have been modified recently, Splunk will never care. In other words, avoid ignoreOlderThan like the plague and remember to take it back out if you ever do have to use it (e.g. first time data onboard).