topic Re: Possible explanations for Index not being refreshed automatically? in Getting Data In

Possible explanations for Index not being refreshed automatically?

_dave_b — Thu, 03 Dec 2015 16:15:23 GMT

Hi.

I created a new index with along with a fresh install on a Win7 system a few days ago. It should be pointing to some log files that are continuously being updated. When I first created it, everything looked fine. However, when I run a search today, I'm still being told by Splunk that the latest event is from the time when I initially set up the index. I thought the index was supposed to refresh automatically, so I am puzzled as to why it's not being updated. Does anybody have any pointers where to look to find out why this is happening?

Thanks

Re: Possible explanations for Index not being refreshed automatically?

jplumsdaine22 — Thu, 03 Dec 2015 16:38:09 GMT

The most likely issue, since the logfiles reside on your indexer, is that the input is not set correctly. Try this search

index=_internal sourcetype=splunkd NOT log_level=INFO

This is splunks internal log, and should tell you if there are any problems with the logfiles you are generating. Especially of interest are messages with the filenames of the logfiles you think you have set up, and messages with TailingProcessor in them. Also check the settings of your input and make sure that they are set correctly.

Re: Possible explanations for Index not being refreshed automatically?

_dave_b — Tue, 29 Sep 2020 08:02:16 GMT

Thanks for your reply. I tried that search and it mostly had errors that look like this:
12-01-2015 15:26:24.792 -0500 WARN SearchResults - Corrupt csv header, 2 columns with the same name '__mv_deviceId' (col #14 and #1, #14 will be ignored)

For my input, I had set it to read a directory with a whitelist to include only the files I'm interested in. I just deleted my whitelist and my search is now pulling events from the most recent logfile, so that's good. I thought the whitelist was good, it looked like:

c:\program files\app\appserver\logs\*\communication\*.txt

Where the wildcards were covering for a datestamp

Re: Possible explanations for Index not being refreshed automatically?

jplumsdaine22 — Thu, 03 Dec 2015 21:05:42 GMT

Ah yes I think the whitelist stanza doesn't support backslashes in windows - so no files were getting indexed.

See http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Specifyinputpathswithwildcards and http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Whitelistorblacklistspecificincomingdata

You should be able to put the whitelist in the monitor part. EG:

[monitor://c:\program files\app\appserver\logs\*\communication\*.txt

and drop the whitelist entry altogether

Re: Possible explanations for Index not being refreshed automatically?

woodcock — Fri, 04 Dec 2015 21:13:38 GMT

You created a new Index or a new Input. Creating a new Index will not cause data to flow into it.