I'm running Splunk 4.3.3 on Windows Server 2008 R2 64-bit.
I have scheduled a small .net console application to overwrite a csv lookup file in "E:\Splunk\etc\apps\myapp\lookups" once a day.
The problem is that only parts of the lookup data is available in searches until I restart Splunk.
Is there a way to reload the lookup file without restarting Splunk?
Hmmm - Thanks BMunson. I will look into that on our hot standby system - it's not a massive file to be honest - only 147 lines so a bit surprised by that (I know some customers have thousands).
Can't find anything called tsidxstats under /opt/splunk/var/lib/splunk - although obviously a lot of *.tsidx files in the db dirs. No real documentation out there (just a scattering of tags on here - http://splunk-base.splunk.com/tags/tsidxstats/) but is it generated when I use tscollect or search accelleration?
Will see if I can goad a server into caching and then try the transforms.conf change.
On Splunk indexer, 5.0.2. We have just had a case where the lookup file was definitely being cached (it was feeding to an event creator into our Zenoss monitoring system). Ran the above debug command successfully and it now seems to be reporting back the correct values without having to restart our main Splunk instance (lots of change requests needed for that).
I can imagine it was done for speed but I wish I had known that this caching was going on beforehand. Not sure it always caches the file either - I can remember some changes worked without having to restart the Splunk.
For larger lookup files splunk indexes the file and keeps results in a TSIDX. This effectively caches it. The advantage of this is it speeds up lookups and any searches using them. The disadvantage is the lookup may return old data.
Finding this TSIDX and deleting it will force it to reindex the lookup. It should be in a subfolder of $SPLUNK_HOME/var/lib/splunk/tsidxstats
I also found a key you can try setting in the transforms.conf. It states it prevents scripts from being cached but it might work for large files.
allow_caching = <bool> * Allow output from lookup scripts to be cached
Lookup files should not need anything special to reload from one search to the next. When a new search runs, the new copy of the lookup file should be used immediately.