Archive

re-read a directory

Explorer

Having some trouble with a directory monitor:

[monitor:///usr/local/ecc_to_splunk/pickup/*.disk.*]

This monitor loaded the data, but I deleted it (for unrelated reasons) and am having trouble getting splunk to read it again. After deleting the data in splunk using |delete, I removed the files from the dir, disabled then enabled the monitor, then put the same files back.

It seems as though splunk isn't reading the files because it already has once. Is there a way to override this and force splunk to read them? Thanks.

Tags (1)

Path Finder

So I had the same problem, I had | delete-ed a bunch of data, but then wanted to re-add to splunk.

When using ./splunk add oneshot all the data was added back to splunk BUT the timestamp for ALL the data was from when it was re-added, not the original modtime of the file (input is a directory with 2000+ log files). is there a way to have it re-index using the timestamp of the files?

Path Finder

well I'm dumb, and should read things first, like putting new data into a test index to make sure it looks ok and test props.conf etc. i guess i can just make a new index and splunk should index with correct timestamps

Splunk Employee
Splunk Employee

You can force Splunk to forget all file history that it has read by cleaning out the fishbucket directory (while Splunk is down) on the machine where it was read from. This probably isn't what you want. You can also have Splunk re-index a specific file using:

./splunk add oneshot /usr/local/ecc_to_splunk/pickup/file1.disk.ext

You can't wildcard this, you have to run this for each specific file name, though you could of course script that in the shell.

Splunk Employee
Splunk Employee

you can add a -sourcetype mysourcetype flag to the commmand line above.

Explorer

Thanks this is exactly what I was looking for. Is there a way to set the sourcetype? I have one specifically defined for these files in tranforms.conf.

Communicator

If I understand you correctly, Splunk has previously indexed the data. Even if you delete the source file(s) and then later on re-add them, I do not think Splunk will re-index them as they already existing within Splunk.

0 Karma