All Apps and Add-ons
Highlighted

splunk monitoring inputs.conf (after deleting file how to add the file that has a same file name as deleted file)

Path Finder

hello,
every one

I wrote like this in inputs.conf.

[monitor:///home/ec2-user/aaa/]
host = ip-10-0-0-xxx
index = aaa
whitelist = aaa[^/]*\.csv$
sourcetype = csv
crcSalt = <SOURCE>

[monitor:///home/ec2-user/bbb]
host = ip-10-0-0-xxx
index = bbb
whitelist=bbb[^/]*\.csv$
sourcetype = csv
crcSalt = <SOURCE>

I can put the new data to 'aaa' index, but If i do as below process, I can't put the data to 'aaa' index.

step 1. put the ' aaa20181004.csv' to the aaa folder. This step is ok. (I can check the data in the aaa index)
step 2. delete the 'aaa
20181004.csv' on the splunk . This step is ok.
of course i deleted 'aaa20181004.csv' on the aaa folder.
step 3. put the ' aaa
20181004.csv' again to the aaa folder. - This step has problem.
I can't read 'aaa20181004.csv'' again even though there is ' aaa20181004.csv' in the aaa folder.
after step 3, There isn't aaa_20181004.csv' in the aaa index.
how should I do.

But If I do that process to bbb. all steps are fine.

Just different thing is the contents of aaa and bbb.

I'll put the file to folder daily.(aaa20181003.csv , aaa20181004.csv , aaa_20181005.csv ...)
I'd like to know daily data.

thank you in advance.

0 Karma
Highlighted

Re: splunk monitoring inputs.conf (after deleting file how to add the file that has a same file name as deleted file)

Path Finder

how do you delete the data in splunk ? (step 2)

with crcSalt = , splunk remembers the file name, so if you like to reindex that file, take another filename (e.g. filename_v2).

What is the exacte use case why you need to reindex this files ? maybe with a better understandig I can provide you some ideas how to solve your problem.

0 Karma
Highlighted

Re: splunk monitoring inputs.conf (after deleting file how to add the file that has a same file name as deleted file)

Path Finder

Thank you for your attention.

how do you delete the data in splunk ? (step 2)
->I used the 'delete' command on the splunk.

I'd like the file to be entered automatically to Splunk, if i place the daily file in specific folder.

0 Karma
Highlighted

Re: splunk monitoring inputs.conf (after deleting file how to add the file that has a same file name as deleted file)

Explorer

If you want to re-index the same file, you can use btprobe to reset the fishbucket for that file so that it is re-sent by the forwarder:

E.g.

  • Stop splunk on the forwarder
  • Run: ./splunk cmd btprobe -d /opt/splunkforwarder/var/lib/splunk/fishbucket/splunk_private_db --file /home/ec2-user/aaa/aaa_20181004.csv --reset
  • Start Splunk

You should find that the forwarder will re-send the file for indexing.

0 Karma