I have some zip files that I need to reindex after cleaning the target index and refining the props.
I cannot get splunk to re-ingest them no matter what--even after cleaning the fishbucket.
Here is the TailingProcessor state for one of them:
<s:key name="/opt/splunkdata/tmp/cm/2014.11/2014001.zip">
<s:dict>
<s:key name="file position">0</s:key>
<s:key name="file size">10239974</s:key>
<s:key name="parent">/opt/splunkdata/tmp/cm/2014.11</s:key>
<s:key name="percent">0.00</s:key>
<s:key name="type">finished reading</s:key>
</s:dict>
</s:key>
While percent is 0, it has finished reading it. As far as I can tell from other Answers, this is not supposed to happen.
Btprobe is no help:
splunk cmd btprobe -d /opt/splunk/var/lib/splunk/fishbucket/splunk_private_db --file /opt/splunkdata/tmp/cm/2014.11/2014001.zip
Using logging configuration at /opt/splunk/etc/log-cmdline.cfg.
record not found
So can't reset it that way. Then I took the radical step of cleaning the fishbucket..still no joy.
I can't see any way around this; and furthermore it looks like some kind of bug--oneshots aren't working either.
Anyone know a solution? Version is 6.4.2
Thanks!
You need to change -d /opt/splunk/var
to -d /opt/splunkforwarder/var
.
OP could also modify a character at the top of the file so Splunk will see it as a new file.
How are you getting the data in? What search are you using to determine that the data is not in? It does not sound like you needed to clear the fishbucket and it may even be that the data actually is in!
Nope. I added a file monitor pointing to a specific index and it was still empty after an hour or so. When I tried the same thing in the lab I was able to see reindexed events after a few minutes (takes a while to unpack the zip files and get the stuff in).
Searched the index, no results found, checked Settings>Indexes, event count 0.
However I will be checking it again tomorrow...just in case 🙂
Try adding crcSalt = <SOURCE> in inputs.conf and restart splunkforwarder on client machines.
No forwarder in play here though we may have to set one up to get around this problem.
This is on the indexer. I pieced together a procedure using btprobe to get the key and reset it. Tried it on my lab system, where it worked.
On the customer system, it didn't. Same version of Splunk and OS linux. The key wasn't being found, even though there was a result from call _internal to get the TailingProcessor state.
Plainly there is some sort of file state being stored somewhere else. What I want is to reindex this zipfile, no questions asked. And since cleaning out the fishbucket should reindex everything, and didn't, I'd dearly like to know what's going on.
Having this same issue now. Did you ever figure this out?
@sahr, you better open a new thread and refer to this one for reference...