Need a little help here. I'm experiencing an error "Search process did not exit cleanly, exit_code=255, description="exited with code 255". Please look in search.log for this peer in the Job Inspector for more info.". I'm using Splunk 6.0.3, because of that error i cant search . I don't know the cause of this error. I also tried to look here in community if there are users out there experienced this, yes i found some but no concrete solution.
Please enlighten me 😞
After the following line of error, you would find 1 more line, which will give you name of lookup or knowledge object which is failing while replication.
"Search process did not exit cleanly, exit_code=255",
...[server] Streamed search execute failed because: Error in 'lookup' command: The lookup table 'abc.csv' does not exist.
Try adding local=t in your search. This will direct Splunk to look for this csv only on search head and not indexer and remove the error.
Good Luck !!!
Is is a distributed search ? and do the remote search.log in the search inspector mentions errors about unknown users or roles ?
Because those are the classic symptoms when the searchbundles were not copied to the search-peers. (or were somehow expired).
A quick test, is to go to the mentioned search-peer and look for the search bundle folder.
$SPLUNK_HOME/var/run/searchpeer/ look at the modification time, and remove the bundle with the name of the search-head.
Then retry a search, you should see the new bundle be copied.