Archive
Highlighted

Getting the error "Search process did not exit cleanly, exit_code=255, description="exited with code 255". Please look in search.log for this peer in the Job Inspector for more info." frequently

Explorer

Dear everyone,
Have a good day ahead.

I am having the following issue that need your advice. Recently, I have deployed Splunk in distributed environment as the following:
- 01 Master + License master
- 01 Search Head
- 02 Indexer
- 01 Heavy Forwarder

Without installing app on Search Head, the application is working fine without any error. However, whenever I install app on SH, the following error is appeared for one of our Indexing system:
"Search process did not exit cleanly, exit_code=255, description="exited with code 255". Please look in search.log for this peer in the Job Inspector for more info."

By checking the search.log, we see a lot of the following error:
12-03-2018 14:53:28.293 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=PREVIEW
12-03-2018 14:53:28.701 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:28.701 WARN SRSSerializer - could not read schema
12-03-2018 14:53:28.723 INFO TimelineCreator - Commit timeline at cursor=1543804147.000000
12-03-2018 14:53:28.724 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=PREVIEW
12-03-2018 14:53:29.073 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:29.073 WARN SRSSerializer - could not read schema
12-03-2018 14:53:29.095 INFO TimelineCreator - Commit timeline at cursor=1543803804.000000
12-03-2018 14:53:29.096 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=PREVIEW
12-03-2018 14:53:29.601 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:29.601 WARN SRSSerializer - could not read schema

Due to this error, I cannot search any event which is indexed by the problematic node.
Can you please advice how I should proceed further to fix this issue?

Thank you for your time in advance.
Regards,
Anh

Tags (1)
0 Karma
Highlighted

Re: Getting the error "Search process did not exit cleanly, exit_code=255, description="exited with code 255". Please look in search.log for this peer in the Job Inspector for more info." frequently

Communicator

Hello Anh,

look at this Troubleshooting...
https://helgeklein.com/blog/2017/07/troubleshooting-splunk-error-search-process-not-exit-cleanly/

Does it work?

Best wishes,
Jacob

Highlighted

Re: Getting the error "Search process did not exit cleanly, exit_code=255, description="exited with code 255". Please look in search.log for this peer in the Job Inspector for more info." frequently

Explorer

thanks Jacob for your help.
I have gone through this post but still cannot fix the error. Actually, my search is very simple: sourcetype=pan:traffic
I have 2 indexers and only 1 indexer is having this issue.
Still cannot figure out what's wrong...

Speak Up for Splunk Careers!

We want to better understand the impact Splunk experience and expertise has has on individuals' careers, and help highlight the growing demand for Splunk skills.