Join the Conversation
- Find Answers
- :
- Splunk Administration
- :
- Admin Other
- :
- Knowledge Management
- :
- Re: Search process did not exit cleanly, exit_code...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Search process did not exit cleanly, exit_code=255, description="exited with code 255
Dear everyone,
Have a good day ahead.
I am having the following issue that need your advice. Recently, I have deployed Splunk in distributed environment as the following:
- 01 Master + License master
- 01 Search Head
- 02 Indexer
- 01 Heavy Forwarder
Without installing app on Search Head, the application is working fine without any error. However, whenever I install app on SH, the following error is appeared for one of our Indexing system:
"Search process did not exit cleanly, exit_code=255, description="exited with code 255". Please look in search.log for this peer in the Job Inspector for more info."
By checking the search.log, we see a lot of the following error:
12-03-2018 14:53:28.293 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=PREVIEW
12-03-2018 14:53:28.701 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:28.701 WARN SRSSerializer - could not read schema
12-03-2018 14:53:28.723 INFO TimelineCreator - Commit timeline at cursor=1543804147.000000
12-03-2018 14:53:28.724 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=PREVIEW
12-03-2018 14:53:29.073 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:29.073 WARN SRSSerializer - could not read schema
12-03-2018 14:53:29.095 INFO TimelineCreator - Commit timeline at cursor=1543803804.000000
12-03-2018 14:53:29.096 INFO ReducePhaseExecutor - ReducePhaseExecutor=1 action=PREVIEW
12-03-2018 14:53:29.601 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:29.601 WARN SRSSerializer - could not read schema
Due to this error, I cannot search any event which is indexed by the problematic node.
Can you please advice how I should proceed further to fix this issue?
Thank you for your time in advance.
Regards,
Anh
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
exit_code = 255 was about no disk space left, in our case, at Why do we get the exited with code 255 errors?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
the system is having nearly 1 TB free space. So I dont think this related to space on system.
Furthermore, I saw the following error on search.log:
12-03-2018 14:53:29.073 ERROR SRSSerializer - could not read number of columns
12-03-2018 14:53:29.073 WARN SRSSerializer - could not read schema
Is there anyone know what this error mean?
Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!
Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.
[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions
Splunk Community Badges!
[Puzzles] Solve, Learn, Repeat: Matching cron expressions
