Setting max_searches_per_process = 1 in limits.conf resolved the issue.
http://docs.splunk.com/Documentation/Splunk/latest/admin/limitsconf
max_searches_per_process = <int>
* On UNIX, specifies the maximum number of searches that each search process
can run before exiting.
* After a search completes, the search process can wait for another search to
start and the search process can be reused.
* When set to “0” or “1”: The process is never reused.
* When set to a negative value: There is no limit to the number of searches
that a process can run.
* Has no effect on Windows if search_process_mode is not "auto”.
* Default: 500
... View more