Using the api I am submitting searches to splunk. Sometimes, the jobs remain in queued state forever. I can see when I query for a queued job that it is saying 'this search could not be dispatched because of role-based concurrency ' - which would be fine - if I could see the other jobs for that user.
Using the /search/jobs api I can see other jobs and I can filter the results for those created by my user. When that search shows me there are other jobs, I have deleted them, so that api returns no jobs for my user - which I would expect should mean I can submit new jobs - but I still see jobs getting stuck in the queued state. It feels like my search for existing jobs is not returning the full list for some reason, but I don't know what that would be.
Any help would be very welcome. this is on splunk enterprise 8.0.4