topic Re: Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results in Deployment Architecture

Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results

hexx — Sat, 23 Apr 2011 18:49:49 GMT

This error banner is displayed every time a search is run, whether from the search bar or for searches embedded in views :

No search results are displayed from the UI, but CLI searches are working.

What is going on here? How can this be resolved?

Re: Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results

hexx — Mon, 28 Sep 2020 09:29:13 GMT

This error means that the search artifact (the file package containing the search results) requested by Splunk Web could not be found in $SPLUNK_HOME/var/run/splunk/dispatch for the search that was just dispatched.

In the error show above, the path to the real-time search artifact requested should be $SPLUNK_HOME/var/run/splunk/dispatch/rt_1302277839.33, but there is nothing there.

This problem commonly happens when the $SPLUNK_HOME/var/run/dispatch directory is hosted on a network device with a time setting behind the system clock of the operating system where splunkd is running.

What happens then is that the search artifacts are created in the dispatch directory with a modification time behind the system time known to splunkd :



[root@splunk-search-head dispatch]# pwd

/opt/splunk/var/run/splunk/dispatch

[root@splunk-search-head dispatch]# touch test ; ls -l test ; date

-rw-r--r-- 1 root root 0 Apr 22 22:16 test      <=== creation time assigned to the file by 

                                                        the NAS device : 22:16pm

Fri Apr 22 22:39:58 GMT 2011                    <=== system time at which the file was

                                                     created : 22:39pm

As you can see here, any file newly-created in the dispatch directory has a modification time 23 minutes behind the system time.

This means that any new search artifact created by an ad-hoc search will be evaluated by splunkd as having exceeded the default TTL (Time To Live) of 15 minutes and will therefore be immediately deleted.

For that reason, it is very important to ensure that the clock of any NAS device hosting the search dispatch directory is in synch with the clock of the system on which splunkd is running.

Re: Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results

mzax — Thu, 09 Jun 2011 20:57:34 GMT

More detailed command to use from the NFS location:

touch var/run/splunk/dispatch/test ;  ls -l --time-style=full var/run/splunk/dispatch/test ; date

On Solaris it's:

touch var/run/splunk/dispatch/test ; ls -l -E var/run/splunk/dispatch/test ; date

Re: Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results

Masa — Thu, 01 Sep 2011 01:27:24 GMT

hexx: This is update;

limits.conf.spec says;

[search]

ttl =
* How long search artifacts should be stored on disk once completed, in seconds.
* Defaults to 600, which is equivalent to 10 minutes.

Re: Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results

mwong — Mon, 20 Feb 2012 04:16:12 GMT

I have a similar case about that issue. Changing the time of the machine and close to NFS server time difference. The issue has gone.

Re: Getting an error banner "[HTTP 404] (...) [{'text':'Unknown sid.','code': None, 'type': 'FATAL'}] when searching in Splunk Web and no results

charles_cui — Thu, 20 Dec 2012 07:38:41 GMT

how can approach to the problem ?