What's the best way for us to take a look at your data model? e.g. are there fields with spaces in fieldname? ... View more

Is there a stack trace var/log/splunk/python.log with more details? Thanks! ... View more

Did it have any hits for source="User session visitor source*"? The issue is that anytime you use a timerange that spans more than 5 minutes, we're searching a summary index rather than the raw data. The timerange you choose dictates which summary index is searched (e.g. wi_summary_hourly for 00:00:00 to 12:15:31 vs wi_summary_daily when you choose "Month to Date"). Can you perhaps search a 5 minute time window within that range to see if there are events in that time period? ... View more

have you tried the same search using a different time range? for the same time range, what do you see if you search: index=wi_summary_hourly | stats count by source Just checking to see if the summary index is filling up correctly. ... View more

Per araitz, use this.diplayedMessages instead of this.messages ... View more

Nick, this code snippet produces the following error in web_service.log: 2011-11-11 14:18:34,698 ERROR [4ebd9f3ab11bb1aa90] utility:63 - name=javascript, class=Splunk.Error, lineNumber=54, message=this.messages is undefined, fileName= /application.js ... View more

Can you search, any 5 minute time range in the day before to see if you see charts showing up on dashboards? It's not an issue of realtime vs not. Basically, any timerange that exceeds 5 minutes will search summary indexes instead of the raw data. ... View more

Do you see any data if you search for a timerange that's less than 5 minutes? For most of the views, any timerange that's over 5 minutes searches against summary indexes. A simple way to sanity check that your app is configured correctly is to try and search for a timerange when you know there is data and that spans less than 5 minutes. ... View more

what version of DM (and core product) are you using? We'll take a look... ... View more

An indexer is "backed up" if its parsingQueue is over 50% full most of the time. It seems like this is the case based on your queue stats (parsingQueue size seems to be >500 and often 1000). It's very likely that one of your regexes to parse events is too complex/inefficient. ... View more

I did some performance modeling based on data from hadoop logs and have recently gotten up to speed writing saved searches in Splunk. I'd be happy to help you put the two together. ... View more