I've been scratching my head for a couple of days trying to figure this out without success. In following the Summary Index Knowledge Manual article titled "Usesummaryindexing" I believe I have the search scheduled correctly. I don't have the karma to post screenshots or links, so I'll do my best to describe the problem. search name "summary_prod_5min" search: index=prod | rex "(?<"LOG">.{300})" | sitop limit=100 LOG (I had to quote "LOG" for the wiki text parser. the actual search contains no quotes around LOG) start time: -6m end time: -1m (to account for the slight lag in indexing..) scheduled every 5 minutes (2,7,12,17,22,27,etc etc) all default alert settings with no email, script, triggered alerts, etc. summary indexing enabled to summary index called "ops_summary" added field "report" with value of "prod_5min" As far as I can tell, this is correctly configured. I also made sure that the ops_summary index is configured on the search head: (root@splunkSH) splunk > bin/splunk cmd btool indexes list | sed -n -e '/ops_summary/,/\[/ p' [ops_summary] assureUTF8 = false blockSignSize = 0 blockSignatureDatabase = _thefishbucket bucketRebuildMemoryHint = auto coldPath = $SPLUNK_DB/ops_summary/colddb coldPath.maxDataSizeMB = 0 coldToFrozenDir = coldToFrozenScript = compressRawdata = true defaultDatabase = _thefishbucket enableOnlineBucketRepair = true enableRealtimeSearch = true frozenTimePeriodInSecs = 188697600 homePath = $SPLUNK_DB/ops_summary/db homePath.maxDataSizeMB = 0 hotBucketTimeRefreshInterval = 10 indexThreads = auto maxBloomBackfillBucketAge = 30d maxBucketSizeCacheEntries = 0 maxConcurrentOptimizes = 1 maxDataSize = auto maxHotBuckets = 3 maxHotIdleSecs = 0 maxHotSpanSecs = 7776000 maxMemMB = 5 maxMetaEntries = 1000000 maxRunningProcessGroups = 8 maxRunningProcessGroupsLowPriority = 1 maxTimeUnreplicatedNoAcks = 300 maxTimeUnreplicatedWithAcks = 60 maxTotalDataSizeMB = 100000 maxWarmDBCount = 300 memPoolMB = auto minRawFileSyncSecs = disable minStreamGroupQueueSize = 2000 partialServiceMetaPeriod = 0 processTrackerServiceInterval = 1 quarantineFutureSecs = 2592000 quarantinePastSecs = 77760000 rawChunkSizeBytes = 131072 repFactor = 0 rotatePeriodInSecs = 60 serviceMetaPeriod = 25 serviceOnlyAsNeeded = true serviceSubtaskTimingPeriod = 30 streamingTargetTsidxSyncPeriodMsec = 5000 suppressBannerList = _audit sync = 0 syncMeta = true thawedPath = $SPLUNK_DB/ops_summary/thaweddb throttleCheckPeriod = 15 tstatsHomePath = volume:_splunk_summaries/$_index_name/datamodel_summary warmToColdScript = The next step was to make sure the search returned results. I went to the search, and clicked "view recent" where I see each run does produce results; in the neighborhood of ~400k per run. I clicked "inspect" and the things that I would suspect to be there, are there. (summary index name is correct, input and output counts are there, summary_index action, etc.) From what I can tell, everything indicates the summary index is configured appropriately. When I do a simple search, I get no results: earliest=-1h index=ops_summary I tried to be more specific, and it's still empty: earliest=-1h index=ops_summary report=prod_5min Can anyone lend some advice on where to look for trouble? I'm happy to provide any additional details as well. ... View more

I've found this on the Splunk wiki that gives great examples on how to graph several sources and their cumulative totals: http://wiki.splunk.com/Community:Search_Report:_How_To_Create_a_Chart_of_Hourly_and_Accumulated_Index_Volume Is it possible to use a similar method, but to only graph the cumulative total and not each of the individual sources that make up that total? ... View more