topic Re: Deployment Monitor not getting data to the summary_* indexes in Deployment Architecture

Deployment Monitor not getting data to the summary_* indexes

mikelanghorst — Mon, 28 Sep 2020 11:24:21 GMT

When I go into the DeploymentMonitor app to All Sourcetypes, the reports show No Results. In fact searching: index=summary_sourcetypes also shows no data. So looking at my search head and indexers, I have no data in $SPLUNK_DB/summary_sourcetypes at all.

When I clicked on the "flush and backfill summary indexes" seems to do little more than creating a very large number of jobs in the dispatch directory on the search head.

What am I missing here? The indexes are created on the indexers and the search head, and other data is forwarded just fine from the search head to the indexer.

Re: Deployment Monitor not getting data to the summary_* indexes

araitz — Thu, 16 Feb 2012 00:20:41 GMT

Can you verify that your issue is not the same as http://splunk-base.splunk.com/answers/34532/deployment-monitor-issue-no-data-in-summary-indexes ?

If not, are you running search head pooling? What version of Splunk are you running?

Re: Deployment Monitor not getting data to the summary_* indexes

mikelanghorst — Mon, 28 Sep 2020 11:24:37 GMT

/app/splunk/var/log/splunk is indeed enabled, and I can search for splunkd messages. But not seeing any messages related to summary_sourcetypes in splunkd.log.

I'm running 4.2.3 with no search head pooling. Looking at each summary_* index:
summary_forwarders - have buckets here
summary_hosts - no buckets
summary_indexers - have buckets here
summary_pools - no buckets
summary_sources - no buckets
summary_sourcetypes - no buckets

Re: Deployment Monitor not getting data to the summary_* indexes

mikelanghorst — Thu, 16 Feb 2012 17:12:44 GMT

Hmm, maybe it's just the saved searches aren't scheduled to feed these reports? I figured if there was a default report that the required search would be scheduled by default.

Re: Deployment Monitor not getting data to the summary_* indexes

mikelanghorst — Mon, 28 Sep 2020 11:24:45 GMT

Following the trail back from the saved search "All sourcetypes regenerator" the macro sourcetype_metrics didn't work. Which led me to: index=_internal source=*license_usage.log, which also had no data. The tailing processor says the file is being read (100%), but can't currently find the data.

Re: Deployment Monitor not getting data to the summary_* indexes

mikelanghorst — Thu, 16 Feb 2012 19:25:57 GMT

Just realized I didn't actually answer your question alex. The sources are enabled.

Re: Deployment Monitor not getting data to the summary_* indexes

mikelanghorst — Mon, 28 Sep 2020 11:24:47 GMT

With the assistance of Genti on IRC, we found the issue:

I'd configured the Search Head as a SplunkForwarder, to send the data to my indexers. This wasn't routing the license_usage file to the indexers, indicated by the following in "cmd btool outputs list":

forwardedindex.1.blacklist = _.*

I've added a monitor for that specific file to route it, adding:
[monitor://$SPLUNK_HOME/var/log/splunk/license_usage.log]
_TCP_ROUTING = *
index = _internal

Now my searches are returning data for this source

Re: Deployment Monitor not getting data to the summary_* indexes

Genti — Thu, 16 Feb 2012 23:49:22 GMT

I'm putting my name here just to:

Splunk > Trolling for upgoats!