Splunk Search

After adding a new Splunk server in a distributed environment, why does it not show up in results unless I include splunk_server=*?

louieb3
Path Finder

I recently added a new splunk server in a distributed environment. Now, when I do this search:

index=os earliest="09/01/2015:09:30:00" latest="09/01/2015:09:35:00" | timechart count by splunk_server

the new splunk server does not show up in the results. However, if I do this search,

index=os splunk_server=* earliest="09/01/2015:09:30:00" latest="09/01/2015:09:35:00" | timechart count by splunk_server

then, it shows up.

Can anyone tell me why? I have the search load-balanced so I have about the same number of events going into each indexer.
Thank you in advance.

0 Karma
1 Solution

woodcock
Esteemed Legend

What is inside distsearch.conf? How did you add the Indexer? Are you using Search Head Pooling?

See this question, too:

http://answers.splunk.com/answers/221468/search-returns-zero-results-searchlog-reports-dist.html

View solution in original post

woodcock
Esteemed Legend

What is inside distsearch.conf? How did you add the Indexer? Are you using Search Head Pooling?

See this question, too:

http://answers.splunk.com/answers/221468/search-returns-zero-results-searchlog-reports-dist.html

louieb3
Path Finder

That was it. I looked at distsearch.conf and saw that all of my indexers except for the new one was in the [distributedSearch:dmc_group_indexer] stanza.

I went into the Distributed Management Console, under Remote instances, edited the Server Role for the new indexer (it was already configured as an indexer), saved it, and then applied the changes and voila, issue resolved. Thanks, woodcock!

To answer your question, in distsearch.conf, I had the stanzas [distributedSearch] which contained all of the indexers and [distributedSearch:dmc_group_indexer] which also contained a list of my indexers except for the recently added one.

0 Karma
Get Updates on the Splunk Community!

Demo Day: Strengthen Your SOC with Splunk Enterprise Security 8.1

Today’s threat landscape is more complex than ever. Security operation centers (SOCs) are overwhelmed with ...

Dashboards: Hiding charts while search is being executed and other uses for tokens

There are a couple of features of SimpleXML / Classic dashboards that can be used to enhance the user ...

Splunk Observability Cloud's AI Assistant in Action Series: Explaining Metrics and ...

This is the fourth post in the Splunk Observability Cloud’s AI Assistant in Action series that digs into how ...