topic Re: index=_interospection in Deployment Architecture

index=_interospection

mitali — Fri, 21 Jan 2022 15:48:02 GMT

Unable to see my host in index=_interospection /_internal

after runing the above query in the same host I can't see the hostname.

Unable to see host ES

Re: index=_interospection

gcusello — Fri, 21 Jan 2022 15:56:22 GMT

Hi @mitali,

let me understand:

you runned a search on _internal and in the result list there isn't the hostname of the ES Search Head, is it correct?

Have you this condition also enlarging the time period of your Search?

A very stupid question: did you checked the hostname of your Search Head on the server.conf file ?

Anyway, sometimes _internal logs indexing is delayed when the Indexers are very busy, but you should have many warning messages about this.

For this reason I hint to check the hostname.

Please share more details.

Ciao.

Giuseppe

Re: index=_interospection

mitali — Fri, 21 Jan 2022 16:01:17 GMT

Yes running index=_internal on ES search head but not showing ES host name

yes the hostname in server.conf file is correct.

yes even after expanding timerange hostname is not available.

Re: index=_interospection

gcusello — Fri, 21 Jan 2022 16:05:01 GMT

Hi @mitali,

did you forwarded the Search Heads logs to the Indexers, as hinted by Splunk best practices?

Ciao.

Giuseppe

Re: index=_interospection

mitali — Fri, 21 Jan 2022 16:08:14 GMT

Can you Please tell me how to do that?

Re: index=_interospection

gcusello — Fri, 21 Jan 2022 16:19:56 GMT

Hi @mitali,

from the Splunk menu on Each Splunk server (except Indexers): [Settings -- Forwarding and Receiving]:

[Forwarding Default -- Save ]

[Configure Forwarding -- New Forwarding host] add indexers

Ciao.

Giuseppe

Re: index=_interospection

mitali — Fri, 21 Jan 2022 16:30:46 GMT

indexers are already aaded

this is correct?

Re: index=_interospection

gcusello — Fri, 21 Jan 2022 16:46:15 GMT

Hi @mitali,

Check if there's the port in the destination: "indexer_name:9997".

Then, remember to click "Save" in the "Default configuration", it will ask you a restart.

Then, On your indexers, do you receive also from Forwarders or only by syslog?

Ciao.

Giuseppe

Re: index=_interospection

mitali — Fri, 21 Jan 2022 16:59:16 GMT

everything is correct just that hostname is not showing up

Re: index=_interospection

mitali — Fri, 21 Jan 2022 17:20:39 GMT

Error [00000080] Instance name "BCCS-P25ES." REST interface to peer is taking longer than 5 seconds to respond on https. Peer may be over subscribed or misconfigured. Check var/log/splunk/splunkd_access.log on the peer Last Connect Time:2022-01-21T22:45:45.000+05:30; Failed 11 out of 11 times.

showing this error in monitoring console.

Re: index=_interospection

gcusello — Sat, 22 Jan 2022 07:05:44 GMT

Hi @mitali,

try to use Ip address instead hostname in the "Add indexers" form:

10.10.10.10:9997 10.10.10.11:9997

and then (using telnet) check if the route between SH and IND is open:

telnet <ip_Indexer> 9997

Ciao.

Giuseppe

Re: index=_interospection

mitali — Tue, 25 Jan 2022 16:10:40 GMT

Problem sloved solution was to create outputs.conf on Search head

Re: index=_interospection

gcusello — Tue, 25 Jan 2022 16:58:30 GMT

Hi @mitali,

really strange: using the GUI as I hinted has the same result!

Anyway, if you solved, please accept the answer for the other people of Community.

Ciao and happy splunking.

Giuseppe

P.S.: Karma Points are appreciated by all the Contributors 😉