Deployment Architecture

distributed search query works (kinda) but only returns single

sdewar83
Path Finder

Hi,

We have 10 sites each with their own splunk server (search head, indexer etc). Each is collecting the same information and has the same index names. I want to run a distributed search queries so that i dont have to log onto each of them and query them individually. I know you can edit the .conf file and create distributed search groups but i'd need to log an RFC for that, so as a proof of concept i just wanted to try and do it using the splunk_server= command. If i choose a search that works fine one search head and add in some logic to try and send it to multiple search heads, it seems to return a single result and I can't seem to get it to show multiple figures.

e.g i'm trying stuff like:

index=* OR index=_* AND splunk_server=yyyyyyyyyyyyy OR splunk server=xxxxxxxxxxxxxxxxx
| fields, sourcetype, _raw
| eval size-len(_raw)
|stats sum(size) as size
| eval size=round(size/1024/1024,2)

but no joy? i'd have hoped it'd show the MB size of raw data capture by the servers at both sites. I think it only shows yyyyyyyyyyyy.

p.s also if i piped it to a table, what field would i have to use to display which search head the respective results came from?

Many thanks,

Tags (1)
0 Karma

adonio
Ultra Champion

try this, what are the results?

index=* OR index=_*  (splunk_server=yyyyyyyyyyyyy OR splunk_server=xxxxxxxxxxxxxxxxx)
| fields, sourcetype, _raw
| eval size=len(_raw)
|stats sum(size) as size by splunk_server
| eval size=round(size/1024/1024,2)
0 Karma

sdewar83
Path Finder

Hmmmn.

I tried your suggestion and it came up with 0 events. I tried using FQDNs for the server names, no joy. Tried FQDN:port, no joy. No joy either for IP or IP:port. Splunk_Server=* seems to work. (p.s is the port the same port number thats in the web console url or is it 8089? i tried both, no joy)

i can't even get it to work at all now. not sure what's changed. I can't even get splunk_server=local to return a result. Either i dont use the command and the search runs as normal or i use splunk_server=*.

0 Karma

adonio
Ultra Champion

i missed an underscore _ in my search, and fixed it

when you are searching this:

index=_internal  splunk_server=*
 | fields, sourcetype, _raw
 | eval size=len(_raw)

do you see the field size ?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...