distributed search query works (kinda) but only returns single

[sdewar83]

Hi,

We have 10 sites each with their own splunk server (search head, indexer etc). Each is collecting the same information and has the same index names. I want to run a distributed search queries so that i dont have to log onto each of them and query them individually. I know you can edit the .conf file and create distributed search groups but i'd need to log an RFC for that, so as a proof of concept i just wanted to try and do it using the splunk_server= command. If i choose a search that works fine one search head and add in some logic to try and send it to multiple search heads, it seems to return a single result and I can't seem to get it to show multiple figures.

e.g i'm trying stuff like:

index=* OR index=_* AND splunk_server=yyyyyyyyyyyyy OR splunk server=xxxxxxxxxxxxxxxxx
| fields, sourcetype, _raw
| eval size-len(_raw)
|stats sum(size) as size
| eval size=round(size/1024/1024,2)

but no joy? i'd have hoped it'd show the MB size of raw data capture by the servers at both sites. I think it only shows yyyyyyyyyyyy.

p.s also if i piped it to a table, what field would i have to use to display which search head the respective results came from?

Many thanks,

[adonio]

try this, what are the results?

index=* OR index=_*  (splunk_server=yyyyyyyyyyyyy OR splunk_server=xxxxxxxxxxxxxxxxx)
| fields, sourcetype, _raw
| eval size=len(_raw)
|stats sum(size) as size by splunk_server
| eval size=round(size/1024/1024,2)

[sdewar83]

Hmmmn.

I tried your suggestion and it came up with 0 events. I tried using FQDNs for the server names, no joy. Tried FQDN:port, no joy. No joy either for IP or IP:port. Splunk_Server=* seems to work. (p.s is the port the same port number thats in the web console url or is it 8089? i tried both, no joy)

i can't even get it to work at all now. not sure what's changed. I can't even get splunk_server=local to return a result. Either i dont use the command and the search runs as normal or i use splunk_server=*.

[adonio]

i missed an underscore _ in my search, and fixed it

when you are searching this:

index=_internal  splunk_server=*
 | fields, sourcetype, _raw
 | eval size=len(_raw)

do you see the field size ?