Consider a search which spans events in indexes in London and New-York.
| mysearch
| bucket _time span=1d
| stats count by _time
When I run this search on a London search head, it seems to work, all _time returned are snapped at the beginning of the London day:
"2015-08-04 00:00:00"
When the same search runs from scheduler, the _time are sometimes (it's not consistent) snapped at the beginning of New York days.
The scheduled search actually re-indexes its results somewhere as summary stats. When I query these summary stats from London search head, I therefore get this _time:
"2015-08-04 05:00:00"
According to me, a same search, whether it's run from a user or the scheduler, should always return the same thing.
Is there a way to force the location which "snaps" the _time in the query language, or from configuration?
This should be controlled by the timezone
setting for the user on the search head where the saved search is run. If it is being run on serverX and userY, then log into serverX as userY and go to serverX
-> Edit account
-> Time zone
and set to what you desire and that is how it will work thereafter.
What happens if there is one single search head cluster which contains search head from various regions? In this case, the user configuration will be from the same deployer which means we can't specify different time zone settings for the same user.
And I think Splunk advise NOT to edit user setting on individual Splunk clustered search head server locally.
Please advise. Thanks.
I do not understand. In a Search Head cluster, the user settings should be automatically aligned so it won't matter which Search Head runs the search because they all have the same TZ settings for userX so the answer is the same for a cluster as it is for a single Search Head.
Thanks for that comment. The answer looks spot on. We are in the process of analysing how this solution can be implemented on our complex set-up. And I'll let you know if it worked.
Did it work?