Getting Data In

Multiple Worldwide Searchers, Multiple Timezones, Multiple Indexers and Search Heads

dskillman
Splunk Employee
Splunk Employee

I've looked at a few Q&As but nothing really seems to answer my question but this one seems to explains the problem most simply:

http://answers.splunk.com/questions/8116/users-searching-in-different-timezones

But the use case is a bit larger than this. Let's say I have people who need to run searches that live on 3 continents but they all need global visibility into the data that lives in indexers in 3 different global data centers. Their job is to make sure that all locations are fufulling time based (relative to the local time zone of the data center) SLAs and would like to have a single view that shows all of these events.

A 'simple' search example would be show me all transactions that finished later than 9:15AM wherever they may occurred.

The 'best' answer is to log into 3 different search heads in each location and get the results, but what if you have 10 DCs and/or timezones? The 'hack' is to mess with timezone settings at the indexer lever and use one search head, which has it own issues of breaking time series correlation.

It's painfully close and seems like Splunk should have a search command, function or subset of distributed search that could ignore the offsets imposed during a default distributed search and take the actual hour of the event. Any ideas?

0 Karma

bbingham
Builder

I'm not sure if this is correct on my assumption, but, Splunk can search based on epoch time, and since epoch is a reference to GMT, in theory it would be the exact same time no matter what the server is that indexed the data. Have you tried just using the epoch time for your "earliest" and "latest" dashboard search?

example:

 index="main" "*Blah*" earliest=1292544000

would search for Blah at midnight GMT on dec. 17. Not sure it will accomplish what you're looking for, but might work.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Get Agentic with Splunk Lantern: Connect to Cisco Cloud Control, Transform ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

July Community Events: Master ITSI 5.0 & Automate Splunk

Struggling with alert fatigue or feeling like you're spending more time on infrastructure maintenance than ...

New Release of Federated Search: Bringing Splunk Analytics to More of Your Data

Organizations today are generating more data than ever and storing it across cloud object stores, data lakes, ...