Getting Data In

Splunk and Different Timezones

conor_splunk
Path Finder

Hi All,

I have a scenario where I am indexing event logs from Windows servers across 5 different time zones:

  • Australia/Queensland
  • Australia/NSW
  • NZ
  • Australia/South
  • Australia/Victoria
  • Australia/West

All of the servers are sending their security log to forwarders and from there they are sent to the indexers. So I've got an issue where when I search logs in real time nothing is showing up, I've got to look forwarder 2 hours or backward 2 hours to see logs coming in. I'm trying to figure out how on a forwarder I can set the TZ for specific servers. Looking at http://docs.splunk.com/Documentation/Splunk/4.1.8/admin/ApplyTimezoneOffsetstotimestamps, it looks like it should be simple but I can't seem to figure it out.

Is there something I can put on the UniversalForwarder config or on the Indexer that will allow me to apply a Time Zone to specific servers? Reading the article above it seems to suggest the Time Zone should be that of the server sending the data. But they all hit the props stanza [WinEventLog://Security] so I'm a bit stuck.

My forwarder configuration is as follows.

inputs.conf

[splunktcp://9997]
disabled = 0

props.conf

[WinEventLog:Security]
TRANSFORMS-security = security_null, security_truncate, security_index

transforms.conf

[security_null]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[security_truncate]
REGEX = (?msi)(.*)(This event is generated|Certificate information is only provided)
DEST_KEY = _raw
FORMAT = $1

[security_index]
REGEX = (?msi)^EventCode=(list of event Id's removed for brevity)\D
DEST_KEY = queue
FORMAT = indexQueue

Everything is working excellent, except how can I specify a different timezone for different servers in the props.conf if the stanza is looking for [WinEventLog:Security]?

The configuration for the servers is all identical.

inputs.conf

 [WinEventLog://Security]
 checkpointInterval = 5
 current_only = 1
 disabled = 0
 index = security
 start_from = oldest

outputs.conf

 [tcpout]
 defaultGroup = forwarder_9997

 [tcpout:forwarder_9997]
 server = forwarder:9997

 [tcpout-server://forwarder:9997]
0 Karma
1 Solution

somesoni2
Revered Legend

You would need to update the props.conf on Splunk INdexer (or heavy forwarder). See the last answer in the post "http://answers.splunk.com/answers/27312/setting-other-tz-in-propsconf"

View solution in original post

somesoni2
Revered Legend

You would need to update the props.conf on Splunk INdexer (or heavy forwarder). See the last answer in the post "http://answers.splunk.com/answers/27312/setting-other-tz-in-propsconf"

conor_splunk
Path Finder

Well it seems to be working at the moment.

I added:

[host::PERTH*]
TZ = Australia/Perth

So the server is this time zone is now reporting the right times. But...another server in another TZ which is still on the default stanza in props.conf is not having any issues either. So I can't tell if it was just a case of the forwarder and/or indexer playing catch up or if the setting fixed it.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...