Getting Data In

How to set the time zone/alias for syslog data

AhmadKhattak20
Explorer

Hi There,

So, the scenario is that we have a central syslog server which receives syslog messages from different servers in the organization. 

When the syslog data is received on the syslog server, there is a universal forwarder agent on the syslog server that forwards it to Splunk. The issue is that some servers are using UTC time zone and therefore the syslog data from those servers contains UTC Timestamp. 

Is there a way to change how the forwarder interprets the syslog data file received from those servers? I've tried editing TZ=UTC and TZ-ALIAS=UTC in props.conf with the source stanza specifying the path for those specific log files that have UTC Timestamp in events. However in Splunk I still see those events with the UTC Time Stamp. 

This is an issue due to which we can't properly search. Any advice would be much appreciated. Thanks.

Labels (4)
0 Karma
1 Solution

javiergn
SplunkTrust
SplunkTrust

OK, I think I know what the problem is thanks to your answer.

Your props.conf needs to go to your indexer (or Intermediate/Heavy Forwarder if there is any before reaching the Indexer). That's because you are using a Universal Forwarder and the TZ setting in props.conf is applied at parsing time:

https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

 

View solution in original post

0 Karma

AhmadKhattak20
Explorer

Hi @richgalloway , yes there is a different directory and file maintained. Although I've not used the inputs.conf to set the TZ property rather I've used the props.conf to set the TZ property.

Hi @javiergn, Yes, I've used this source stanza in the props.conf file however the events from the file still show up with UTC Timestamp. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You're right, the TZ property is set in props.conf, but it should be done on the UF.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

javiergn
SplunkTrust
SplunkTrust

Can you send a screenshot of the event's raw and _time?

Simple run  your search:

index=foo sourcetype=bar source=yoursyslogsource
| head 1
| table _time, _raw

And also confirm what your user timezone is within the interface:

Screenshot 2021-05-18 at 09.20.19.png

Regards,

J

 

0 Karma

AhmadKhattak20
Explorer

Hi @javiergn ,

I ran the command that you gave me,

 

index=indexname sourcetype=syslog source=sourcename
| head 1
| _raw, _time

 


The result that I got was the following, (only showing the timestamps result)

_raw_time
May 19 04:30:01 2021-05-19 04:30:01

 

It seems that Splunk is still extracting the time stamp from the event data itself and not converting the UTC time stamp to the one being used in Splunk Preferences. 

The time zone preference set in Splunk User Preferences is below

s1.PNG

 

 

 

 

 

 

For reference what I've done in props.conf on the UF running on the Syslog Server is following,

 

[source::/path/*.log]
TZ=UTC
TZ_ALIAS=UTC

 

Tags (1)
0 Karma

javiergn
SplunkTrust
SplunkTrust

Hi @AhmadKhattak20,

Couple of things:

- You don't need the TZ_ALIAS. TZ = UTC is perfectly valid

- Where is this props.conf located within your Splunk installation directory?

- Can you also paste the value of your source so that we can validate the stanza? Use the following query

index=indexname sourcetype=syslog source=sourcename
| head 1
| table _raw, _time, source
0 Karma

AhmadKhattak20
Explorer

Hi @javiergn,

I've removed the TZ_ALIAS from the props.conf.

I'm pushing the app from the deployment server onto the Syslog Server where a Splunk UF is installed.

On the Syslog Server, this is located under /opt/splunk/etc/apps/custom-app-folder/local/props.conf

I ran the query that you mentioned and the results are following,

_raw_timesource
May 19 09:10:01 ….2021-05-19 09:10:01/var/splunk/path/ipaddress/2021-05-19-servername.log


I verified that the source that was showing up in the query results is the same that I'm using in the props.conf stanza for source, (this props.conf is pushed on the syslog server - it is not present in any indexers/search head)

[source::/var/splunk/path/ipaddress/*.log]
TZ = UTC

 

 

0 Karma

javiergn
SplunkTrust
SplunkTrust

OK, I think I know what the problem is thanks to your answer.

Your props.conf needs to go to your indexer (or Intermediate/Heavy Forwarder if there is any before reaching the Indexer). That's because you are using a Universal Forwarder and the TZ setting in props.conf is applied at parsing time:

https://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

 

0 Karma

AhmadKhattak20
Explorer

Thank you, I pushed the props.conf with the below stanza on indexers and now I'm getting expected results.

[source::/var/splunk/path/ipaddress/*.log]
TZ = UTC
0 Karma

javiergn
SplunkTrust
SplunkTrust

Great. Glad it worked.

Please don't forget to upvote the answers if you are happy with them.

Regards,

J

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Have the syslog server put each source server's data into a different file.  The Universal Forwarder should monitor each file.  The inputs.conf file for the UF will have the appropriate TZ setting for each monitored file.

---
If this reply helps you, an upvote would be appreciated.
0 Karma

javiergn
SplunkTrust
SplunkTrust

Hi,

You can use the source:: and host:: stanzas within props.conf in order to edit the time zone for specific filepaths or host names. I normally use the source as it is unique to this particular type of data (Syslog) as opposed to the host. For instance:

 

[source::/var/log/syslog/firewall/myserverfilename*.log]
TZ=Europe/Madrid

 

See https://en.wikipedia.org/wiki/List_of_tz_database_time_zones

Regards,

J

 

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...