topic Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers? in Getting Data In

Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Fri, 22 Sep 2017 18:26:27 GMT

In our Slave-Apps directory on the 2 peers/indexers we have a custom app created by the prev admin which has setting for TZ to UTC for network devices that are on UTC. Now i am adding new data source (i.e. AD security logs) using UFs on DCs and our DCs are all in EST TZ and hence i would need to list EST TZ in the props.conf.

My Questions are

1) Is this the right stanza for EST time entry
[WinEventLog://Security]
TZ = US/Eastern

I understand i will have to do this on master-apps folder on cluster master and then apply config bundle

2) Will this require a reboot of any peers ?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Fri, 22 Sep 2017 19:22:38 GMT

Hi there,

1 - Yes thats correct

2 - Yes, the cluster master will initiate a restart of its cluster members once you apply the new cluster bundle. Please see here for what requires a restart and what doesnt..

http://docs.splunk.com/Documentation/Splunk/6.6.3/Indexer/Updatepeerconfigurations#Restart_or_reload_after_configuration_bundle_changes.3F

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Sat, 23 Sep 2017 15:45:54 GMT

Are you sure the time zone is right? or should it be EST?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Sat, 23 Sep 2017 16:15:17 GMT

Sure am:

http://docs.splunk.com/Documentation/Splunk/6.6.3/Admin/Propsconf --search for "The following example sets Eastern Time Zone"

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Tue, 29 Sep 2020 15:52:08 GMT

This is my props.conf on indexers

[cisco:asa]
TZ = UTC

[cisco:ise:syslog]
TZ = UTC

[cisco:acs]
TZ = UTC

[cisco:ios]
TZ = UTC

[cisco:sourcefire]
TZ = UTC

[f5:bigip:syslog]
TZ = UTC

[pan:log]
TZ = UTC

[WinEventLog://Security]
TZ = US/Eastern

[catchall:catchall]
TZ = UTC

it is not working for events that are coming for [WinEventLog://Security] because if i search for last 15mins or 60 mins i dont get results ONLY when i select last 4hours i can see results. I also tried switching my user time zone from UTC to EST through settings>users>my user timezone as EST and log out/login but still the same issue.

And I have installed Splunk_TA_windows on my UF that sits on DC

Any help will be appreciated

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Sat, 23 Sep 2017 18:52:33 GMT

Ok yeah i see that but its not working for medid u see my props.conf in the below comment and also i am defining TZ by sourcetype and not host will that make a difference ?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Sat, 23 Sep 2017 19:03:07 GMT

I tried with host entry as well it did not work , still when i go to last 4hrs only then i can see events otherwise in realtime search or last 15 or 60mins it does not show up

[host::dc1-corpdc01]
TZ = US/Eastern

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Sun, 24 Sep 2017 08:09:15 GMT

What happens if you dont apply a TZ ? What time are you getting for your sourcetype then?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Sun, 24 Sep 2017 08:43:10 GMT

Same. I did not apply a TZ before only when this issue started i realised i should enter TZ in props.conf and entering did not make a difference. I also tried entering TZ by creating a props.conf in UFs local but no joy

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Sun, 24 Sep 2017 10:03:53 GMT

Can you send me an example of the search you are running and a snip of the results?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Sun, 24 Sep 2017 10:10:29 GMT

I am simply typing sourcetype="WinEventLog:Security" in search and i do not find anything when i do last15mins or 60 mins i can only see for last 4hrs and event time is real time like if you convert from UTC to IST (which is the TZ I live in).

Also it does not allow me here to paste an image only an url

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Tue, 29 Sep 2020 15:51:36 GMT

Check if there is some kind of indexing lag in your environment -

source=mysource | eval delay_sec=_indextime-_time | timechart min(delay_sec) avg(delay_sec) max(delay_sec) by host

Alternatively - has that DC host been set a timezone in another app? Can you run a btool to check that?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Tue, 29 Sep 2020 15:51:39 GMT

Use this search to verify the source type, the time stamp detected (_time), the time of the user on the search head (now), and the time zone applied (date_zone)

source=mysource host=myhost | eval delay_sec=_indextime-_time | convert ctime(_indextime) AS indextime | eval now=now() | table _time indextime now date_zone source sourcetype host

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Tue, 29 Sep 2020 15:55:33 GMT

Thanks, I ran the queries and below are the results, there seems to be a 3-4hrs delay and time diff I do not know why, if I run these queries for other sources it does not show any delay or time difference.

source=mysource | eval delay_sec=_indextime-_time | timechart min(delay_sec) avg(delay_sec) max(delay_sec) by host

_time

2017-09-25 06:45:00

avg(delay_sec)

8618

max(delay_sec)
8656

min(delay_sec)
8593

source=mysource host=myhost | eval delay_sec=_indextime-_time | convert ctime(_indextime) AS indextime | eval now=now() | table _time indextime now date_zone source sourcetype host

_time

2017-09-25 06:59:30

indextime

09/25/2017 09:16:39

source

WinEventLog:Security

Also please note to troubleshoot this now I have changed the timezone for all my Splunk servers to match with my Domain controller so now both indexer and source have same TZ = EST but still I am not able to search logs in last 60mins or 15mins.

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

sbbadri — Tue, 29 Sep 2020 15:55:35 GMT

@hrithiktej

props.conf seems correct.

[host::yourhostdefinition]
TZ = US/Eastern

[source::yousourcedefinition]
TZ = US/Eastern

or
[yoursourcetypedefinition]
TZ = US/Eastern

Can you please check path of the props.conf and check it has enough permission for splunk to read.

CM - $SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf --- use this location if you don't have separate app.
CM - $SPLUNK_HOME$/etc/master-apps/your app/local/props.conf -- use this location if you have separate app for it.

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Mon, 25 Sep 2017 14:16:55 GMT

Can you send a snip of your input stanza for the security logs in wineventlog ?

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Tue, 29 Sep 2020 15:55:41 GMT

@mwdbhyat

I see your comment in email notification but not here.

And this is my inputs I am using the default inputs.conf from the Splunk_TA_Windows app.

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
blacklist1 = EventCode="4662" Message="Object Type:(?!\s*groupPolicyContainer)"
blacklist2 = EventCode="566" Message="Object Type:(?!\s*groupPolicyContainer)"
index = wineventlog
renderXml=false

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

mwdbhyat — Mon, 25 Sep 2017 14:30:54 GMT

You should make changes in the local folder, just in case someone comes along and creates a stanza for the same sourcetype and overwrites your settings in default(it wont fix your problem but is a best practice).

Regarding you inputs it looks fine - but there is clearly a lag in indexing.. Can your environment handle the amount of data that is flowing into your indexers from wineventlog?

This article has a few tricks you can try:

https://docs.splunk.com/Documentation/Splunk/6.6.3/Troubleshooting/Troubleshootingeventsindexingdelay

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Tue, 29 Sep 2020 15:55:49 GMT

@mwdbhyat

I changed these two values in inputs.conf

start_from = newest
current_only = 1

and it resolved my issue THANK YOUUUUU SO MUCH MAN! for some reason i dont see your comment here can you please paste it again I want to mark that as accepted answer.

Thanks once again

Re: Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej — Tue, 29 Sep 2020 15:55:52 GMT

@sbbadri Thank you fr your reply

mwdbhyat
was very helpful and kind enough to help me so much. This is resolved by changing these two values in my inputs.conf

start_from = newest
current_only = 1