Getting Data In

Props.conf timezone settings for Eastern? And do I need to reboot any peers?

hrithiktej
Communicator

In our Slave-Apps directory on the 2 peers/indexers we have a custom app created by the prev admin which has setting for TZ to UTC for network devices that are on UTC. Now i am adding new data source (i.e. AD security logs) using UFs on DCs and our DCs are all in EST TZ and hence i would need to list EST TZ in the props.conf.

My Questions are

1) Is this the right stanza for EST time entry
[WinEventLog://Security]
TZ = US/Eastern

I understand i will have to do this on master-apps folder on cluster master and then apply config bundle

2) Will this require a reboot of any peers ?

1 Solution

mwdbhyat
Builder

Hi there,

1 - Yes thats correct

2 - Yes, the cluster master will initiate a restart of its cluster members once you apply the new cluster bundle. Please see here for what requires a restart and what doesnt..

http://docs.splunk.com/Documentation/Splunk/6.6.3/Indexer/Updatepeerconfigurations#Restart_or_reload...

View solution in original post

sbbadri
Motivator

@hrithiktej

props.conf seems correct.

[host::yourhostdefinition]
TZ = US/Eastern

or

[source::yousourcedefinition]
TZ = US/Eastern

or
[yoursourcetypedefinition]
TZ = US/Eastern

Can you please check path of the props.conf and check it has enough permission for splunk to read.

CM - $SPLUNK_HOME$/etc/master-apps/_cluster/local/props.conf --- use this location if you don't have separate app.
CM - $SPLUNK_HOME$/etc/master-apps/your app/local/props.conf -- use this location if you have separate app for it.

hrithiktej
Communicator

@sbbadri Thank you fr your reply

mwdbhyat
was very helpful and kind enough to help me so much. This is resolved by changing these two values in my inputs.conf

start_from = newest
current_only = 1

mwdbhyat
Builder

Hi there,

1 - Yes thats correct

2 - Yes, the cluster master will initiate a restart of its cluster members once you apply the new cluster bundle. Please see here for what requires a restart and what doesnt..

http://docs.splunk.com/Documentation/Splunk/6.6.3/Indexer/Updatepeerconfigurations#Restart_or_reload...

hrithiktej
Communicator

This is my props.conf on indexers

[cisco:asa]
TZ = UTC

[cisco:ise:syslog]
TZ = UTC

[cisco:acs]
TZ = UTC

[cisco:ios]
TZ = UTC

[cisco:sourcefire]
TZ = UTC

[f5:bigip:syslog]
TZ = UTC

[pan:log]
TZ = UTC

[WinEventLog://Security]
TZ = US/Eastern

[catchall:catchall]
TZ = UTC

it is not working for events that are coming for [WinEventLog://Security] because if i search for last 15mins or 60 mins i dont get results ONLY when i select last 4hours i can see results. I also tried switching my user time zone from UTC to EST through settings>users>my user timezone as EST and log out/login but still the same issue.

And I have installed Splunk_TA_windows on my UF that sits on DC

Any help will be appreciated

mwdbhyat
Builder

What happens if you dont apply a TZ ? What time are you getting for your sourcetype then?

0 Karma

hrithiktej
Communicator

@mwdbhyat

I changed these two values in inputs.conf

start_from = newest
current_only = 1

and it resolved my issue THANK YOUUUUU SO MUCH MAN! for some reason i dont see your comment here can you please paste it again I want to mark that as accepted answer.

Thanks once again

hrithiktej
Communicator

Hi, Guys, I have run into issues with UF again this time it has stopped working altogether I have started a new question for this. if you have some time please help

https://answers.splunk.com/answers/578313/splunk-winevtlog-wineventlogchanneldeletecheckpoin.html

thanks in advance

sbbadri
Motivator

@hrithiktej please vote or accept the answer if mwdbhyat solved your issue.

0 Karma

mwdbhyat
Builder

Haha no worries man !.. Dunno why my comments are disappearing!

0 Karma

hrithiktej
Communicator

Same. I did not apply a TZ before only when this issue started i realised i should enter TZ in props.conf and entering did not make a difference. I also tried entering TZ by creating a props.conf in UFs local but no joy

mwdbhyat
Builder

Can you send me an example of the search you are running and a snip of the results?

0 Karma

hrithiktej
Communicator

I am simply typing sourcetype="WinEventLog:Security" in search and i do not find anything when i do last15mins or 60 mins i can only see for last 4hrs and event time is real time like if you convert from UTC to IST (which is the TZ I live in).

Also it does not allow me here to paste an image only an url

0 Karma

mwdbhyat
Builder

Check if there is some kind of indexing lag in your environment -

source=mysource | eval delay_sec=_indextime-_time | timechart min(delay_sec) avg(delay_sec) max(delay_sec) by host

Alternatively - has that DC host been set a timezone in another app? Can you run a btool to check that?

mwdbhyat
Builder

Use this search to verify the source type, the time stamp detected (_time), the time of the user on the search head (now), and the time zone applied (date_zone)

source=mysource host=myhost | eval delay_sec=_indextime-_time | convert ctime(_indextime) AS indextime | eval now=now() | table _time indextime now date_zone source sourcetype host

0 Karma

hrithiktej
Communicator

Thanks, I ran the queries and below are the results, there seems to be a 3-4hrs delay and time diff I do not know why, if I run these queries for other sources it does not show any delay or time difference.

source=mysource | eval delay_sec=_indextime-_time | timechart min(delay_sec) avg(delay_sec) max(delay_sec) by host

_time

2017-09-25 06:45:00

avg(delay_sec)

8618

max(delay_sec)
8656

min(delay_sec)
8593

source=mysource host=myhost | eval delay_sec=_indextime-_time | convert ctime(_indextime) AS indextime | eval now=now() | table _time indextime now date_zone source sourcetype host

_time

2017-09-25 06:59:30

indextime

09/25/2017 09:16:39

source

WinEventLog:Security

Also please note to troubleshoot this now I have changed the timezone for all my Splunk servers to match with my Domain controller so now both indexer and source have same TZ = EST but still I am not able to search logs in last 60mins or 15mins.

hrithiktej
Communicator

@mwdbhyat

I see your comment in email notification but not here.

And this is my inputs I am using the default inputs.conf from the Splunk_TA_Windows app.

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
blacklist1 = EventCode="4662" Message="Object Type:(?!\s*groupPolicyContainer)"
blacklist2 = EventCode="566" Message="Object Type:(?!\s*groupPolicyContainer)"
index = wineventlog
renderXml=false

mwdbhyat
Builder

You should make changes in the local folder, just in case someone comes along and creates a stanza for the same sourcetype and overwrites your settings in default(it wont fix your problem but is a best practice).

Regarding you inputs it looks fine - but there is clearly a lag in indexing.. Can your environment handle the amount of data that is flowing into your indexers from wineventlog?

This article has a few tricks you can try:

https://docs.splunk.com/Documentation/Splunk/6.6.3/Troubleshooting/Troubleshootingeventsindexingdela...

hrithiktej
Communicator

Yeah thx man I have made it in local only and i do not see accept option for your comment How do i accept your answer?

0 Karma

mwdbhyat
Builder

You could accept the main initial answer - would sitll guide people here.. there should be an "accept as answer" option

0 Karma

mwdbhyat
Builder

Can you send a snip of your input stanza for the security logs in wineventlog ?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...