Deployment Architecture

System clock not identical in SH and indexer.

arun_kant_sharm
Path Finder

Hi Experts,

I am getting below error in Setting -> Search peers ->Distributed search in my Splunk Search heads and Splunk deployment server.

In Deployment Server:
The times on the system clocks for the machines running this search head and the intended search peer at uri=https://:8089 differ from each other.
You must make the times identical, so that time comparisons are valid across the deployment. skew_seconds=250.
The max tolerable skew may be configured in [search]:max_tolerable_skew in limits.conf on the search head. Last Connect Time:2019-09-11T06:36:57.000+00:00; Failed 11 out of 11 times.

In Search heads:
Error [00000040] Instance name "" The times on the system clocks for the machines running this search head and the intended search peer at uri=https://:8089 differ from each other.
You must make the times identical, so that time comparisons are valid across the deployment. skew_seconds=132. The max tolerable skew may be configured in [search]:max_tolerable_skew in limits.conf on the search head.
Last Connect Time:2019-09-11T06:45:08.000+00:00; Failed 8 out of 8 times.

I already updated limits.conf file on the Search Head (path: /opt/splunk/etc/system/local).

[search]
remote_timeline_fetchall = 0

Please suggest what i need to do to fix it?

0 Karma
1 Solution

arun_kant_sharm
Path Finder

I configured chrony insted of NTP, actually my all SH, indexers and deployment server are on AWS EC2 instance.
I already configured ntp.conf to configure NTP, but unfortunately its not work in my case.
Then I configured crony in all my EC2 instances, and its work properly now.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html

View solution in original post

0 Karma

arun_kant_sharm
Path Finder

I configured chrony insted of NTP, actually my all SH, indexers and deployment server are on AWS EC2 instance.
I already configured ntp.conf to configure NTP, but unfortunately its not work in my case.
Then I configured crony in all my EC2 instances, and its work properly now.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html

0 Karma

chinmoya
Communicator
0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

You need to discuss with your OS Administrator to fix time skew issue on OS, it is mostly due to NTP is not configured on your server or it is not working as expected.

I am not sure why you set remote_timeline_fetchall = 0 in limits.conf, this setting doesn't have any relation with clock skew errors in your splunk instances.

0 Karma
Get Updates on the Splunk Community!

Build Scalable Security While Moving to Cloud - Guide From Clayton Homes

 Clayton Homes faced the increased challenge of strengthening their security posture as they went through ...

Mission Control | Explore the latest release of Splunk Mission Control (2.3)

We’re happy to announce the release of Mission Control 2.3 which includes several new and exciting features ...

Cloud Platform | Migrating your Splunk Cloud deployment to Python 3.7

Python 2.7, the last release of Python 2, reached End of Life back on January 1, 2020. As part of our larger ...