Hello,
We have installed the latest version of Elastic Search on Splunk configured the inputs.conf but we are getting errors while looking into the logs.
https://splunkbase.splunk.com/app/4175/
inputs.conf
[elasticsearch_json://esearch]
date_field_name = timestamp
elasticsearch_indice = eh
elasticsearch_instance_url = http://eshost
greater_or_equal = 2019-01-01
index = es_edr
interval = 60
lower_or_equal = now
port = 9200
use_ssl = False
verify_certs = False
Error
2019-10-15 16:09:24,394 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:29,526 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:30,862 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:32,089 INFO pid=4988 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2019-10-15 16:09:32,099 ERROR pid=4988 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/home/splunk/etc/apps/TA-elasticsearch/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/home/splunk/etc/apps/TA-elasticsearch/bin/elasticsearch_json.py", line 104, in collect_events
input_module.collect_events(self, ew)
File "/home/splunk/etc/apps/TA-elasticsearch/bin/input_module_elasticsearch_json.py", line 49, in collect_events
opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'
Also i would like to ask if we can disabled the authentication since my Elastic Search does not need authentication.
Thank you.
Having almost the exact same issue with Data Integrator errors, any insight would be great.
2020-01-09 11:56:54,144 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:55,714 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:57,708 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:59,852 INFO pid=30826 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2020-01-09 11:56:59,852 ERROR pid=30826 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 49, in collect_events
opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'
Seeing this in the splunkd.log... Possibly a timestamp issue?
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.799 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" ERROR'NoneType' object has no attribute 'strip'
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" Traceback (most recent call last):
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" self.collect_events(ew)
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" input_module.collect_events(self, ew)
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 49, in collect_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" opt_ca_certs_path = opt_ca_certs_path.strip()
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" AttributeError: 'NoneType' object has no attribute 'strip'
01-10-2020 09:49:36.123 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:36 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:36.123 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:36 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
We can leave user & secret blank if there is no authentication required by elasticsearch cluster.