All Apps and Add-ons

Elasticsearch Data Integrator - Modular Input Errors

farrukhahmed
Explorer

Hello,

We have installed the latest version of Elastic Search on Splunk configured the inputs.conf but we are getting errors while looking into the logs.

https://splunkbase.splunk.com/app/4175/

inputs.conf

[elasticsearch_json://esearch]
date_field_name = timestamp
elasticsearch_indice = eh
elasticsearch_instance_url = http://eshost
greater_or_equal = 2019-01-01
index = es_edr
interval = 60
lower_or_equal = now
port = 9200
use_ssl = False
verify_certs = False

Error

2019-10-15 16:09:24,394 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:29,526 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:30,862 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:32,089 INFO pid=4988 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2019-10-15 16:09:32,099 ERROR pid=4988 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/home/splunk/etc/apps/TA-elasticsearch/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/home/splunk/etc/apps/TA-elasticsearch/bin/elasticsearch_json.py", line 104, in collect_events
    input_module.collect_events(self, ew)
  File "/home/splunk/etc/apps/TA-elasticsearch/bin/input_module_elasticsearch_json.py", line 49, in collect_events
    opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'

Also i would like to ask if we can disabled the authentication since my Elastic Search does not need authentication.

Thank you.

spdenolan
New Member

Having almost the exact same issue with Data Integrator errors, any insight would be great.

2020-01-09 11:56:54,144 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:55,714 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:57,708 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:59,852 INFO pid=30826 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2020-01-09 11:56:59,852 ERROR pid=30826 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 49, in collect_events
opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'

0 Karma

spdenolan
New Member

Seeing this in the splunkd.log... Possibly a timestamp issue?

01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.799 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" ERROR'NoneType' object has no attribute 'strip'
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" Traceback (most recent call last):
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" self.collect_events(ew)
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" input_module.collect_events(self, ew)
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 49, in collect_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" opt_ca_certs_path = opt_ca_certs_path.strip()
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" AttributeError: 'NoneType' object has no attribute 'strip'
01-10-2020 09:49:36.123 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:36 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:36.123 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:36 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72

0 Karma

farrukhahmed
Explorer

We can leave user & secret blank if there is no authentication required by elasticsearch cluster.

0 Karma