Elasticsearch Data Integrator - Modular Input Errors

[farrukhahmed]

Hello,

We have installed the latest version of Elastic Search on Splunk configured the inputs.conf but we are getting errors while looking into the logs.

https://splunkbase.splunk.com/app/4175/

inputs.conf

[elasticsearch_json://esearch]
date_field_name = timestamp
elasticsearch_indice = eh
elasticsearch_instance_url = http://eshost
greater_or_equal = 2019-01-01
index = es_edr
interval = 60
lower_or_equal = now
port = 9200
use_ssl = False
verify_certs = False

Error

2019-10-15 16:09:24,394 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:29,526 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:30,862 INFO pid=4988 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2019-10-15 16:09:32,089 INFO pid=4988 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2019-10-15 16:09:32,099 ERROR pid=4988 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/home/splunk/etc/apps/TA-elasticsearch/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/home/splunk/etc/apps/TA-elasticsearch/bin/elasticsearch_json.py", line 104, in collect_events
    input_module.collect_events(self, ew)
  File "/home/splunk/etc/apps/TA-elasticsearch/bin/input_module_elasticsearch_json.py", line 49, in collect_events
    opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'

Also i would like to ask if we can disabled the authentication since my Elastic Search does not need authentication.

Thank you.

[a1bg503461]

Hello we are trying to implement without authentication and we have the same error, here is my stanza please adivse if there is some mistake :

[elasticsearch_json://TEST_INPUT]
cust_source_type = elastic-msgs-sms
date_field_name = timestamp
elasticsearch_indice = msgs-sms-v1.0-*
elasticsearch_instance_url = vlelasp-fe-vip.at
index = main
interval = 60
port = 9200
secret =
time_preset = 24h
user =
disabled = 0

ERROR FROM LOGS :

2025-06-10 12:21:15,012 ERROR pid=1333302 tid=MainThread file=base_modinput.py:log_error:309 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/aob_py3/modinput_wrapper/base_modinput.py", line 128, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 96, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 147, in collect_events
opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'

[PrewinThomas]

@a1bg503461 
The error highlights that, opt_ca_certs_path is not defined in your configuration

Are you using SSL/TLS with Elasticsearch? If yes
Make sure you mention your .crt path in your config
Eg:
opt_ca_certs_path = /path/to/your/ca.crt

If you are not using SSL/TLS, then try below in your config

use_ssl = 0
# opt_ca_certs_path =

Regards,
Prewin
Splunk Enthusiast | Always happy to help! If this answer helped you, please consider marking it as the solution or giving a kudos/Karma. Thanks!

[a1bg503461]

Thank you for the kind reply.

We have an elasticsearch setup without authentication and without Certificates

so I tried to comment stanza as you suggest

use_ssl = 0
# opt_ca_certs_path =

but no success 😞

Than i decided to go into the python scripts and try to comment out the cert(strip) line

/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py

#opt_ca_certs_path = opt_ca_certs_path.strip()

Now I get another SSL  Error in log :

2025-06-11 12:00:03,503 ERROR pid=2813813 tid=MainThread file=base_modinput.py:log_error:309 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/aob_py3/modinput_wrapper/base_modinput.py", line 128, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 96, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 153, in collect_events
results = search_index(opt_elasticsearch_instance_url, opt_port, opt_user, opt_secret, opt_elasticsearch_indice, opt_date_field_name, opt_time_preset, size, from_number, opt_ca_certs_path)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 102, in search_index
response = client.search(**search_params, scroll="1m")
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/_sync/client/utils.py", line 414, in wrapped
return api(*args, **kwargs)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/_sync/client/__init__.py", line 3859, in search
return self.perform_request( # type: ignore[return-value]
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/_sync/client/_base.py", line 285, in perform_request
meta, resp_body = self.transport.perform_request(
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elastic_transport/_transport.py", line 329, in perform_request
meta, raw_data = node.perform_request(
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elastic_transport/_node/_http_urllib3.py", line 199, in perform_request
raise err from None
elastic_transport.TlsError: TLS error caused by: TlsError(TLS error caused by: SSLError([SSL: UNKNOWN_PROTOCOL] unknown protocol (_ssl.c:1161)))

If anyone managed to onboard elasticsearch data without authentication or certificate validation please adivse 

[spdenolan]

Having almost the exact same issue with Data Integrator errors, any insight would be great.

2020-01-09 11:56:54,144 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:55,714 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:57,708 INFO pid=30826 tid=MainThread file=connectionpool.py:_new_conn:758 | Starting new HTTPS connection (1): 127.0.0.1
2020-01-09 11:56:59,852 INFO pid=30826 tid=MainThread file=setup_util.py:log_info:114 | Log level is not set, use default INFO
2020-01-09 11:56:59,852 ERROR pid=30826 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 49, in collect_events
opt_ca_certs_path = opt_ca_certs_path.strip()
AttributeError: 'NoneType' object has no attribute 'strip'

[spdenolan]

Seeing this in the splunkd.log... Possibly a timestamp issue?

01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.796 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:34 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:34.799 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" ERROR'NoneType' object has no attribute 'strip'
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" Traceback (most recent call last):
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/modinput_wrapper/base_modinput.py", line 127, in stream_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" self.collect_events(ew)
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" input_module.collect_events(self, ew)
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 49, in collect_events
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" opt_ca_certs_path = opt_ca_certs_path.strip()
01-10-2020 09:49:36.122 -0500 ERROR ExecProcessor - message from "/opt/splunk/bin/python2.7 /opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py" AttributeError: 'NoneType' object has no attribute 'strip'
01-10-2020 09:49:36.123 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:36 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72
01-10-2020 09:49:36.123 -0500 WARN DateParserVerbose - Failed to parse timestamp in first MAX_TIMESTAMP_LOOKAHEAD (128) characters of event. Defaulting to timestamp of previous event (Fri Jan 10 09:49:36 2020). Context: source=/opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log|host=localhost.localdomain|ta_elasticsearch_data_integrator_modular_input_elasticsearch_json-too_small|72

[farrukhahmed]

We can leave user & secret blank if there is no authentication required by elasticsearch cluster.