All Apps and Add-ons

How to get data in whit elasticsearch-data-integrator app?

Expl
Loves-to-Learn

hi,

We need to configure the TA-elasticsearch-data-integrator---modular-input app and we receive data.
The problem is : we do receive data, but not all...

here is the app conf:

Name ALogName
Intervalle 3600
Index MyIndex
Statut Activated
Elasticsearch instance URL: MyName
Port #: MyPort
Use SSL 1
Verify Certs 1
CA Certs Path: /my/ca.pem
User: MyUser
Secret / Password: MyPassword
Elasticsearch Indice: MyIndice
Elasticsearch Date field name: @timestamp
Time Preset: 30d
Custom Source Type: json

If i use CLI, with the exact same configuration, except i use match, I receive the good datas.

curl -u "MyUser:MyPassword" -k "https://MyName:MyPort/MyIndice/_search?&scroll=1m&size=1000" -H 'Content-Type: application/json' -d'{"query": {"match": {"message": "MyMessage"}}, "sort": { "@timestamp": "desc" }}'
{"_scroll_id":"[...]","took":695,"timed_out":false,"_shards":{"total":8,"successful":8,"skipped":0,"failed":0},"hits":{"total":{"value":3,"relation":"eq"},"max_score":null,"hits":[...MyData...]

here is the logs of the app:

2021-12-06 13:29:00,073 INFO pid=26584 tid=MainThread file=base.py:log_request_success:271 | POST https://MyName:MyPort/MyIndice/_search?scroll=2m&size=1000 [status:200 request:0.870s]
2021-12-06 13:37:12,701 WARNING pid=26584 tid=MainThread file=base.py:log_request_fail:299 | POST https://MyName:MyPort/_search/scroll [status:404 request:0.076s]
2021-12-06 13:37:12,703 INFO pid=26584 tid=MainThread file=base.py:log_request_success:271 | DELETE https://MyName:MyPort/_search/scroll [status:404 request:0.002s]
2021-12-06 13:37:12,705 ERROR pid=26584 tid=MainThread file=base_modinput.py:log_error:309 | Get error when collecting events.
Traceback (most recent call last):
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/aob_py3/modinput_wrapper/base_modinput.py", line 128, in stream_events
self.collect_events(ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/elasticsearch_json.py", line 104, in collect_events
input_module.collect_events(self, ew)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/input_module_elasticsearch_json.py", line 109, in collect_events
for doc in res:
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/helpers/actions.py", line 589, in scan
body={"scroll_id": scroll_id, "scroll": scroll}, **scroll_kwargs
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/client/utils.py", line 168, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/client/__init__.py", line 1513, in scroll
"POST", "/_search/scroll", params=params, headers=headers, body=body
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/transport.py", line 415, in perform_request
raise e
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/transport.py", line 388, in perform_request
timeout=timeout,
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/connection/http_urllib3.py", line 275, in perform_request
self._raise_error(response.status, raw_data)
File "/opt/splunk/etc/apps/TA-elasticsearch-data-integrator---modular-input/bin/ta_elasticsearch_data_integrator_modular_input/elasticsearch/connection/base.py", line 331, in _raise_error
status_code, error_message, additional_info
elasticsearch.exceptions.NotFoundError: NotFoundError(404, 'search_phase_execution_exception', 'No search context found for id [9884105]')

Any help would be great, thanks!

Labels (2)
0 Karma

Brenny
Explorer

Hi, we facing the same problem.

Did you get any help ?

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...