All Apps and Add-ons

Working of Elastic search Add on ... Reindexing of same data ?


Hi @gaurav_maniar @larmesto

I have enabled the modular input for getting data from the Elasticsearch into Splunk and it is working and getting the data in.

But I am not sure how the interval is suppose to work.

It is pulling in the same data again and again with everytime the input runs in the interval.

Is it supposed to work that way ?



Hi Nawaz


Please you can share python code and step config for get log ELK.

I'd like get log ELK and found issue same you.



0 Karma


Hi @nawazns5038

The Add-On is using always the same range you specify during the input creation. I found this piece of code in the app which is executed each time of input invocation to create filter for API call.

body={"query": {
    "bool": {"filter": [ 
        { "range":  { opt_date_field_name: {"gte": opt_greater_or_equal, "lte": opt_lower_or_equal}}}

There is no checkpoint mechanism. It will pull the same time range data on each invocation. If you want to increment the time range on each invocation you may have to modify the script to store epoch time in KVStore as a checkpoint and use that on next invocation for getting new data.

Hope this was helpful,

0 Karma


Hi Nawaz,
Why did you decide you yous this ELK Connector instead of e.g. standard Splunk Forwarder?
What's your use case?

Best regards.

0 Karma


I am trying to get data from the elastic search instance . I am not sure if we can install a forwarder there .


0 Karma


Thanks for your quick response!

Ok, so are satisfied with the App? Reliable? Bugs, issues?
What's the format of files where your ELK instance resides?
Plain files? Log/CSV/txt...

I'm just wondering, why do I need this connector if I can install Splunk Forwarder on my ELK server... Advantages of FW: online input, strong reliability, flexible filtering. Advantages of the Connector: no installation.

0 Karma


I am not 100% satisfied with the Addon but it is working okay after making some python changes.

The only disadvantage is it not real time , It can only work for pulling all the data once and not updating on a regular basis.

The files are in JSON format.

I cannot install the forwarder on the ELK server. I am not sure if it works the same way like Splunk like storing the data in buckets or something., I can look into installing only if the data on the ELK side is stored in the form of logs.

0 Karma


Hello @nawazns5038 ,

it's been a long time since your post but I'm facing the same problem.

It seems to me that the App can not be use to continually ingest data (give a start and "now" as end date) at given intervals. Did you find a workaround for this or got some reply from the creator? 

Thanks in advanced for sharing this info!

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...