Fetching logs from Elasticsearch

[Mojal]

Hi,

I have an Elastic DB that receive logs from various services directly and I want to send these logs to Splunk Enterprise.
Is there any documentation about install instruction of the Elasticsearch Data Integrator?
I couldn't  config it to make it work and I don't find any documentation on how to install and configure this add-on.

Please help me with that.@larmesto 

Kind Regards,
Mohammad

[Mojal]

Thank you for your help @marnall 

You are correct, I did enter my elastic search information in the app but it did not pull any data.

When I go thorough _Internal logs, I see some error logs that contains users like proxy and root, but I dont have any of this users in my configs nor in my database credentials and also I didnt active the proxy option in the Elasticsearch Data Integrator add-on.

I could mention that I can connect to elastic database via curl from splunk server which means the connection is open.

[canoop]

Hi @Mojal  @marnall 

I am facing the same issue with my Splunk Cluster. Were y'all able to find any workarounds/solutions?

P.S: I have deployed the splunk cluster via splunk-operator in my kubernetes environment.

[marnall]

As a test, does the app still complain when you add a filler proxy user+password combination in the settings?

There is also a different app that is often suggested for the use case of searching Elasticsearch data from Splunk. If it is not strictly necessary for you to migrate the data from Elasticsearch into Splunk, then this may be an option: https://github.com/brunotm/elasticsplunk

[Mojal]

Yes, still it does generate proxy logs even when fill fake settings.

The problem with those apps you mentioned is that they dont support authentication.

My Elasticsearch database is protected by authentication.

[marnall]

Are you able to find working values for the inputs of the app? It seems like you can enter in your Elasticsearch domain name, port, user, secret, interval, etc, then theoretically it should pull data from your elasticsearch instance.

If you enter in the values but it does not work, then you could try searching your _internal index for keywords like "elasticsearch" to see if the app generates any errors that would explain why it is not pulling data from your elasticsearch instance.