Hello all, we have a problem that our Splunk's Elastic Search Integrator addon is using a forbidenn character inside it's Splunk index due to connecting to a "frontend" cluster. Let me explain. The problem is that they have a “frontend” cluster which uses index patterns to search between clusters. This is also the cluster which’s endpoint is connected to our Elasticsearch Data Integrator app for Splunk. The “backend” cluster is the one containing our index. So the infrastructe is like this: Cluster Backend > Cluster Frontend > Splunk’s addon Backend’s Cluster index: security-audit-XXX Frontend’s cluster index pattern: *:security-audit-* As it is stated in Elastic's documentation here, the use of colon (:) inside index is forbidden. However we are using it and index pattern. Does anybody have any suggestions how to tackle this ? ... View more

Hello all, I am trying to solve an issue with your addon for Splunk. Our client has an indice with name *:security-audit-*. Since we have a distributed environment we have heavy forwarders in cloud and indexers and searcheads in the on prem environment. The URL is resolvable via nslookup and the endpoint is giving us a correct response if we try to connect to it via our username and password. As you can see its transforming the : character into %3. Now, I am not sure if this is an actual issue, but our on-prem team is not receiving any data. I also tried using *security-audit-* which solved the errors but still no data has been recieved. [elasticsearch_json://srvadm] ca_certs_path = /opt/splunk/etc/auth/VWAG date_field_name = @timestamp elasticsearch_indice = *:security-audit-* elasticsearch_instance_url = https://redacted:9243 greater_or_equal = {{ ansible_date_time.date }} index = vw_de_aws_mlaas_apps interval = 300 lower_or_equal = now secret = {{ es_password }} use_ssl = 1 user = siem_readonly verify_certs = 0   root@fpea:/var/snap/amazon-ssm-agent/6312# cat /opt/splunk/var/log/splunk/ta_elasticsearch_data_integrator_modular_input_elasticsearch_json.log | grep security-audit 2022-10-21 11:52:38,943 WARNING pid=811606 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.032s] 2022-10-21 11:57:40,619 WARNING pid=814893 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.095s] 2022-10-21 12:02:41,357 WARNING pid=818040 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.091s] 2022-10-21 12:07:39,512 WARNING pid=820807 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.066s] 2022-10-21 12:12:46,422 WARNING pid=823706 tid=MainThread file=base.py:log_request_fail:299 | POST https://redacted:9243/*%3Asecurity-audit-*/_search?scroll=2m&size=1000 [status:403 request:0.036s]   ... View more

Hello one of our clients would like to send us data from its Kafka cluster to our AWS enviroment consisting of Heavy Forwarders. We then forward the data to on prem team which has indexers and searcheads.   Now from reading the documentation I am kinda lost what is needed by us. Until now we were integrating new log sources via the pull method using SNS + SQS since there was not such a large amount of data as it is now.   From my understanding in order to use a Kafka push method with Kafka Connect: 1. Require the client to install Kafka Connect on their cluster 2. Create a kafka topic But what steps are required on the Heavy Forwarder side ? How do we subscribe to that topic ? Do we only need a HEC collector with token in order to forward the data to on prem team ? Thank you.   ... View more