Hey! Since I was searching for this topic/solution, I'll just add what I think is the right solution for this case. To preserve the _time, host, source and sourcetype: (...) | collect index=main  output_format=hec   ------------ If this was helpful, some karma would be appreciated. ... View more

I have the same issue in version 8.1.6.  Any better solution for this issue? ... View more

Hello, do you know an SPL to help me find a list of my saved + skipped searches in ES plus the reason for the failure please? ... View more

it is recommended solution, as i remember. But it will be only one HF versus some searchheads, so if HF will down, my script will not work until HF recovered. ... View more

Splunk needs more tuning, upgrades and restarts than does syslog-ng so if you go directly to Splunk, without a buffer capability on the sending side, you will have far more data loss. You can update yslog-ng configurations with SIGHUP without a restart or data outage. You cannot do that with Splunk. Use syslog-ng. ... View more

you are right about HEC configuration if sourcetype, host and over fields are not defined in event. But if sender define this fields, he will override valued defined in inputs.conf. ... View more

May I suggest you change this concept completely? Instead of working against the clustering and updating configuration files on the individual search heads, work with the clustering by updating saved searches via the REST API You could an updated search via REST and that will work with the clustering... https://localhost:8000/en-US/debug/refresh?entity=saved/searches Might also work, I'm unsure if that is officially supported or not, note that you would have to update all search heads at the same time and then trigger something like that, and I'm not sure it would work as expected...but it's something you could try ... View more

Two ways to solve: option 1- Let the forwarder create multi-line. Use "transaction" keyword to group similar based on common field you prepare for. Option2- try using trail method for the log. Let rsync write to file.log, but use another crontab to send data to another file - rysnclistner.log. Now listen to this log file. Reason: 1. Changes the frequency of reading data for fowarder 2. You can use it in form of Queue and data purges as new entries keeps come. (FIFO) Hope any of the option can help. But most important, we also would like to see some sample _raw to be able to decide. Thank You ... View more

Yes. One way is you could configure the python script to output the lookup table and put it somewhere that splunk could see it, and configure an external lookup within splunk to get it as needed or as updated. (for instance, by monitoring the landing directory for changes.) http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Configureexternallookups Here's another good source of links - https://answers.splunk.com/answers/377934/use-rest-to-create-a-lookup.html ... View more